Training, training…and more training!

the animalAdmit it – you love training!  I love training!  We all scream for…what, that’s not right.  At any rate, I’ve been doing allot of training right now – taking allot of classes and reading up on the latest trends for IT and gaining some traction on new projects.  So, I apologize for the lack of content here.  Frankly, I’m thinking that shooting for one post per week would be awesome, but miraculous at this point (which, again I do apologize about).

So what new projects, tasks, software, process, etc, etc, etc, am I working on?  Well, it’s a mixed bag of things!  But here is a fairly decent summary:

  • Hadoop – If you haven’t heard of this….I’m not sure what to say but Google it along with big data.
  • Vagrant – Development environments on the fly – it’s a neat concept and I’m checking into it.
  • Chef/Puppet/(insert automation tool here) – these are hot, Hot, HOT right now!!!  If you are a sysadmin and you are NOT using one or both or at least learning about these tools…you are in danger of becoming obsolete!!!!
  • Virtualization – Focusing on certification goals more then anything, I’m looking into Vmware, KVM and OpenStack for right now.
  • Security – Uhm….yeah.  Anything that has to do with Cryptology, SIEM, HIDS/IDS/IPS, etc….you get the picture – and for very obvious reasons (if recent news articles aren’t enough…)

All of these tools fall under two huge primary groupings – cloud and virtualization(even though that’s a bullet up there, it’s also a primary grouping and driver).  I say this because that is the latest trend technology that has got everyone talking.  Amazon, RackSpace, OpenStack, Eucalyptus, RightScale, whatever the end goal is, it seems evident to me that the cost savings, the offsetting of responsibilities compared to old brick and mortar data centers, among other reasons, are too compelling not to be knowledgeable of these services.

So…I’m training on them.  It’s amazing to me how you can be so completely soaked in technology and still feel like a newbie when you start learning about new things like Hadoop (Hadoop will really, really make you feel like you’ve been in the dark ages when you start learning about it – also, you will definitely have at least one, “wow…where has this been” moment).  So, for right now I will continue my training march!  I’m using allot of free resources and books which I plan to compile into a list and post here.  

If you have any suggestions for resources, new technologies to look at, or just want to comment, drop a line!

Posted in News | Leave a comment

Coming soon……

I know, I know……”we’re sick of hearing this Matt, what’s up man”…..Same excuses but I’m trying to get better! I’m hoping to post some new stuff soon. Until then, stay tuned!

Posted in Notices | Leave a comment

Old Infrastructure – The daytime Soap Opera of IT


So, the wife and I were talking the other day (one of those awkward moments when the house is quiet….you know, creepy quiet) and I was asking her about some of the daytime shows she watches.  Specifically, I was curious as to what the allure was to shows with tons of drama (I won’t name names….safer that way) but somewhere along the way she asked me why I wasn’t interested in allot of drama.  “Well, because I deal with it so much I guess….”, to which she replied, “Uhm…, you deal with computers.”  Oh, how I chuckled to myself; If only she knew……And that’s why I thought up this article; to explain to folks to happen across this blog and wonder, “What the hell sorta drama do computer people deal with?  I mean, they deal with computers, right?  Computers don’t have drama.”  Yeah………………….Right.  Those folks have never been in a situation the likes of which I am about to explain; and those that have, will nod in agreement with me on a single point; we know drama.

See, I don’t know about you, but nothing is more adventurous dramatic, more dangerous, more stressful, and more…..whatever, then getting handed a environment that lacks everything, including documentation!  It’s one of life’s greatest mysteries how these things happen, and we’ve all been there:  It’s a mission critical application, running some sort of core business function (been doing so, quietly, for eons)…..and the environment is running off of what could possibly be the oldest desktop/franken-server ever…….Oh, and the previous guy, who was also the finance person and application developer, he didn’t write stuff down, just kept it in his head, no worries because, you know, it’s rock solid….until today…at 5pm….when it decided to pass on.

And now we pan the camera back to you….packing your bag, looking forward to a weekend destined to die a slow death in what will become a nightmare of puzzle solving the likes of which General Hospital and House would envy.  Because you’re the IT guy – you can fix it!

So, see…we know Drama!

What does this have to do with anything?  Well, much like those soap operas of yore, most of these problems can be solved with due diligence, planning and foresight.  The two biggest items I usually see are:

  • Documentation (I know, I know; I hate it as much as the next person, but come on, it’s a required evil and you’ll be the better for it, I promise) really does come into play here; 75% of what comes up for me in aging environments can be resolved with proper documentation (just like most problems on the soap operas can be solved with wills or life directive papers…).
  • Proper planning, building and maintenance of the environment is also crucial (Look, I know mgt wants you to run that application on a single, depreciated server that came from storage and smells like it was in service when you could smoke in a data center – and you might want to give in to it – DON’T if you can help it; remember your job is to build the best you can, not just pinch the pennies like they would want).

I promise, that much like those people on the daytime soaps, if we did a little prep work, a little more planning, and allot more documentation  we would be in better situations in general – hopefully not working a weekend because someone didn’t write down key pieces of information on a 10yr old environment.

Posted in Rant | Leave a comment

Happy New Year – We’re still here!!!

Yeah, I know…I’m late to the party but at least I showed! Still working on various projects and trying desperately to find the time for this poor, neglected blog but I promise we’ll get there. My next big thing appears to be some refresher in Cisco land so I am working on a lab setup – pretty cool. I’ll keep you posted but in the meantime, to motivate me, what big projects do you have planned this year??

Posted in Notices | Leave a comment

What’s next?





This is a post I’ve been thinking about for awhile now.  What next?  See, the problem with me, is that I get bored easily.  I get bored and I get into trouble (or tick the wife off, something like that; either way, very much bad).

So now, I’m looking at my next set of certifications.  I’ve gotten my CEH and CISSP recently, which were good, but I need to get it up.  So, the next step for me, is to go back to the platform of choice and obtain (if nothing else, it only seems fair since many, many moons ago, I was a Windows guy and got my MCP/MCSA/MCITP) the coveted RHCSA/RHCE.

I’ve been doing allot of reviews and reading about it and I think I got my chosen material/study path down and setting a cert date of January (2mos…we’ll see).

After that, who knows….plenty of other projects going so we’ll see how it goes!


Posted in News | Leave a comment

Sometimes…I say what I think….And other little mistakes along the way





Interesting title, yes?  yes, I agree

So, in reviewing some of my drafts, I was starting to write an article about teamwork… an article which I have recently decided to, continue to hold on to but probably not publish for awhile.  Frankly, the article gives a little too much into some issues I was experiencing on a team and I don’t feel the article paints team work in a fair light.  Some day, maybe, I’ll publish it…who knows.

In light of that article, I will share a few things that I think are worth saying from that article, something I believe that allot of us on teams run into that are underlying given truths that we have to accept and deal with.

Let me start off with this:  I’ve never met anyone that hasn’t had a negative experience on a team.  I’ve also never met anyone that hasn’t been on a team with someone that needed to be carried.  So, before jumping on me about this revised article, keep that in mind along with the fact that these are just opinions and observations, period.

  • Teams can be a help just as much as a hindrance
  • You will end up carrying someone….and most likely they will end up carrying you (so keep that in mind when you start complaining about carrying someone)
  • Teams are there to ensure YOU don’t have to be the one man/woman army
  • Your ego gets in the way as much as theirs does…period.  The fact that you don’t buy that….well, kinda proves it
  • Your mouth and opinions, when left unchecked, can get you into a bind; remember your brain to mouth filter
And that’s pretty much it.  That pretty much, without all of the negative undertones serves as the entire venting, ranting message I had about team work.  Bottom line, team work, when used properly and with a team that has a good mesh, works well.  I have no real point to the article but I did want to get some of it out there without being too negative.

Do you have some input?  Got some team experience advice and/or experience you want to share anonymously?  Drop me a line, I’m always interested!

Posted in News | Leave a comment

Been awhile….

Sorry all, been super busy and just haven’t had the time to write. I got a few things in the hopper that are drafts and I’ll get them out soon!

Posted in Notices | Leave a comment

FreeNAS 8…………What did they do to you???

I’ve wrote about FreeNAS before.  If it’s not clear from the previous articles, I’m a pretty big fan of the product.  That being said…………I got a bit of a rant here about the latest version, FreeNAS 8….and I don’t think anyone will like what I got to say.

So, I finally broke down and replaced my patchwork storage server with a serious rig for storage.  I didn’t go completely crazy but I did build a descent machine, based on the ATOM processor (don’t knock it, it was cheap and it’s just a file server, more on this later), 2gb ram, 2 gb nics and of course, 6TB of usable storage (wanted more but come on, drive prices are still a little high right now), plenty of storage and technology from this decade.  So I thought, okay, new machine, time for a new OS, I’ll try FreeNAS 8…..And that’s where the pain misery suffering aggravation fun started.

Now, up until this point, I’ve been using the last build of FreeNAS 7 and it’s been working fine, no issues.  But I did want to try out the new FreeNAS 8, especially since iX systems took it over and made some pretty significant changes, including:

  • Significant code rewrite (not sure to what extent but according to a interview with FLOSS weekly, it’s quite large)
  • Resource requirements (i.e. we now need a minimum of 2GB of ram to run it (though I can attest to the fact that 2GB doesn’t cut it))
  • GUI (I will say this, the interface is damn pretty)
  • Feature set (it’s been reduced in the interest of time and rewrite though they are working on a plugin system to replace the lost features)

Now, that’s allot of work in a relatively short amount of time.  I certainly commend the iX Team for what they have done so far, it looks great!  However, for me, that’s where it ends and begins to fall apart.

Now, like a I said, the machine I built is a simple atom with 2gb of ram but for FreeNAS 7x this is more then enough to run everything you need (with the exception of ZFS) but with FreeNAS 8, that’s not the case at all.  In fact, a quick scan of  of google will tell that without 4GB of ram, forgot about running ZFS, let alone FreeNAS 8, it’s just not worth it.  Now, there are some mixed reviews on that; I’ve read some sites that say they got it running with 2GB just fine but they are doing minimal IO loads as well, so take it for what it’s worth.

The second issue is stability.  Now, I grabbed FreeNAS-8.0.4-RELEASE-p2 from the home page and gave it a try; maybe there is something funky with that version, maybe it’s the Atom proc/motherboard combo but right from the get-go, I had nothing but issues with it – the biggest of which was network configuration and drive stability. So I’ll start with the network configuration and go from there; in the interest of time, I’ll do this in a quick and dirty line method.

  • Network issues – configuring two NICs on the same subnet caused a ton of headaches including loss of gateway, loss of routing and reboots would result in only the primary nic being configured.  The work around:  configure both NICS in the GUI, reboot and DO NOT TOUCH IT AGAIN, EVER, EVER, EVER – not even to change the alias…
  • Performance – UFS+CIFS=DIRT…SLOW…TRANSFERS…when it worked….
  • Stability – It seems that allot of work went into ZFS (understandably) but some of us are interested in a easier file system.  That being said, UFS+CIFS = Kernel panic.  Large transfers (10GB or more) seemed to take forever and most of the time, the box would hang, freeze with a kernel panic and reboot.  When it rebooted, the file system had to be manually checked and it ALWAYS found issues

So, after two days of fighting with it, I started troubleshooting the hardware.  Keep in mind that before I put this into production (as with any system) I did a thorough burn in for about 48hrs including cpu stress testing, memory testing and drive tests.  No issues, nothing, not a hiccup.  So I went through and redid the tests again, with different software just to be sure….aaannnddd again, no issues.  Okay, that’s odd….hmmm.  I know, I’ll slap good ole’ FreeNAS 7 in there and see how that performs.


So, what does that mean, exactly?  Well, it could be a couple of things:

My new NAS sucks – could be the case, again, it’s just a file server, not a powerhouse and I built it as such.


FreeNAS 8 sucks….might not be quite there yet – Could also be the case….Not too sure on this but I’ll elaborate.

For me, I’m going with the second option for a variety of reasons.  I’ll give a few of the major points and we’ll stick with those for this discussion.

  • 4GB+ram – Really, this is a big beef item for me.  Sorry but I don’t care how cheap RAM is, this is a NAS box….Not a memory box!  Why should I need 4GB of ram?  For ZFS?  Fine, I understand that…barely but I get it (IMHO, ZFS needs more work; just google “zfs memory hog” and read a few entries…no one is happy about it).  But since I wasn’t using ZFS, why do I need more then say 512mb?  Answer:  No clue and on the FreeNAS wiki, there is no answer to this (; they simply focus on ZFS…
  • Network Configuration – I got nothing on this one.  Seems like a bug but I couldn’t find anything definitive on this.
  • UFS performance – Even using NFS, the performance was crap…..Again, nothing in the bug reports but you know how it goes….
  • UFS + CIFS = Crap – This is where the real fun started happening; random crashes, horrible performance, lots of timeouts…….  I did find a thread related to this, sorta, and the attitude is typical of people protecting a project, not helping or fixing problems (side note:  Folks, if you want people to take the open source community seriously, let’s try to be serious and not act like children…./side note)

So, that leads me to where I am today: with FreeNAS 7, last build before the iX takeover.

How’s it running?  Perfect, no issues whatsoever.  I’m running my music off it, sharing files, running all backups too it (windows/unix/linux based) and I’ve had ZERO issues.  No crashes, no reboots, no hung states, nothing.  The box is fast (all over gig traffic) and has had no hiccups in read/write performance.

So let me bring this write up to a wrap and get my final thoughts on this down.  Again, IMHO, FreeNAS 8 needs some work.  Obviously, they are trying to go after a niche market in the small to medium size sector.  I understand that completely and they are, of course, offering commercial support in addition to hardware to seal the deal.  Again, I understand.  I guess what ticks me off if that for users like me that don’t need a powerhouse file server and enjoyed the ease and light weight of FreeNAS 7, it feels like we were tossed aside by iX in favor of a pretty GUI, a heavier investment in ZFS and a drive for sales.  Again, they are a business and need to make money so I understand but it doesn’t make it any less true that the overhaul of FreeNAS 7 to 8 was geared strictly at commercial sales and had little to do with the community that helped make FreeNAS the popular product it is.

All too often this happens when a open source tool like this takes off and it’s sad really.  iX…. Make your money so you can stay in business, but try not to forget the roots of the product you are making money off of…

Posted in Rant, Storage | Tagged , , , , | Leave a comment

Java – Here’s another product….

Java is a product in desperate need of stability and identity, IMHO. It’s a wonderful product, don’t get me wrong….I just don’t know too many IT people that get excited when talking about Java….or who bring it up without cursing it’s name in light of a problem that they are experiencing in direct relation to Java.  I’m not saying every problem…..but you get the idea.

Arnold saying what we're all thinking

Preach it Arnold, preach it!

Java, for those of you fresh in IT, just born (welcome, I say!) or if you have been hiding under a rock, is a platform independent language.  Java gives us javascript (those neat web apps we see), it gives us applications (remote control software on remote control cards LOVE Java) and it gives Oracle heartburn (google for a little more information; I won’t be getting into that mess…).  “Java works everywhere” is a statement comparable to “Air is everywhere” and therein lies the problem.  Or rather, maybe just a issue of time; see the world has had time to adapt to air in it’s various forms and while java, like air, is everywhere, that doesn’t imply that it works well everywhere…..

Case in point; next time you are board, do a quick google search for java sucks.  Ignore the hating (because haters will hate as they say) and really dig into the mailing lists, trouble shooting boards, etc…you will find no shortage of issues.  It’s disappointing when you think about it.  I mean, given the size of Java (in terms of usage and installation), it’s hard to not see why there are so many inherit issues (memory management, broken versions of the JRE, etc…) and on the flip side, you have to wonder that with so many problems, why would anyone still use java??

Einstein has a point....

And how about learning Java to help improve it and maybe fix some of those issues that are plaguing you?  Well……sure, I could go and learn to be a Java Zen Master…………..

But I think the picture to the right really sums up that route quite well……….Not saying I suppose Ruby/Rails, just saying that if it takes that many books to learn something, I might not have the required time and dedication…

So, what can we do to help Java along?  I’m not sure; this is a rant folks, not a real problem solving post but a rant.  My gut tells me that Java needs a more reasonable standardizing solution and control method.  I.E. I shouldn’t have to run multiple versions of the JRE in order to access different remote control cards (yes, that’s the problem that triggered this rant….not completely, it’s been brewing, just the straw that broke the camel’s back if you will) or run different java based apps.  And for the sake of my sanity, please…PLEASE cleanup the damn dumps for Java; they ALWAYS look like crap!  I just need to know the line number and the offender class/function, not all of that other “fatal exception” crap.  Finally, if you want some help from the community maintaining the free Java based products and/or working on them, I’m sure there are plenty that would be willing to…..but come on folks, let’s make it a little easier!  I can buy one or two books for Python, Perl, Ruby/Rails, etc….and figure it out fairly easily; shouldn’t Java be the same?  Since it’s independent of the platform and I’m coding to the JRE, shouldn’t it be direct?  Again, I am sure I am oversimplifying this issue but still, I refer to the picture up above and ask you, doesn’t that seem a little excessive?

At any rate, this rant has come to an end.  I just wanted to vent a little about Java and see what other thought.  Am I wrong?  Am I right?  Do I have a third head?  Let me know!

Posted in Rant | Tagged , , | Leave a comment

Patch process – wha…what process?

Okay, so another big project that I’ve been working on is patch automation (along with sudo streamlining but that’s another nightmare, errr….I mean project) and I’ve got to say that patching, in and of itself is not a bad process. But when you put more then 100 servers in the room, tie in 3 to 5 teams that need to be involved, then it becomes a issue.

So, how do we get over that hump? Good question. When I figure it out, I’ll give you a shout……

Posted in Rant | Tagged | Leave a comment