Monthly Archives: February 2011

Marvel Yukon Ethernet Card connected to 3Com or HP switch

It’s been a long time since we had issues in group policy pushing to clients, running XP, Vista orWindows 7.Those issues were NOT found on all machines in our network/client networks. For more than one year we understood that something was wrong. Too many “usernev” (event id 1053 and 1052) error events in the event logs and generally latencies in network traffic. See more in http://www.eventid.net/display.asp?eventid=1053&eventno=1584&source=Userenv&phase=1

In the beginning we thought that this was due to issues of windows XP connected to w2k3 DCs over 1000 Mbit networks as Microsoft itself admits and released some fix ups. Anyway it was late December 2010 that we faced the same issue in a customer having a 100 Mbit network over an HP managed 100 Mbit Switch.Same issue but no gigabit….

In February the 15th 2011, we finished up some changes in our 1000Mbit network. New cat6e cabling, no intermediate switches and fully virtualized environment for all our servers. All workstations are now directly connected to our 3COM family switches. Right after this major change I personally saw eventID 1052 and 1053 in my Windows 7 Ultimate event log. I use a Sony Vaio VGN-NR21S/S which features a Marvel Yukon ethernet 10/100 Mbit card. As you may understand as an administrator I cannot stick to problems referring to solutions such as “contact your system administrator”…I should always ask myself :p????… I tried updating my drivers directly from Marvel Yukon web site….nothing. No change, still no correct group policy appliance, pings to local area network destinations timing out out of the blue – with no reason- and other network issues coming up every now and then.

We remembered that another client in our network experienced the same problem…such as the client with the 100 Mbit Lan mentioned above.

Appears to be a lucky guess. I visited the colleague’s office where we faced the this same problem, with a Realtek Chip enabled NIC in my hand. I check his nic brand….and got ya!!!! Marvel Yukon!. It is an on-board card, so I fire up the bios. We disable the on-board nic and put another PCI 10/100/1000 Mbit NIC that I had with me. Worked like a charm!!!

What about me now…I have a laptop, not a workstation….I put a smaller uplinked switch to the 3COM and immediately all problems were solved… Apparently, prior to the change in our cablings all IT Workstations were on a smaller switch uplinked to our 3COM switch family. So till that time I had not faced a problem like this. The colleague having the issue was directly connected to the master 3COM switches.

We made a crazy thought…what about the client with the HP switch? Bingo…2 servers and 3 clients having Marvel Yukon onboard cards…but this time on an HP 100 Mbit switch…We changed all 5 NICs and everything worked better than expected. Faster downloads, faster copying to NAS, successful group policy appliances and so on.

Are we missing a point here? Either it is a problem of Marvel Yukon itself, either it is combination problem resulted from Marvel Yukon connections to 3COM or HP business series switches. Actually HP has acquired 3COM a couple of years ago and I guess HP just puts its brand on switches rather than producing them itself.

I don’t know the solution to the problem since no network drivers changes, or switches firmware upgrades solved the problem.

At the time I cannot spend more time researching this, since I am into deep with some projects but I just know….DON’T USE MARVEL YUKON NICS WITH HP OR 3COM SWITCHES.

….once again loosing our hair in the name of technology everyday….fortunately I have a lot left and a lot of years to serve my job.

Good luck!

PS. If anyone has an idea why this happens, is more than welcome to post below 🙂

Special Thanks to PA

Creativepeople.gr

Advertisements

CBL blacklists my Datacenter IPs every day for almost 3 months.

There are many times in this job, that I managed not to cry….There are several others that I said….I quit, I can’t stand it no more….but it’s the IT virus inside us that makes us keep on doing this job with a smile of self-fulfillment on every success.
This particular problem started early December 2010 and we were over it till the 21st of February 2011. We have a quite complicated set of Datacenters, one for inhouse operations and another one stated in another territory outside our headquarters just for services. The public IPs on our services datacenter started being blacklisted in CBL (http://cbl.abuseat.org/) on December 15th 2010.
Thanks to great Blacklist check tool http://www.mxtoolbox.com/ we were able to monitor changes made to listings and walk us through our checking procedures.
Based on our application services that are: email hosting, DNS hosting, Apache Web Hosting, IIS Web Hosting, VOIP, all behind an array of ISA2004 servers, we started checking what is wrong.
We checked all customers served in-front and servers/clients behind the Firewall set for malware, virus and rootkit like apps that sent out spam. No results capable of producing this listing were found. I should state that every time we saw ourselves listed we went in the process of looking at another issue and delisted our selves accordingly in CBL. We got listed and requested delisting more than 50 times during this period, and as you may know there was the threat of inability to re-delist after so many times. We forced user password changes, stopped services that we didn’t need to, disabled ftps, disabled contact forms on all our websites and waited for google to expire its cache on them, checked robots for all sites –one by one, double-checked our DNS spf records on all domains hosted, checked EVERYTHING!!! Some of the servers where rebuilt in just a few hours, routers where changed, hell knows what we did not test.
It took us that long cause we had to make changes, check logs and Mxtoolbox and wait. We finally found out that every one or two days after the delisting we were relisted in CBL AGAIN! Just image how frustrating this was.
Actually if RTFM (Read the f@@@@ing manual) worked better, as we ITs tend to not read, we would have faster understood that all this was JUST a matter of a proxy. CBL clearly states this in its FAQ, however I believe they should somehow let you contact them and ask for a log stating the particular incidents occurring and resulting the listing, just in order to understand about timings between your checks and compare them with your own logs…No contact link is still available on their website, and every mail submitting I tried did not reach anyone (My mails where from other IPs, not the ones blasklisted, therefore should be accepted).
In order to make a long story short. The actual problem was quite similar to another guy’s post in the following url
This God sent guy refers to a continuous CBL listing due to lunix server he had. This gave me the idea to start looking for an open proxy in our Elastix servers. Bingo!!! There was one elastix PBX server left as a virtual machine on our HYPERV with 2 nics (one on a standalone intranet and one on the extranet). Since we had not done any modifications to it, and no ports where addressed to it through firewall and routers and it was just for testing purposes we never installed a firewall on it! HOW STUPID of mine. Note that all the rest production servers were fully protected during this period, but not THIS ONE! Apparently even without any port publishing on him the little bastard managed to open a web proxy….and that was just the start. So we thought that some clever spammers started using him for their purposes resulting the CBL blacklisting. So we left him as it was for a couple of days while we checked apache logs and my mail logs (mail.err, mail.info, mail.log). Nothing AGAIN, but still relisting!!!
I was frustrated once again, but somehow convinced that all this came from this.
Took the vhd to labs and started checking again, and again. During another installation, i forgot to give a name to the server, so the hostname was:  myprovider.com instead of the correct one.
So I Edited /etc/postfix/main.cf, and add/edit this line:
relayhost = my.server.comThen restart the postfix service:
# /etc/init.d/postfix restart
After stopping the Virtual PC –voila…not blacklisted in CBL!!!
What does not make sense to me is that CBL says it does not count over phishing. According to them, they use their own spam traps to hunt down spam sending IPs. So if it was not that, what was it? I still have the vhd in my labs and try to understand what is wrong with it by using the netstat -tap command. I guess I will know soon, if this small testing environment was hacked, but if it was hacked –how???? We never published it. It’s only connection to the web was to get updates through the FreePbx environment, nothing else!!! I will not stop this virtual pc unless I find out why….
The good outcome of this is that it made us study really hard about processes, procedures and email spam checks that in another case we would have not….Knowledge is power! So next time I will read more carefully what FAQs say, the problem is that you always need time and it is always against you.
Special Thanks to Peter Antakis.
%d bloggers like this: