Accidentally all cooling

What happens when you have 32GB of FB-DIMM and you compile a kernel .. with two thirds of the cooling power removed? You know that burning electronics smell? That’s what happens bigsmile

But like a miracle, everything seems to still work. And big disappointment of this old DL140 G3 because it only flashed a yellow led intermittently, which had a power symbol and I let it slide as I figured it is running near limit its power capabilities. But no, it was indicating that there was no power for the fans! It should have shut itself down immediately. Like even older ProLiant servers do if you try to run them without proper cooling.

I of course tried to take power from CD-ROM and use it for additional SSD instead, but they had done pretty interesting power routing decisions and the cooling used the power provided by that same molex.

Lesson learned: never dismiss warning lights. And I actually, with my hand, tried if there was air moving, and now that I think back, I certainly had a small thought that doesn’t seem too good. But apparently it wasn’t something that I figured was worth looking into. Luckily everything still works.

Just-in-time backups

[4055216.550294] ata3.00: exception Emask 0x0 SAct 0x80000 SErr 0x0 action 0x0
[4055216.575925] ata3.00: irq_stat 0x40000008
[4055216.601659] ata3.00: failed command: READ FPDMA QUEUED
[4055216.627747] ata3.00: cmd 60/00:98:a8:54:9a/01:00:bc:00:00/40 tag 19 ncq 131072 in
[4055216.627747]          res 41/40:00:f8:54:9a/00:00:bc:00:00/40 Emask 0x409 (media error) <F>
[4055216.680268] ata3.00: status: { DRDY ERR }
[4055216.706079] ata3.00: error: { UNC }
[4055216.732840] ata3.00: configured for UDMA/133
[4055216.758594] sd 2:0:0:0: [sda] tag#19 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[4055216.784648] sd 2:0:0:0: [sda] tag#19 Sense Key : Medium Error [current] [descriptor]
[4055216.811002] sd 2:0:0:0: [sda] tag#19 Add. Sense: Unrecovered read error - auto reallocate failed
[4055216.837236] sd 2:0:0:0: [sda] tag#19 CDB: Read(10) 28 00 bc 9a 54 a8 00 01 00 00
[4055216.863528] blk_update_request: I/O error, dev sda, sector 3164230904
[4055216.890243] ata3: EH complete
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       1

Not catastrophic situation but something’s wrong with the disk. Will probably get its missing pair for the mirror this weekend. Has survived without one for 18700 hours.

Got the 28 tapes now and taking good use of them as fast as I can @ 55MB/s. Nice to have a big row of tapes where you can grab one and it doesn’t get any shorter.

Seems to be that one sector that has failed and the drives tries to read recover it but has failed so far. So 512 bytes gone.

Optimizing energy consumption

After playing with the old DL385 and after installing pfSense on it to perhaps use it as a firewall, I measured how much power does it use: 290 W idle. So that plan is now buried as it would cost 200 € per year to run a firewall, which isn’t economical.

And once I was at it I began to think that even 400W year round is a lot but I cannot fit everything into one 200 W server. But I can!

The problem has been that I have wanted to use SAS drives with that 2.5″ server but I don’t necessarily need to. And since I have 8 x 2.5″ disk bays I can easily run 4 of them in SAS and have 4 bays left for SATA, which are cheaper and come in greater storage capacity.

So with that sort of setup I can easily run my whole infrastructure on a single 200W server and have at least 8TB of storage. That would be 4TB mirrored. Then if I need more I can get USB 3.0 PCIe controller and have as much data as I want.

Went on a shopping spree

It is official that I don’t need tapes no more.

I saw this on eBay earlier this week, and asked if shipping to Finland would be possible, and less than two hours before the auction was closing I got a reply that the shipping would be £20.

So then this happened:


28 tapes for total of £69 or about 77 €. That’s 0,003 € per gigabyte. An order of magnitude less than hard drives. 22TB of tape. No need to worry about space usage anymore.

And someone tried to be very sneaky and bid at the last seconds but I was luckily quick enough to clicking the highest amount available button and got all this for myself!

Also the recent drop in British Pound valuation against major currencies will give me 6% discount (already accounted for) unless it bounces back. And also trying to cancel the previous 100 € purchase of new LTO-4 tapes.

Of course it is possible that these are very old tapes and basically garbage. Or have been dumped because of that. So ideally I would maybe take random selection of these tapes and run them end-to-end multiple times to see that they last and hold the data.

Tried to find if the cartridge memory would hold any information about the usage of the tape, but could not find any specification, or program to read the data. If I was part of the standard designer team, I would try to include that sort of information, because it would make sense for businesses to know how much life their tapes have left.

Upgrading main server to dual PSU

Not currently in production but will be once it has been properly equipped.

26 € delivered:


Although I don’t know why people bother with dual PSU because my experience is these sort of quality power supplies don’t fail. But of course if you have hundreds of servers then probably the odds are some of them will fail and if it is a production server then that is a problem.

But good deal nevertheless. Next in line upgrade from 6 cores to 12 cores and from 32GB to 64GB.

More Israeli shopping

They have some good stuff for good price. Jews know how to sell and that is a compliment!


These are available for $10 and small postage fee, but still trying to negotiate some USD off of the price:



Don’t really need these, but I might put together a pfSense firewall, load balancer and a router just because it’s simple and fun. Too much advanced things going on all the time, so maybe some small project instead would be nice, for a change.

With these the router would have 8 gigabit Ethernets which is just about enough for a basic setup.

I have an old ProLiant that these would go to, and I have had similar but 32 bit machine running non-stop for the last 3 years and it has not sneezed once, so these old ProLiants are workhorses which will never seem to fail.


And the seller accepted the offer, so unless these get stuck to the customs, that was $31 well spent.


0,0127€/GB storage gained

Just when I was about to order a single tape to accompany my single other tape, I found this person:


So I removed 10 pieces of new 800GB LTO-4 tapes from his collection for 101,40 € delivered and everything paid. That’s 1,27 euro-cents per gigabyte. Or about three times less than hard drive gigabyte. So if somebody is still bashing the tapes’ ability to provide cheap and reliable long-term storage, then those people need to be educated!

Sadly I don’t generate too much data. I have a need for 8 more tapes after which all of the data that I have created since about 2000 is stored on tapes and after that there is not much data to go to tapes. But I might get another 10 from my next salary just in case I want to store 8TB of data.

And if I get my 10 then I will definitely try to flip these because the price is ridiculous. 10% profit with no troubles.

Backing up Youtube playlist

Last time I did this was probably close to three years ago so it is about time to do it again.

Because if you use Youtube yourself, you know that if you go back year or two, approximately at least 5-10% of the videos will have been deleted or made private. So backing up Youtube makes a lot of sense.

I am using the following to store everything that I can:

sudo -u nobody youtube-dl -u username -p password --no-mark-watched -4 --playlist-start 1 --playlist-end 76 --output "\"%(playlist)s\"-\"%(uploader_id)s\"-\"%(upload_date)s\"-\"%(title)s\"-\"%(id)s\"-\"%(format)s\".%(ext)s" --continue --write-description --write-info-json --write-annotations --cookies .youtube-dl.cookies --cache-dir .youtube-dl.cache --write-all-thumbnails --no-call-home --format mp4,ogg,flv --write-sub --write-auto-sub --all-subs

Downloading 76 latest videos from the playlist in mp4, ogg and flv formats (if available) along with all the other information, including auto-generated subtitles.

Related Post

Disassembly guide for HP Ultrium 1760 External

Important document if the drive ever needs a repair.


Could probably inspect the power supply and replace all the electrolytics just in case. Perhaps once I have done all the backups.


Related Post

Cataloging the tape content

So how I am going about this are couple of simple Linux commands.

First to save information about all of the files:

find -ls . >backup2.content.8.txt

Which will store the content of file 8 of backup number 2. Same process for all of the files on the tape (or logical backup, more precisely).

Then the md5 checksum of every file on the tape:

find . -type f -exec md5sum {} \; >backup2.content.8.md5

Then both of these will be stored on a archival grade CD-R discussed earlier and kept with the tapes at all times.

I will probably fill most of the whole CD-R with this and the other meta data just to be sure that if one part of the media gets damaged, then the other parts will still have this information on it.

Related Post