Good setup, but way too expensive for what adds up to a highly-advanced NAS.
I don't at all disagree with the OpenSolaris/ZFS assessment, but the hardware is a bit on the heavy side. You really don't need that server motherboard (knock 100 dollars off the price). The CPU is also overkill, and ECC RAM isn't necessary though at that price it's not exactly a bad idea either.
The remaining cost is just HDs - you can knock one off the list if you use RAID-Z instead of RAID-Z2 and remember to replace a drive as soon as it goes bad.
OpenSolaris itself doesn't need a full 320GBs for the OS and its software: an 80GB hard drive will suffice and then some.
I think ECC memory is a good idea if you can get it; turning memory errors from "random data corruption and crashes" into "harmless fmd/syslog messages" is pretty nice, and last time I looked, would add about £5 to a 4GB DDR2 system.
Of course the problem then is that Intel don't support ECC on anything but Xeons; AMD are rather better, all you need is a supporting motherboard (which includes some cheap and cheerful integrated ones), any AMD CPU in the past few years will do.
Now, if only Intel would get off their arses and release the W3xxx Xeons so I can build an ECC protected i7 system. 12GB without ECC is getting a bit ridiculous.
I'd also go for a pair of dynamically striped mirrors instead of RAID-Z2; modern disks can already pretty much saturate GigE individually, so I'd rather go for increased overall IOPS rather than increased sequential throughput at the cost of making every drive take part in each IO.
Yup, pretty much; if you have more than one vdev in a pool (mirror, RAIDZ, whatever), ZFS will stripe over them, varying what goes where depending on how fast it thinks they are.
I've heard reports of ZFS doing things like putting more data on one vdev because the disks underlying it had 16MB of cache and those in the other vdev only had 8MB; bit fancier than what you normally associate with RAID10.
For the really cheap solution, you can just get a mini-itx atom board, mini-itx case, and a few external usb enclosures for the hard drives. It's what I have at home, running opensolaris on Xen. I mapped the usb drives natively to opensolaris so I'm not creating disk images on each. It's still fast enough to stream movies on. Not counting the external hard drives, it'll probably cost you around $200 for the PC, and about $125 per terabyte (usually around $25 per usb enclosure).
One of the reasons I used Xen is because OpenSolaris still lacks proper network drivers for just about all the network cards I own. It also had issues with my external usb drives, so it was just easier getting linux to handle the hardware interfaces to opensolaris.
I use it as timemachine backups for my macs & media storage... but on a cheap budget.
I see... but unfortunately Xen dom0 has becoming unavailable in the latest Linux distributions, e.g. Fedora and Ubuntu. Also Red Hat seems that it will go with KVM in its next version.
I don't at all disagree with the OpenSolaris/ZFS assessment, but the hardware is a bit on the heavy side. You really don't need that server motherboard (knock 100 dollars off the price). The CPU is also overkill, and ECC RAM isn't necessary though at that price it's not exactly a bad idea either.
The remaining cost is just HDs - you can knock one off the list if you use RAID-Z instead of RAID-Z2 and remember to replace a drive as soon as it goes bad.
OpenSolaris itself doesn't need a full 320GBs for the OS and its software: an 80GB hard drive will suffice and then some.