In order to use again the same drive in ZFS, if you are sure the disk is not faulty, doing DD to the first 10 GB of the drive is a good start, but you need to do it also at the end of the drive. But there is no need to DD 10 GB, I believe the first and last MB is enough.

I solved it this way:

dd bs=512 if=/dev/zero of=/dev/sdk count=2048 seek=$(($(blockdev --getsz /dev/sdk) - 2048))
dd bs=512 if=/dev/zero of=/dev/sdk count=2048 

Then just add the "new" disk back to the ZFS pool. No need to labelclear, scrub, wipe the disk. DD is all you need. You also should not partition the disk after DD. ZFS will not accept any previously partitioned disk.

Then just zpool replace <pool-name> <old-device> <new-device>

normally is the disk-by-id and is the device listed in lsblk.

After digging endlessly this night I finally found the solution. The short answer is that you can use the disks' GUIDs (which persist even after disconnecting a drive) with the zpool command.

Long answer: I got the disk's GUID using the zdb command which gave me the following output

root@zeus:/dev# zdb
hermes:
    version: 28
    name: 'hermes'
    state: 0
    txg: 162804
    pool_guid: 14829240649900366534
    hostname: 'zeus'
    vdev_children: 1
    vdev_tree:
        type: 'root'
        id: 0
        guid: 14829240649900366534
        children[0]:
            type: 'raidz'
            id: 0
            guid: 5355850150368902284
            nparity: 1
            metaslab_array: 31
            metaslab_shift: 32
            ashift: 9
            asize: 791588896768
            is_log: 0
            create_txg: 4
            children[0]:
                type: 'disk'
                id: 0
                guid: 11426107064765252810
                path: '/dev/disk/by-id/ata-ST3300620A_5QF0MJFP-part2'
                phys_path: '/dev/gptid/73b31683-537f-11e2-bad7-50465d4eb8b0'
                whole_disk: 1
                create_txg: 4
            children[1]:
                type: 'disk'
                id: 1
                guid: 15935140517898495532
                path: '/dev/disk/by-id/ata-ST3300831A_5NF0552X-part2'
                phys_path: '/dev/gptid/746c949a-537f-11e2-bad7-50465d4eb8b0'
                whole_disk: 1
                create_txg: 4
            children[2]:
                type: 'disk'
                id: 2
                guid: 7183706725091321492
                path: '/dev/disk/by-id/ata-ST3200822A_5LJ1CHMS-part2'
                phys_path: '/dev/gptid/7541115a-537f-11e2-bad7-50465d4eb8b0'
                whole_disk: 1
                create_txg: 4
            children[3]:
                type: 'disk'
                id: 3
                guid: 17196042497722925662
                path: '/dev/disk/by-id/ata-ST3200822A_3LJ0189C-part2'
                phys_path: '/dev/gptid/760a94ee-537f-11e2-bad7-50465d4eb8b0'
                whole_disk: 1
                create_txg: 4
    features_for_read:

The GUID I was looking for is 15935140517898495532 which enabled me to do

root@zeus:/dev# zpool offline hermes 15935140517898495532
root@zeus:/dev# zpool status
  pool: hermes
 state: DEGRADED
status: One or more devices has been taken offline by the administrator.
        Sufficient replicas exist for the pool to continue functioning in a
        degraded state.
action: Online the device using 'zpool online' or replace the device with
        'zpool replace'.
  scan: scrub repaired 0 in 2h4m with 0 errors on Sun Jun  9 00:28:24 2013
config:

        NAME                         STATE     READ WRITE CKSUM
        hermes                       DEGRADED     0     0     0
          raidz1-0                   DEGRADED     0     0     0
            ata-ST3300620A_5QF0MJFP  ONLINE       0     0     0
            ata-ST3300831A_5NF0552X  OFFLINE      0     0     0
            ata-ST3200822A_5LJ1CHMS  ONLINE       0     0     0
            ata-ST3200822A_3LJ0189C  ONLINE       0     0     0

errors: No known data errors

and then

root@zeus:/dev# zpool replace hermes 15935140517898495532 /dev/disk/by-id/ata-ST3500320AS_9QM03ATQ
root@zeus:/dev# zpool status
  pool: hermes
 state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Sun Jun  9 01:44:36 2013
    408M scanned out of 419G at 20,4M/s, 5h50m to go
    101M resilvered, 0,10% done
config:

        NAME                            STATE     READ WRITE CKSUM
        hermes                          DEGRADED     0     0     0
          raidz1-0                      DEGRADED     0     0     0
            ata-ST3300620A_5QF0MJFP     ONLINE       0     0     0
            replacing-1                 OFFLINE      0     0     0
              ata-ST3300831A_5NF0552X   OFFLINE      0     0     0
              ata-ST3500320AS_9QM03ATQ  ONLINE       0     0     0  (resilvering)
            ata-ST3200822A_5LJ1CHMS     ONLINE       0     0     0
            ata-ST3200822A_3LJ0189C     ONLINE       0     0     0

errors: No known data errors

After resilvering had been completed everything worked well again. It would have been nice to include this information, that you can use a disk's GUID obtained through zdb with the zpool command, with the manpage of zpool.

Edit

As pointed out by durval below the zdb command may not output anything. Then you may try to use

zdb -l /dev/<name-of-device>

to explicitly list information about the device (even if it already is missing from the system).

There's no difference between the options when all is said and done — drives are interchangeable. "Just add sdi as the new spare" requires fewer steps and minimizes the amount of time that you spend in a resyncing state, so it's the natural choice.

Probably have to look into the details. I haven't used whatever produces that UI. But somewhere you should be able to get the SMART values for the drive. Or more generally you have to dig into the details to figure out what is a actually wrong.

export the pool and import it with zpool import -d /dev/disk/by-id/ instead. If using an sdX name is all that was wrong, it should find the correct disk and use that.

ZFS detects disks not by their name in the filesystem, but by their UUID that is written onto the disk (or at least something similar -- not 100% sure that it's actually a UUID). When zpool import runs, the disks are enumerated, ZFS rebuilds all the pools, and then uses the device name (without actually including any directory IME, usually it's something like sda rather than /dev/sda) in the zpool status output. As such, if you move the drives around (or if the kernel moves the drives around, which can happen with modern kernels on modern hardware), zpool will still detect the disks in the same order as it did before; disks that appeared first in the output will again appear first in the output, even if the kernel doesn't enumerate them in that output anymore.

What happened to you here is probably that due to the fact that the original zpool import didn't work, the kernel could complete its boot, udev did a lot more work, and then by the time you did the manual zpool import, the default enumeration of all your disks turned out to have the serial number-based ones first, rather than the sdX-based ones. Most likely, the next time you reboot the machine, the used names will be back to the sdX scheme.

Luckily, resolving the names from one naming scheme to the other is fairly straightforward:

wouter@gangtai:/dev/disk/by-id$ ls -l
total 0
lrwxrwxrwx. 1 root root  9 Mar 31 18:15 ata-SAMSUNG_MZ7TE256HMHP-00004_S1RKNSAFC04685 -> ../../sda
lrwxrwxrwx. 1 root root 10 Mar 31 18:15 ata-SAMSUNG_MZ7TE256HMHP-00004_S1RKNSAFC04685-part1 -> ../../sda1
lrwxrwxrwx. 1 root root  9 Mar 31 18:15 wwn-0x50025388a089e89c -> ../../sda
lrwxrwxrwx. 1 root root 10 Mar 31 18:15 wwn-0x50025388a089e89c-part1 -> ../../sda1

There are multiple naming schemes (by-id, by-uuid, and by-path), all of which can be found under /dev/disk.

Having said all that, I must say I don't agree with your claim that it would be easier to figure out which disk is which by looking at the sdX names. Modern kernels no longer assign static device names to particular devices; this is why modern distributions use UUID-based fstab files, rather than sdX-based ones. The serial number, in fact, is a far more reliable way to figure out which is the broken disk; after all, it's written on the actual disk, in contrast to the sdX name, which may differ from boot to boot (I've actually encountered that on a ZFS box with sixteen hard disks). Any one of the other methods (by-uuid, by-id, and especially by-path in the enterprise-level multi-disk enclosures) is much more reliable than that.

If you have failing drives, I would not attempt to replace them one-by-one. Buy a new disk shelf, build a new pool with the new disks and copy your data over ONCE.

why don't you just use zpool replace as suggested on the screen? replace the faulty drive with the new one and let zfs create the partitions and resilver the drive? maybe I didn't get the question, sorry