mdadm reports arrays with two different names

Details

  • Type: Bug Bug
  • Status: Open Open
  • Priority: Unknown Unknown
  • Resolution: Unresolved
  • Affects Version/s: 0.10.2
  • Fix Version/s: None
  • Component/s: None
  • Labels:
    None
  • Environment:

    Ubuntu 11.04

Description

(really this is a bug in mdadm i think):
root@monitoring-prod-i-e980049f:~# mdadm --detail --scan
ARRAY /dev/md/0 metadata=1.2 name=monitoring-prod-i-e980049f:0 UUID=645523eb:c90aad2a:bf22135c:bcd0c5ad
root@monitoring-prod-i-e980049f:~# mount |grep md
/dev/md0 on /srv type ext4 (rw,user_xattr,acl)
root@monitoring-prod-i-e980049f:~# cat /proc/mdstat |grep md
md0 : active raid1 xvdf[1] xvdg[0]

Now, note the additional / in the output from scan.
Given:

mdadm "/dev/md0" do
  devices ["/dev/xvdf","/dev/xvdg"]
  action [:create, :assemble ]
  level ebs["level"] || 1
  chunk 64
end
mount "/srv" do
  device "/dev/md0"
  fstype "ext4"
  options "rw,user_xattr,acl"
  action [:mount,:enable]
end

results in:

---- Begin output of yes | mdadm --create /dev/md0 --chunk=64 --level 1 --raid-devices 2 /dev/xvdg /dev/xvdf ----
STDOUT: 
STDERR: mdadm: device /dev/xvdg not suitable for any style of array
---- End output of yes | mdadm --create /dev/md0 --chunk=64 --level 1 --raid-devices 2 /dev/xvdg /dev/xvdf ----

Changing out "/dev/md0" for "/dev/md/0" results in:

---- Begin output of mount -t ext4 -o rw,user_xattr,acl /dev/md/0 /srv ----
STDOUT: 
STDERR: mount: /dev/md0 already mounted or /srv busy
mount: according to mtab, /dev/md0 is already mounted on /srv
---- End output of mount -t ext4 -o rw,user_xattr,acl /dev/md/0 /srv ----

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Miah Johnson added a comment -

Specifying write-intent bitmaps should clear up this issue. We basically need to add '-b internal' to action_create.

-Miah

Show
Miah Johnson added a comment - Specifying write-intent bitmaps should clear up this issue. We basically need to add '-b internal' to action_create. -Miah
Hide
Permalink
Mike Fiedler added a comment -

I don't know how `write-intent` affects things, but I've found that despite the mdadm resource creating the array, it doesn't persist it correctly (aka mdadm.conf).
Upon reboot, mdadm scans the partitions, and since it sees an MD superblock, it mounts that as a single array with the wrong name.

Show
Mike Fiedler added a comment - I don't know how `write-intent` affects things, but I've found that despite the mdadm resource creating the array, it doesn't persist it correctly (aka mdadm.conf). Upon reboot, mdadm scans the partitions, and since it sees an MD superblock, it mounts that as a single array with the wrong name.
Hide
Permalink
John E. Vincent added a comment -

This is an ubuntu-specific issue as far as I can tell. It started with 11.x series and bit me hard in the ass on the round of AWS reboots.

You can read more about it here:

http://ubuntuforums.org/showthread.php?t=1763854

Essentially ubuntu changed the naming scheme for autodetected arrays.

For your own safety, on ubuntu 11.x and higher you should custom create the mdadm.conf from a template and explicitly define the array name (as well as a DEVICE line). Additionally, if you're mounting the volume at boot I would highly advise adding 'nobootwait,_netdev' options to the fstab entry for it. On AWS, if you reboot with the fstab entry without those options, you'll lose the instance because it's waiting for keyboard input.

Show
John E. Vincent added a comment - This is an ubuntu-specific issue as far as I can tell. It started with 11.x series and bit me hard in the ass on the round of AWS reboots. You can read more about it here: http://ubuntuforums.org/showthread.php?t=1763854 Essentially ubuntu changed the naming scheme for autodetected arrays. For your own safety, on ubuntu 11.x and higher you should custom create the mdadm.conf from a template and explicitly define the array name (as well as a DEVICE line). Additionally, if you're mounting the volume at boot I would highly advise adding 'nobootwait,_netdev' options to the fstab entry for it. On AWS, if you reboot with the fstab entry without those options, you'll lose the instance because it's waiting for keyboard input.
Hide
Permalink
John E. Vincent added a comment -

I also forgot to add, you'll need to run `update-initramfs` after changing mdadm.conf for it to persist to the ramdisk. All in all it's a pain in the ass.

Show
John E. Vincent added a comment - I also forgot to add, you'll need to run `update-initramfs` after changing mdadm.conf for it to persist to the ramdisk. All in all it's a pain in the ass.
Hide
Permalink
John E. Vincent added a comment -

Here's a working stanza that we use for our cassandra instances (m1.large) that will survive a reboot and handles all the raid stuff:

https://gist.github.com/7e2579421eb1eb009964

Show
John E. Vincent added a comment - Here's a working stanza that we use for our cassandra instances (m1.large) that will survive a reboot and handles all the raid stuff: https://gist.github.com/7e2579421eb1eb009964
Hide
Permalink
Thom May added a comment -

The best thing (best used lightly) about this is that "nobootwait" is only valid as a mount option on ubuntu (possibly debian, not tried), so we wound up with:

mount ebs["mount"] do
      device device
      fstype "ext4"
      options value_for_platform(
        "ubuntu" => { "default" => "rw,user_xattr,acl,nobootwait,nofail,_netdev"},
        "default" => "rw,user_xattr,acl,nofail"
      )
      action [:mount,:enable]
    end

Hadn't known about the need to update_initramfs; thanks for that!

Show
Thom May added a comment - The best thing (best used lightly) about this is that "nobootwait" is only valid as a mount option on ubuntu (possibly debian, not tried), so we wound up with: mount ebs[ "mount" ] do device device fstype "ext4" options value_for_platform( "ubuntu" => { " default " => "rw,user_xattr,acl,nobootwait,nofail,_netdev" }, " default " => "rw,user_xattr,acl,nofail" ) action [:mount,:enable] end Hadn't known about the need to update_initramfs; thanks for that!
Bryan McLellan [Opscode] made changes -
Field Original Value New Value
Component/s Providers [ 10005 ]
Bryan McLellan [Opscode] made changes -
Workflow jira [ 13914 ] New OSS [ 17584 ]
Bryan McLellan [Opscode] made changes -
Workflow New OSS [ 17584 ] OSS: Product Management [ 25209 ]

People

  • Assignee:
    Unassigned
    Reporter:
    Thom May
Vote (2)
Watch (2)

Dates

  • Created:
    Updated: