From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: MIME-Version: 1.0 References: <1fa4195021526f8ef22b79c11ce5f1b39f0ebbaa.camel@pengutronix.de> In-Reply-To: From: Brian Hutchinson Date: Fri, 30 Jul 2021 10:20:32 -0400 Message-ID: Content-Type: multipart/alternative; boundary="000000000000bfd8f605c857ec2b" Subject: Re: [RAUC] Stumped, have a appfs partition that is encrypted, how to get RAUC to update it To: =?UTF-8?Q?Jan_L=C3=BCbbe?= Cc: rauc@pengutronix.de List-ID: --000000000000bfd8f605c857ec2b Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi all, update On Fri, Jul 30, 2021 at 9:33 AM Brian Hutchinson wrote: > Hi Jan, > > > On Fri, Jul 23, 2021 at 9:40 AM Jan L=C3=BCbbe wrote= : > >> Hi Brian, >> >> On Thu, 2021-07-22 at 08:55 -0400, Brian Hutchinson wrote: >> > Hi Jan, >> > >> > On Thu, Jul 22, 2021 at 8:16 AM Jan L=C3=BCbbe wr= ote: >> > > On Thu, 2021-07-22 at 08:11 -0400, Brian Hutchinson wrote: >> > > > I'm wanting to have a rootfs that is read-only SquashFS and a appf= s >> that >> > > > is encrypted. >> > > I assume you want to have a A/B appfs. >> > >> > Yes, have A/B for Kernel, dtb, rootfs and appfs. >> >> OK, as a side-node: I'd suggest storing the kernel in the rootfs, as tha= t >> gets >> rid of potential inconsistencies, avoids the need to reserve space in th= e >> kernel >> partitions and reduces the number of build artifacts to keep track of. >> >> > > How do you encrypt your appfs? dm-crypt or fscrypt? >> > So process in factory will set everything up on eMMC the first time >> with: >> > >> > cryptsetup luksFormat /dev/mmcblk2p1 & /dev/mmcblk2p2 >> > cryptsetup luksOpen /dev/mmcblk2p1 crypt_appfs1 (same thing for >> > /dev/mmcblk2p2) >> > mkfs.ext4 /dev/mapper/crypt_appfs1 & crypt_appfs2 >> >> So dm-crypt with luks header. >> >> > Then in normal use just have a script that figures out which slots we >> are >> > starting, A or B to determine with appfs partition to use and cryptset= up >> > luksOpen then mount /dev/mapper. >> >> I'd open both and only mount the active one. (see below) >> >> > > > I know a bundle can have pre and post triggers so maybe I can use >> those to >> > > > cryptsetup luksOpen the partition and then mount it and then RAUC >> can do >> > > > it's >> > > > normal thing ... but I've not researched that enough to know if >> that's the >> > > > way >> > > > to go so thought I'd ask for some guidance to point me in the righ= t >> > > > direction >> > > > first. >> >> There is the "pre-install" slot hook: >> https://rauc.readthedocs.io/en/latest/using.html#slot-hooks >> >> It's not appropriate for your use-case, though, as it's called after RAU= C >> has >> mounted the target slot, as that would be too late. >> >> > > If you use dm-crypt, you can just use the device-mapper path for the >> slot's >> > > device=3D propert in system.conf. That way, the encryption is >> transparent to >> > > rauc. >> >> > Not following how that would work since the inactive appfs would be >> > "closed/encrypted". >> >> You'd luksOpen both apps partitions during boot, before starting RAUC. >> >> From your other mail: >> > Sorry, forgot to reply-all to last message. So when I did my >> luksFormat etc., >> > I used a key-file that I created with openssl rand -base64 32 > >> > luks_appfs_key. >> >> Hmm, I thought you'd use a TPM or kernel trusted keyrings to store the >> key. >> Where do you store this key file, so it's not easily readable by the >> attacker? >> > > It's a long story, no requirement for "secure boot" only "encrypt files a= t > rest". I know, I know. I just follow schedules ;) > > >> > Are you telling me that if I add a key and put it in the rauc key ring >> in >> > /etc/rauc and in my system.conf refer to my appfs by /dev/mapper name >> rauc >> > will know what to do to "open" the inactive appfs to do the update? >> >> No. The rauc keyring is only for checking the signature on the bundle. >> >> >> > I guess I'm hung up on how the "open" will take place and how to tell >> rauc >> > about the key to use etc. >> >> rauc has no special support for any specific type of block device, as it >> just >> uses the abstraction as provided by the Linux kernel, similar to >> mkfs.ext4. >> >> So anything that can be used by i.e. ext4 can be used by rauc, you only >> have to >> setup the devices before starting rauc. This means that rauc works with >> HDDs, >> SSDs, USB-Sticks, SD-Cards, eMMCs, NVMe, RAID, LVM, >> dm-verity/-crypt/-integrity >> and anything else that's represented as a Linux block device, without >> needing >> specific code for each. >> >> In the case of block device encryption, this also avoids the need to giv= e >> rauc >> access to the key material. Having a service/script during boot be the >> only >> place where the key is handled, avoids exposing it in the rest of the >> system, >> where it could be compromised. >> >> So my suggestion is: During boot, get the key material in your >> project-specific >> way (TPM/HSM/OP-TEE/...) and use cryptsetup/dmsetup to create both >> /dev/mapper/crypt_appfs[12] and then discard the key material from >> userspace, so >> only the dm-crypt target keeps it alive. In rauc's system.conf, you set >> device=3D/dev/mapper/crypt_appfs1 and /dev/mapper/crypt_appfs2 for the a= ppfs >> slots. This way, it rauc can use them as any other block device. >> >> So here is the test I did ... that didn't work. > > I nfs booted my board. rauc thinks I've booted from slot A so it's going > to try to update slot B. > > I do: > > cryptsetup luksFormat /dev/mmcblk2p2 /boot/luks_appfs_key > cryptsetup luksOpen /dev/mmcblk2p2 crypt_appfs2 --key-file > /boot/luks_appfs_key > mkfs.ext4 /dev/mapper/crypt_appfs2 > > My /etc/rauc/system.conf looks like: > > [system] > compatible=3DMyBoard > bootloader=3Duboot > > [keyring] > path=3D/etc/rauc/ca.cert.pem > > [slot.kernel.0] > device=3D/dev/mmcblk2gp0p1 > type=3Dvfat > parent=3Drootfs.0 > > [slot.kernel.1] > device=3D/dev/mmcblk2gp1p1 > type=3Dvfat > parent=3Drootfs.1 > > [slot.rootfs.0] > device=3D/dev/mmcblk2gp1p2 > type=3Dext4 > bootname=3DA > > [slot.rootfs.1] > device=3D/dev/mmcblk2gp1p2 > type=3Dext4 > bootname=3DB > > [slot.appfs.0] > device=3D/dev/mmcblk2p1 > type=3Dext4 > parent=3Drootfs.0 > > [slot.appfs.1] > device=3D/dev/mapper/crypt_appfs2 > type=3Dext4 > parent=3Drootfs.1 > > So at this point, /dev/mapper/crypt_appfs2 is open but not mounted. > > I have my bundle scp to /tmp so I try to install it and get: > > installing > 0% Installing > 0% Determining slot states > 20% Determining slot states done. > 20% Checking bundle > 20% Verifying signature > 40% Verifying signature done. > 40% Checking bundle done. > 40% Checking manifest contents > 60% Checking manifest contents done. > 60% Determining target install group > 80% Determining target install group done. > 80% Updating slots > 80% Checking slot kernel.1 > 83% Checking slot kernel.1 done. > 83% Copying image to kernel.1 > 86% Copying image to kernel.1 done. > 86% Checking slot rootfs.1 > 90% Checking slot rootfs.1 done. > 90% Copying image to rootfs.1 > [ 1901.504350] EXT4-fs (mmcblk2gp1p2): mounted filesystem with ordered > data mode. Opts: (null) > 93% Copying image to rootfs.1 done. > [ 1927.854400] EXT4-fs (mmcblk2gp1p2): mounted filesystem with ordered > data mode. Opts: (null) > 93% Checking slot appfs.1 > 96% Checking slot appfs.1 done. > 96% Copying image to appfs.1 > 100% Copying image to appfs.1 failed. > 100% Updating slots failed. > 100% Installing failed. > LastError: Installation error: Failed updating slot appfs.1: failed to ru= n > mkfs.ext4: Child process exited with code 1 > Installing `/tmp/./update-myboard.raucb` failed > > But yet I can do mkfs.ext4 /dev/mapper/crypt_appfs2 and mount it and the > filesystem is fine. > > Looks like I'm missing something still. > > So I think my issue was because I was nfs booted. Slot A was activated but not booted. But it looks like maybe it was using slot A /etc/rauc/system.conf instead of the currently running nfs instance /etc/rauc/system.conf because what I tried before worked once I mounted /dev/mmcblk2gp0p2 and changed that /etc/rauc/system.conf to: [slot.appfs.1] device=3D/dev/mapper/crypt_appfs2 type=3Dext4 parent=3Drootfs.1 So this brings up a question. If I have boards out in the field and appfs goes from plain ext4 to encrypted, I somehow need to update the currently running /etc/rauc/system.conf file first before performing an update??? How to handle system.conf changes? Regards, Brian --000000000000bfd8f605c857ec2b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi all, update

On Fr= i, Jul 30, 2021 at 9:33 AM Brian Hutchinson <b.hutchman@gmail.com> wrote:
Hi = Jan,


On Fri, Jul 23, 2021 at 9:40 AM Jan L=C3=BCbbe <= ;jlu@pengutronix.de= > wrote:
= Hi Brian,

On Thu, 2021-07-22 at 08:55 -0400, Brian Hutchinson wrote:
> Hi Jan,
>
> On Thu, Jul 22, 2021 at 8:16 AM Jan L=C3=BCbbe <jlu@pengutronix.de> wrote:
> > On Thu, 2021-07-22 at 08:11 -0400, Brian Hutchinson wrote:
> > > I'm wanting to have a rootfs that is read-only SquashFS = and a appfs that
> > > is encrypted.
> > I assume you want to have a A/B appfs.
>
> Yes, have A/B for Kernel, dtb, rootfs and appfs.

OK, as a side-node: I'd suggest storing the kernel in the rootfs, as th= at gets
rid of potential inconsistencies, avoids the need to reserve space in the k= ernel
partitions and reduces the number of build artifacts to keep track of.

> > How do you encrypt your appfs? dm-crypt or fscrypt?
> So process in factory will set everything up on eMMC the first time wi= th:
>
> cryptsetup luksFormat /dev/mmcblk2p1 & /dev/mmcblk2p2
> cryptsetup luksOpen /dev/mmcblk2p1 crypt_appfs1 (same thing for
> /dev/mmcblk2p2)
> mkfs.ext4 /dev/mapper/crypt_appfs1 & crypt_appfs2

So dm-crypt with luks header.

> Then in normal use just have a script that figures out which slots we = are
> starting, A or B to determine with appfs partition to use and cryptset= up
> luksOpen then mount /dev/mapper.

I'd open both and only mount the active one. (see below)

> > > I know a bundle can have pre and post triggers so maybe I ca= n use those to
> > > cryptsetup luksOpen the partition and then mount it and then= RAUC can do
> > > it's
> > > normal thing ... but I've not researched that enough to = know if that's the
> > > way
> > > to go so thought I'd ask for some guidance to point me i= n the right
> > > direction
> > > first.

There is the "pre-install" slot hook:
https://rauc.readthedocs.io/en/latest/usi= ng.html#slot-hooks

It's not appropriate for your use-case, though, as it's called afte= r RAUC has
mounted the target slot, as that would be too late.

> > If you use dm-crypt, you can just use the device-mapper path for = the slot's
> > device=3D propert in system.conf. That way, the encryption is tra= nsparent to
> > rauc.

> Not following how that would work since the inactive appfs would be > "closed/encrypted".

You'd luksOpen both apps partitions during boot, before starting RAUC.= =C2=A0

>From your other mail:
> Sorry, forgot to reply-all to last message.=C2=A0 So when I did my luk= sFormat etc.,
> I used a key-file that I created with=C2=A0openssl rand -base64 32 >= ;
> luks_appfs_key.=C2=A0

Hmm, I thought you'd use a TPM or kernel trusted keyrings to store the = key.
Where do you store this key file, so it's not easily readable by the at= tacker?

It's a long story, no requi= rement for "secure boot" only "encrypt files at rest".= =C2=A0 I know, I know.=C2=A0 I just follow schedules ;)


> Are you telling me that if I add a key and put it in the rauc key ring= in
> /etc/rauc and in my system.conf refer to my appfs by /dev/mapper name = rauc
> will know what to do to "open" the inactive appfs to do the = update?

No. The rauc keyring is only for checking the signature on the bundle.


> I guess I'm hung up on how the "open" will take place an= d how to tell rauc
> about the key to use etc.=C2=A0=C2=A0

rauc has no special support for any specific type of block device, as it ju= st
uses the abstraction as provided by the Linux kernel, similar to mkfs.ext4.=

So anything that can be used by i.e. ext4 can be used by rauc, you only hav= e to
setup the devices before starting rauc. This means that rauc works with HDD= s,
SSDs, USB-Sticks, SD-Cards, eMMCs, NVMe, RAID, LVM, dm-verity/-crypt/-integ= rity
and anything else that's represented as a Linux block device, without n= eeding
specific code for each.

In the case of block device encryption, this also avoids the need to give r= auc
access to the key material. Having a service/script during boot be the only=
place where the key is handled, avoids exposing it in the rest of the syste= m,
where it could be compromised.

So my suggestion is: During boot, get the key material in your project-spec= ific
way (TPM/HSM/OP-TEE/...) and use cryptsetup/dmsetup to create both
/dev/mapper/crypt_appfs[12] and then discard the key material from userspac= e, so
only the dm-crypt target keeps it alive. In rauc's system.conf, you set=
device=3D/dev/mapper/crypt_appfs1 and /dev/mapper/crypt_appfs2 for the appf= s
slots. This way, it rauc can use them as any other block device.

So here is the test I did ... that didn't work.

I nfs booted my board.=C2=A0 rauc thinks I've b= ooted from slot A so it's going to try to update slot B.

=
I do:

cry= ptsetup luksFormat /dev/mmcblk2p2 /boot/luks_appfs_key
cryptsetup luksOpen /dev/mmcblk2p2 cr= ypt_appfs2 --key-file /boot/luks_appfs_key
mkfs.ext4 /dev/mapper/c= rypt_appfs2

My /etc/rauc= /system.conf looks like:

<= /span>
[system]
compatible=3DMyBoard
bootloader=3Duboot

[keyring]
path=3D/etc/rauc/ca.cert.pem
=C2=A0
[slot.kernel.0]
device=3D/dev/mmcblk2gp0p1
type=3Dvfat
parent=3Drootfs.0

[slot.kernel.1]
device=3D/dev/mmcblk2gp1p1
type=3Dvfat
parent=3Drootfs.1

[slot.rootfs.0]
device=3D/dev/mmcblk2gp1p2
type=3Dext4
bootname=3DA

[slot.rootfs.1]
device=3D/dev/mmcblk2gp1p2
type=3Dext4
bootname=3DB

[slot.appfs.0]
device=3D/dev/mmcblk2p1
type=3Dext4
parent=3Drootfs.0

[slot.appfs.1]
device=3D/dev/mapper/crypt_appfs2
type=3Dext4
parent=3Drootfs.1

=
So at this point, /dev/mapper/crypt_appfs2 is open but not mounted.

I have my bundle scp = to /tmp so I try to install it and get:

installing
=C2=A00% Installing
=C2=A00% Determining slot states
20% Determining slot states done.
20% Checking bundle
20% Verifying signature
40% Verifying signature done.
40% Checking bundle done.
40% Checking manifest contents
60% Checking manifest contents done.
60% Determining target install group
80% Determining target install group done.
80% Updating slots
80% Checking slot kernel.1
83% Checking slot kernel.1 done.
83% Copying image to kernel.1
86% Copying image to kernel.1 done.
86% Checking slot rootfs.1
90% Checking slot rootfs.1 done.
90% Copying image to rootfs.1
[ 1901.504350] EXT4-fs (mmcblk2gp1p2): mounted filesystem with ordered = data mode. Opts: (null)
93% Copying image to rootfs.1 done.
[ 1927.854400] EXT4-fs (mmcblk2gp1p2): mounted filesystem with ordered = data mode. Opts: (null)
93% Checking slot appfs.1
96% Checking slot appfs.1 done.
96% Copying image to appfs.1
100% Copying image to appfs.1 failed.
100% Updating slots failed.
100% Installing failed.
LastError: Installation error: Failed updating slot appfs.1: failed to = run mkfs.ext4: Child process exited with code 1
Installing `/tmp/./update-myboard.raucb` failed

But yet I can do mkfs.ext4 /dev/mapper/= crypt_appfs2 and mount it and the filesystem is fine.<= /span>

= Looks like I'm missing something still.


So I think my issue was because I was nfs booted.=C2=A0 Slot A was ac= tivated but not booted. But it looks like maybe it was using slot A /etc/ra= uc/system.conf instead of the currently running nfs instance /etc/rauc/syst= em.conf because what I tried before worked once I mounted /dev/mmcblk2gp0p2= and changed that /etc/rauc/system.conf to:

[slo= t.appfs.1]
device=3D/dev/mapper/crypt_appfs2
type=3Dext4
parent=3Drootfs.1

So this brings up a question.=C2=A0 If I have boards out in the field and= appfs goes from plain ext4 to encrypted, I somehow need to update the curr= ently running /etc/rauc/system.conf file first before performing an update?= ??=C2=A0 How to handle system.conf changes?

Regards,

Brian
--000000000000bfd8f605c857ec2b--