[RAUC] Query regarding rauc

[RAUC] Query regarding rauc

[Abhishek Kumar Rai]

[-- Attachment #1.1: Type: text/plain, Size: 2391 bytes --]

Hi Team,


To achieve a specific requirement for us the "rauc install" command won't
work i.e due to a configuration that we have the rauc install would fail.
Though we would still like to go ahead and flashload to that partition. We
would like to know if the approach below is correct


Our understanding of rauc install suggests that it does the following
things:

   1. Verify signature of the bundle
   2. Run partition selection algorithm
   3. Extract the tar balls from the bundle
   4. Create new ext4 FS on the selected partition
   5. Write appropriate tar ball to the partition


Since "rauc install" would give us errors due to our configuration we plan
to achieve this using the steps below:

   1. Verify signature of the bundle (rauc info ...)
   2. Partition selection algorithm (we would like to skip this)
   3. Extract the tar balls from the bundle (rauc extract ..)
   4. Create new ext4 FS on the selected partition + Write appropriate tar
   ball to the partition (rauc write-slot ... ....)


Is the above way of bypassing the "rauc install" command for our
requirement correct? Any issues you feel might arise with this approach?

Regards,

Abhishek Rai

-- 














The information contained in this e-mail message (including 
any 


attachments) may be confidential, proprietary, privileged, or 
otherwise


exempt from disclosure under applicable laws. It is intended to 
be 


conveyed only to the designated recipient(s). Any use, dissemination, 



distribution, printing, retaining or copying of this e-mail (including 
its 


attachments) by unintended recipient(s) is strictly prohibited and 
may 


be unlawful. If you are not an intended recipient of this e-mail, or 
believe 


that you have received this e-mail in error, please notify the 
sender 


immediately (by replying to this e-mail), delete any and all 
copies of 


this e-mail (including any attachments) from your system, and 
do not


disclose the content of this e-mail to any other person. Thank you 
for your cooperation.






-- 
_This e-mail message (including any attachments) may be confidential, 
proprietary, privileged, or otherwise exempt from disclosure under 
applicable laws. If you are not an intended recipient, please delete this 
message. Thank you.
_

[-- Attachment #1.2: Type: text/html, Size: 4985 bytes --]

[-- Attachment #2: Type: text/plain, Size: 65 bytes --]

_______________________________________________
RAUC mailing list

Re: [RAUC] Query regarding rauc

[Enrico Joerns]

Hi Abhishek,

On 12/4/18 12:12 PM, Abhishek Kumar Rai wrote:
> Hi Team,
> 
> 
> To achieve a specific requirement for us the "rauc install" command won't work i.e due to a configuration that we have the rauc install would fail. Though we would still like to go ahead and flashload to that partition. We would like to know if the approach below is correct
> 
> 
> Our understanding of rauc install suggests that it does the following things:
> 
>  1. Verify signature of the bundle
>  2. Run partition selection algorithm
>  3. Extract the tar balls from the bundle
>  4. Create new ext4 FS on the selected partition
>  5. Write appropriate tar ball to the partition
> 
> 
> Since "rauc install" would give us errors due to our configuration we plan to achieve this using the steps below:
> 
>  1. Verify signature of the bundle (rauc info ...)
>  2. Partition selection algorithm (we would like to skip this)
>  3. Extract the tar balls from the bundle (rauc extract ..)
>  4. Create new ext4 FS on the selected partition + Write appropriate tar ball to the partition (rauc write-slot ... ....)

how do you trigger the custom installation? Do you use the full-custom RAUC install handler?

Extracting the tar's should not be required in all cases. With the full-custom handler you
will already have it mounted, otherwise you can simply mount the bundle after verification
as it is basically a squashfs.

For rauc write-slot you also rely on parts of the system.conf, but not in the fixed scheme.
If that is where your issue resides, you should be fine with that.


Could you give me a hint what forces you to do this kind of manual handling?
Maybe then we can think about a possible solution more targeted.

But basically it should work as described above, yes.
One thing will miss, depending on what you actually try to do.
The key for atomic updates is to deactivate the bootable slot you
write your image to before writing and set it as primary after successful writing.

You can achieve this behavior with

   rauc status mark-bad <slot-to-update>

   [ perform update ]

   rauc status mark-active <slot-to-update>


Best regards, Enrico

-- 
Pengutronix e.K.                           | Enrico Jörns                |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |


_______________________________________________
RAUC mailing list

Re: [RAUC] Query regarding rauc

[Abhishek Kumar Rai]

[-- Attachment #1.1: Type: text/plain, Size: 6119 bytes --]

Hi Enrico,

Thanks for the prompt response!

Please find our responses inline (blue colored)

Regards,
Abhishek Rai

On Tue, Dec 4, 2018 at 4:54 PM Enrico Joerns <ejo@pengutronix.de> wrote:

> Hi Abhishek,
>
> On 12/4/18 12:12 PM, Abhishek Kumar Rai wrote:
> > Hi Team,
> >
> >
> > To achieve a specific requirement for us the "rauc install" command
> won't work i.e due to a configuration that we have the rauc install would
> fail. Though we would still like to go ahead and flashload to that
> partition. We would like to know if the approach below is correct
> >
> >
> > Our understanding of rauc install suggests that it does the following
> things:
> >
> >  1. Verify signature of the bundle
> >  2. Run partition selection algorithm
> >  3. Extract the tar balls from the bundle
> >  4. Create new ext4 FS on the selected partition
> >  5. Write appropriate tar ball to the partition
> >
> >
> > Since "rauc install" would give us errors due to our configuration we
> plan to achieve this using the steps below:
> >
> >  1. Verify signature of the bundle (rauc info ...)
> >  2. Partition selection algorithm (we would like to skip this)
> >  3. Extract the tar balls from the bundle (rauc extract ..)
> >  4. Create new ext4 FS on the selected partition + Write appropriate tar
> ball to the partition (rauc write-slot ... ....)
>
> how do you trigger the custom installation? Do you use the full-custom
> RAUC install handler?
> No for now we have replaced the "rauc install" call with the calls to the
> commands "rauc info + rauc extract + rauc write-slot". We were thinking
> that we might not get control due to slot selection algorithm. But we will
> explore this further starting today. Get a feeling it would be what we
> want. Though we read in the documentation that the pre/post install hooks
> would not work with full custom install. We might need them. Not too sure
> though. We might be able to merge the "post-install" hook work with the
> "install" hook
>


> Extracting the tar's should not be required in all cases. With the
> full-custom handler you
> will already have it mounted, otherwise you can simply mount the bundle
> after verification
> as it is basically a squashfs.
> Sure..
>


> For rauc write-slot you also rely on parts of the system.conf, but not in
> the fixed scheme.
> If that is where your issue resides, you should be fine with that.
> Sure. We were also considering referring to the manifest file when the
> bundle gets mounted
>
> Could you give me a hint what forces you to do this kind of manual
> handling?
> Maybe then we can think about a possible solution more targeted.
>

   - For now, we have 4 partitions containing rfs.0, rfs.1, appfs.0 and
   appfs.1. We would like to have a scheme where only the partition that has
   been flashed with bundle data is  changed i.e lets say currently we are
   using rfs.0 and appfs.0. Now if we flash a bundle containing appfs, on next
   reboot, system should use rfs.0 and appfs.*1*. Similarly for rfs.


   - To achieve this we started without a parent child relationship in
   system.conf. So when we are executing from appfs.0 and rfs.0 and we try to
   flash a bundle containing appfs, it would try to write to appfs.0 which
   would already be in use and hence we would get errors. So we introduced the
   parent child relationship


   - Now *with* parent child relationship the issue above would be
   resolved. Now when we boot from "rfs.0" and execute apps from "appfs.0" and
   flash a appfs bundle it would write to appfs.1. On reboot, as per our
   scheme we would boot from "rfs.0" and execute apps from "appfs.1". Now when
   we try to do a rauc install it would write to "appfs.1" as it would
   consider appfs.0 as active partition due to parent-child relationship.


   - Hence as we were running into all these issues we felt that it would
   help our cause if we had our own slot selection logic. Hence the deviation
   to rauc info + rauc extract + rauc write-slot.


> But basically it should work as described above, yes.
> One thing will miss, depending on what you actually try to do.
> The key for atomic updates is to deactivate the bootable slot you
> write your image to before writing and set it as primary after successful
> writing.
>
> You can achieve this behavior with
>
>    rauc status mark-bad <slot-to-update>
>
>    [ perform update ]
>
>    rauc status mark-active <slot-to-update>
> Sure.. we will include this in our scheme..
>
> Best regards, Enrico
>
> --
> Pengutronix e.K.                           | Enrico Jörns                |
> Industrial Linux Solutions                 | http://www.pengutronix.de/  |
> Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
>
>

-- 














The information contained in this e-mail message (including 
any 


attachments) may be confidential, proprietary, privileged, or 
otherwise


exempt from disclosure under applicable laws. It is intended to 
be 


conveyed only to the designated recipient(s). Any use, dissemination, 



distribution, printing, retaining or copying of this e-mail (including 
its 


attachments) by unintended recipient(s) is strictly prohibited and 
may 


be unlawful. If you are not an intended recipient of this e-mail, or 
believe 


that you have received this e-mail in error, please notify the 
sender 


immediately (by replying to this e-mail), delete any and all 
copies of 


this e-mail (including any attachments) from your system, and 
do not


disclose the content of this e-mail to any other person. Thank you 
for your cooperation.






-- 
_This e-mail message (including any attachments) may be confidential, 
proprietary, privileged, or otherwise exempt from disclosure under 
applicable laws. If you are not an intended recipient, please delete this 
message. Thank you.
_

[-- Attachment #1.2: Type: text/html, Size: 8108 bytes --]

[-- Attachment #2: Type: text/plain, Size: 65 bytes --]

_______________________________________________
RAUC mailing list

Re: [RAUC] Query regarding rauc

[Enrico Joerns]

Hi Abhishek,

On 12/4/18 1:06 PM, Abhishek Kumar Rai wrote:
> Hi Enrico,
> 
> Thanks for the prompt response!
> 
> Please find our responses inline (blue colored)

you're lucky that I've a graphical mailer... ;)
Better use standard mail quoting.

>> Our understanding of rauc install suggests that it does the
>> following things:
>> 
>> 1. Verify signature of the bundle 2. Run partition selection
>> algorithm 3. Extract the tar balls from the bundle 4. Create new
>> ext4 FS on the selected partition 5. Write appropriate tar ball to
>> the partition
>> 
>> 
>> Since "rauc install" would give us errors due to our configuration
>> we plan to achieve this using the steps below:
>> 
>> 1. Verify signature of the bundle (rauc info ...) 2. Partition
>> selection algorithm (we would like to skip this) 3. Extract the tar
>> balls from the bundle (rauc extract ..) 4. Create new ext4 FS on
>> the selected partition + Write appropriate tar ball to the
>> partition (rauc write-slot ... ....)
> 
> how do you trigger the custom installation? Do you use the
> full-custom RAUC install handler? No for now we have replaced the
> "rauc install" call with the calls to the commands "rauc info + rauc
> extract + rauc write-slot". We were thinking that we might not get
> control due to slot selection algorithm. But we will explore this
> further starting today. Get a feeling it would be what we want.
> Though we read in the documentation that the pre/post install hooks
> would not work with full custom install. We might need them. Not too
> sure though. We might be able to merge the "post-install" hook work
> with the "install" hook

You want to have a look at this sections:

https://rauc.readthedocs.io/en/latest/using.html#full-custom-update

> Extracting the tar's should not be required in all cases. With the
> full-custom handler you will already have it mounted, otherwise you
> can simply mount the bundle after verification as it is basically a
> squashfs. Sure..
> 
> For rauc write-slot you also rely on parts of the system.conf, but
> not in the fixed scheme. If that is where your issue resides, you
> should be fine with that. Sure. We were also considering referring to
> the manifest file when the bundle gets mounted
> 
> Could you give me a hint what forces you to do this kind of manual
> handling? Maybe then we can think about a possible solution more
> targeted.
> 
> * For now, we have 4 partitions containing rfs.0, rfs.1, appfs.0 and
> appfs.1. We would like to have a scheme where only the partition that
> has been flashed with bundle data is  changed i.e lets say currently
> we are using rfs.0 and appfs.0. Now if we flash a bundle containing
> appfs, on next reboot, system should use rfs.0 and appfs.*1*.
> Similarly for rfs.
> 
> * To achieve this we started without a parent child relationship in
> system.conf. So when we are executing from appfs.0 and rfs.0 and we
> try to flash a bundle containing appfs, it would try to write to
> appfs.0 which would already be in use and hence we would get errors.
> So we introduced the parent child relationship
> 
> * Now _with_ parent child relationship the issue above would be
> resolved. Now when we boot from "rfs.0" and execute apps from
> "appfs.0" and flash a appfs bundle it would write to appfs.1. On
> reboot, as per our scheme we would boot from "rfs.0" and execute apps
> from "appfs.1". Now when we try to do a rauc install it would write
> to "appfs.1" as it would consider appfs.0 as active partition due to
> parent-child relationship.
> 
> * Hence as we were running into all these issues we felt that it
> would help our cause if we had our own slot selection logic. Hence
> the deviation to rauc info + rauc extract + rauc write-slot.

Ok, I think I got your base motivation.

What do you do to assure that the right appfs is used?
Is that mechanism atomic? I.e. when you are interrupted between update
of appfs, does it still boot the valid one?

In general, as you might have read, our strict design case and
recommendation is that you only install well-tested sets of all pieces
of software. If you use it as an 'app store' maybe that is note a task
for RAUC?

But, what speaks against having both rootfs and appfs updated together?
Is that installation time / bundle size?

Note that RAUC will skip the actual update of the rootfs anyway if the
slot's hash matches the hash of the image to install. Thus if you do not
change the rootfs content across multiple instalations, it will skip
the rootfs writing, anyway.


Another concept to thinks about is (if you insist on updating the
appfs separately), if you do not move the switching point entirely to
be in the rootfs. Then RAUC would incorporate with the appfs-selection
logic you currently use. Might need some extension for using custom boot
selection scripts, but this is on our list anyway.

The open question would be if you also intend to update your rootfs and
if it is ok to never update the appfs and rootfs together, as this would
not work with my approach.


Regards, Enrico

-- 
Pengutronix e.K.                           | Enrico Jörns                |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |


_______________________________________________
RAUC mailing list

Re: [RAUC] Query regarding rauc

[Abhishek Kumar Rai]

Hi Enrico,

Thanks for the detailed response. Helped change our perspective and
understand rauc better.

Please find our responses inline

Regards,
Abhishek Rai

On Tue, Dec 4, 2018 at 6:20 PM Enrico Joerns <ejo@pengutronix.de> wrote:
>
> Hi Abhishek,
>
> On 12/4/18 1:06 PM, Abhishek Kumar Rai wrote:
> > Hi Enrico,
> >
> > Thanks for the prompt response!
> >
> > Please find our responses inline (blue colored)
>
> you're lucky that I've a graphical mailer... ;)
> Better use standard mail quoting.
>

Sure. Lesson learnt :)

> >> Our understanding of rauc install suggests that it does the
> >> following things:
> >>
> >> 1. Verify signature of the bundle 2. Run partition selection
> >> algorithm 3. Extract the tar balls from the bundle 4. Create new
> >> ext4 FS on the selected partition 5. Write appropriate tar ball to
> >> the partition
> >>
> >>
> >> Since "rauc install" would give us errors due to our configuration
> >> we plan to achieve this using the steps below:
> >>
> >> 1. Verify signature of the bundle (rauc info ...) 2. Partition
> >> selection algorithm (we would like to skip this) 3. Extract the tar
> >> balls from the bundle (rauc extract ..) 4. Create new ext4 FS on
> >> the selected partition + Write appropriate tar ball to the
> >> partition (rauc write-slot ... ....)
> >
> > how do you trigger the custom installation? Do you use the
> > full-custom RAUC install handler? No for now we have replaced the
> > "rauc install" call with the calls to the commands "rauc info + rauc
> > extract + rauc write-slot". We were thinking that we might not get
> > control due to slot selection algorithm. But we will explore this
> > further starting today. Get a feeling it would be what we want.
> > Though we read in the documentation that the pre/post install hooks
> > would not work with full custom install. We might need them. Not too
> > sure though. We might be able to merge the "post-install" hook work
> > with the "install" hook
>
> You want to have a look at this sections:
>
> https://rauc.readthedocs.io/en/latest/using.html#full-custom-update
>

Thanks!

> > Extracting the tar's should not be required in all cases. With the
> > full-custom handler you will already have it mounted, otherwise you
> > can simply mount the bundle after verification as it is basically a
> > squashfs. Sure..
> >
> > For rauc write-slot you also rely on parts of the system.conf, but
> > not in the fixed scheme. If that is where your issue resides, you
> > should be fine with that. Sure. We were also considering referring to
> > the manifest file when the bundle gets mounted
> >
> > Could you give me a hint what forces you to do this kind of manual
> > handling? Maybe then we can think about a possible solution more
> > targeted.
> >
> > * For now, we have 4 partitions containing rfs.0, rfs.1, appfs.0 and
> > appfs.1. We would like to have a scheme where only the partition that
> > has been flashed with bundle data is  changed i.e lets say currently
> > we are using rfs.0 and appfs.0. Now if we flash a bundle containing
> > appfs, on next reboot, system should use rfs.0 and appfs.*1*.
> > Similarly for rfs.
> >
> > * To achieve this we started without a parent child relationship in
> > system.conf. So when we are executing from appfs.0 and rfs.0 and we
> > try to flash a bundle containing appfs, it would try to write to
> > appfs.0 which would already be in use and hence we would get errors.
> > So we introduced the parent child relationship
> >
> > * Now _with_ parent child relationship the issue above would be
> > resolved. Now when we boot from "rfs.0" and execute apps from
> > "appfs.0" and flash a appfs bundle it would write to appfs.1. On
> > reboot, as per our scheme we would boot from "rfs.0" and execute apps
> > from "appfs.1". Now when we try to do a rauc install it would write
> > to "appfs.1" as it would consider appfs.0 as active partition due to
> > parent-child relationship.
> >
> > * Hence as we were running into all these issues we felt that it
> > would help our cause if we had our own slot selection logic. Hence
> > the deviation to rauc info + rauc extract + rauc write-slot.
>
> Ok, I think I got your base motivation.
>
> What do you do to assure that the right appfs is used?
> Is that mechanism atomic? I.e. when you are interrupted between update
> of appfs, does it still boot the valid one?
>

We have yet not reached a stage where we would start releasing out the
app bundles. Probably we would do it in a month or two.
Yes if we are interrupted between update of appfs we would not switch
to using the target partition and continue using the same appfs
partition

> In general, as you might have read, our strict design case and
> recommendation is that you only install well-tested sets of all pieces
> of software. If you use it as an 'app store' maybe that is note a task
> for RAUC?
>

Ohh. But we are able to achieve the appfs upgrades by mending rauc a
little bit. Isnt that ok? Would you be able to provide more details as
to why it might not be a good idea to use rauc for app-only upgrades?
This would help us better design our update scheme. The core idea
behind all the experiments is we wish to maintain symmetry and across
reboots change only the partition that has been updated. So thats how
the idea of having a rfs.0 + appfs.1 combination in place of rfs.1 +
appfs.1 comes into the picture.
For now, we have 3 use-cases (that might increase)
- rfs only updates
- appfs only updates
- rfs + appfs updates

> But, what speaks against having both rootfs and appfs updated together?
> Is that installation time / bundle size?
>

Yes. Its mainly the bundle size. We would be updating our apps more
frequently as compared to rfs. Hence we would want to release as small
a bundle as we possibly can.

> Note that RAUC will skip the actual update of the rootfs anyway if the
> slot's hash matches the hash of the image to install. Thus if you do not
> change the rootfs content across multiple instalations, it will skip
> the rootfs writing, anyway.
>
>

Ok..

> Another concept to thinks about is (if you insist on updating the
> appfs separately), if you do not move the switching point entirely to
> be in the rootfs. Then RAUC would incorporate with the appfs-selection
> logic you currently use. Might need some extension for using custom boot
> selection scripts, but this is on our list anyway.
>

Yes. For now, we are planning to have that intelligence of
appfs-selection in our boot script.

> The open question would be if you also intend to update your rootfs and
> if it is ok to never update the appfs and rootfs together, as this would
> not work with my approach.
>
>

Yes. For now we have 3 different target use-cases
- rfs only : an update might happen once every 6 months or so
- appfs only : an update might happen monthly to start with and then
on a need basis
- rfs + appfs : probably once every 6 months or so

> Regards, Enrico
>
> --
> Pengutronix e.K.                           | Enrico Jörns                |
> Industrial Linux Solutions                 | http://www.pengutronix.de/  |
> Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
> Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |
>

-- 














The information contained in this e-mail message (including 
any 


attachments) may be confidential, proprietary, privileged, or 
otherwise


exempt from disclosure under applicable laws. It is intended to 
be 


conveyed only to the designated recipient(s). Any use, dissemination, 



distribution, printing, retaining or copying of this e-mail (including 
its 


attachments) by unintended recipient(s) is strictly prohibited and 
may 


be unlawful. If you are not an intended recipient of this e-mail, or 
believe 


that you have received this e-mail in error, please notify the 
sender 


immediately (by replying to this e-mail), delete any and all 
copies of 


this e-mail (including any attachments) from your system, and 
do not


disclose the content of this e-mail to any other person. Thank you 
for your cooperation.






-- 
_This e-mail message (including any attachments) may be confidential, 
proprietary, privileged, or otherwise exempt from disclosure under 
applicable laws. If you are not an intended recipient, please delete this 
message. Thank you.
_

_______________________________________________
RAUC mailing list

Re: [RAUC] Query regarding rauc

[Abhishek Kumar Rai]

[-- Attachment #1.1: Type: text/plain, Size: 9168 bytes --]

Hi Team,

Please provide your valuable suggestion regarding the above query .

Regards,
Abhishek Rai

On Wed, Dec 5, 2018 at 12:15 PM Abhishek Kumar Rai <
abhishekr@eximiusdesign.com> wrote:

> Hi Enrico,
>
> Thanks for the detailed response. Helped change our perspective and
> understand rauc better.
>
> Please find our responses inline
>
> Regards,
> Abhishek Rai
>
> On Tue, Dec 4, 2018 at 6:20 PM Enrico Joerns <ejo@pengutronix.de> wrote:
> >
> > Hi Abhishek,
> >
> > On 12/4/18 1:06 PM, Abhishek Kumar Rai wrote:
> > > Hi Enrico,
> > >
> > > Thanks for the prompt response!
> > >
> > > Please find our responses inline (blue colored)
> >
> > you're lucky that I've a graphical mailer... ;)
> > Better use standard mail quoting.
> >
>
> Sure. Lesson learnt :)
>
> > >> Our understanding of rauc install suggests that it does the
> > >> following things:
> > >>
> > >> 1. Verify signature of the bundle 2. Run partition selection
> > >> algorithm 3. Extract the tar balls from the bundle 4. Create new
> > >> ext4 FS on the selected partition 5. Write appropriate tar ball to
> > >> the partition
> > >>
> > >>
> > >> Since "rauc install" would give us errors due to our configuration
> > >> we plan to achieve this using the steps below:
> > >>
> > >> 1. Verify signature of the bundle (rauc info ...) 2. Partition
> > >> selection algorithm (we would like to skip this) 3. Extract the tar
> > >> balls from the bundle (rauc extract ..) 4. Create new ext4 FS on
> > >> the selected partition + Write appropriate tar ball to the
> > >> partition (rauc write-slot ... ....)
> > >
> > > how do you trigger the custom installation? Do you use the
> > > full-custom RAUC install handler? No for now we have replaced the
> > > "rauc install" call with the calls to the commands "rauc info + rauc
> > > extract + rauc write-slot". We were thinking that we might not get
> > > control due to slot selection algorithm. But we will explore this
> > > further starting today. Get a feeling it would be what we want.
> > > Though we read in the documentation that the pre/post install hooks
> > > would not work with full custom install. We might need them. Not too
> > > sure though. We might be able to merge the "post-install" hook work
> > > with the "install" hook
> >
> > You want to have a look at this sections:
> >
> > https://rauc.readthedocs.io/en/latest/using.html#full-custom-update
> >
>
> Thanks!
>
> > > Extracting the tar's should not be required in all cases. With the
> > > full-custom handler you will already have it mounted, otherwise you
> > > can simply mount the bundle after verification as it is basically a
> > > squashfs. Sure..
> > >
> > > For rauc write-slot you also rely on parts of the system.conf, but
> > > not in the fixed scheme. If that is where your issue resides, you
> > > should be fine with that. Sure. We were also considering referring to
> > > the manifest file when the bundle gets mounted
> > >
> > > Could you give me a hint what forces you to do this kind of manual
> > > handling? Maybe then we can think about a possible solution more
> > > targeted.
> > >
> > > * For now, we have 4 partitions containing rfs.0, rfs.1, appfs.0 and
> > > appfs.1. We would like to have a scheme where only the partition that
> > > has been flashed with bundle data is  changed i.e lets say currently
> > > we are using rfs.0 and appfs.0. Now if we flash a bundle containing
> > > appfs, on next reboot, system should use rfs.0 and appfs.*1*.
> > > Similarly for rfs.
> > >
> > > * To achieve this we started without a parent child relationship in
> > > system.conf. So when we are executing from appfs.0 and rfs.0 and we
> > > try to flash a bundle containing appfs, it would try to write to
> > > appfs.0 which would already be in use and hence we would get errors.
> > > So we introduced the parent child relationship
> > >
> > > * Now _with_ parent child relationship the issue above would be
> > > resolved. Now when we boot from "rfs.0" and execute apps from
> > > "appfs.0" and flash a appfs bundle it would write to appfs.1. On
> > > reboot, as per our scheme we would boot from "rfs.0" and execute apps
> > > from "appfs.1". Now when we try to do a rauc install it would write
> > > to "appfs.1" as it would consider appfs.0 as active partition due to
> > > parent-child relationship.
> > >
> > > * Hence as we were running into all these issues we felt that it
> > > would help our cause if we had our own slot selection logic. Hence
> > > the deviation to rauc info + rauc extract + rauc write-slot.
> >
> > Ok, I think I got your base motivation.
> >
> > What do you do to assure that the right appfs is used?
> > Is that mechanism atomic? I.e. when you are interrupted between update
> > of appfs, does it still boot the valid one?
> >
>
> We have yet not reached a stage where we would start releasing out the
> app bundles. Probably we would do it in a month or two.
> Yes if we are interrupted between update of appfs we would not switch
> to using the target partition and continue using the same appfs
> partition
>
> > In general, as you might have read, our strict design case and
> > recommendation is that you only install well-tested sets of all pieces
> > of software. If you use it as an 'app store' maybe that is note a task
> > for RAUC?
> >
>
> Ohh. But we are able to achieve the appfs upgrades by mending rauc a
> little bit. Isnt that ok? Would you be able to provide more details as
> to why it might not be a good idea to use rauc for app-only upgrades?
> This would help us better design our update scheme. The core idea
> behind all the experiments is we wish to maintain symmetry and across
> reboots change only the partition that has been updated. So thats how
> the idea of having a rfs.0 + appfs.1 combination in place of rfs.1 +
> appfs.1 comes into the picture.
> For now, we have 3 use-cases (that might increase)
> - rfs only updates
> - appfs only updates
> - rfs + appfs updates
>
> > But, what speaks against having both rootfs and appfs updated together?
> > Is that installation time / bundle size?
> >
>
> Yes. Its mainly the bundle size. We would be updating our apps more
> frequently as compared to rfs. Hence we would want to release as small
> a bundle as we possibly can.
>
> > Note that RAUC will skip the actual update of the rootfs anyway if the
> > slot's hash matches the hash of the image to install. Thus if you do not
> > change the rootfs content across multiple instalations, it will skip
> > the rootfs writing, anyway.
> >
> >
>
> Ok..
>
> > Another concept to thinks about is (if you insist on updating the
> > appfs separately), if you do not move the switching point entirely to
> > be in the rootfs. Then RAUC would incorporate with the appfs-selection
> > logic you currently use. Might need some extension for using custom boot
> > selection scripts, but this is on our list anyway.
> >
>
> Yes. For now, we are planning to have that intelligence of
> appfs-selection in our boot script.
>
> > The open question would be if you also intend to update your rootfs and
> > if it is ok to never update the appfs and rootfs together, as this would
> > not work with my approach.
> >
> >
>
> Yes. For now we have 3 different target use-cases
> - rfs only : an update might happen once every 6 months or so
> - appfs only : an update might happen monthly to start with and then
> on a need basis
> - rfs + appfs : probably once every 6 months or so
>
> > Regards, Enrico
> >
> > --
> > Pengutronix e.K.                           | Enrico Jörns
> |
> > Industrial Linux Solutions                 | http://www.pengutronix.de/
> |
> > Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080
> |
> > Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555
> |
> >
>

-- 














The information contained in this e-mail message (including 
any 


attachments) may be confidential, proprietary, privileged, or 
otherwise


exempt from disclosure under applicable laws. It is intended to 
be 


conveyed only to the designated recipient(s). Any use, dissemination, 



distribution, printing, retaining or copying of this e-mail (including 
its 


attachments) by unintended recipient(s) is strictly prohibited and 
may 


be unlawful. If you are not an intended recipient of this e-mail, or 
believe 


that you have received this e-mail in error, please notify the 
sender 


immediately (by replying to this e-mail), delete any and all 
copies of 


this e-mail (including any attachments) from your system, and 
do not


disclose the content of this e-mail to any other person. Thank you 
for your cooperation.






-- 
_This e-mail message (including any attachments) may be confidential, 
proprietary, privileged, or otherwise exempt from disclosure under 
applicable laws. If you are not an intended recipient, please delete this 
message. Thank you.
_

[-- Attachment #1.2: Type: text/html, Size: 11288 bytes --]

[-- Attachment #2: Type: text/plain, Size: 65 bytes --]

_______________________________________________
RAUC mailing list

Re: [RAUC] Query regarding rauc

[Enrico Joerns]

Hi Abhishek,

On 12/6/18 10:24 AM, Abhishek Kumar Rai wrote:
> Hi Team,
> 
> Please provide your valuable suggestion regarding the above query .

please do not put too much pressure on community mailing lists.
There are business support channels available for this.

>      > > Extracting the tar's should not be required in all cases. With the
>      > > full-custom handler you will already have it mounted, otherwise you
>      > > can simply mount the bundle after verification as it is basically a
>      > > squashfs. Sure..
>      > >
>      > > For rauc write-slot you also rely on parts of the system.conf, but
>      > > not in the fixed scheme. If that is where your issue resides, you
>      > > should be fine with that. Sure. We were also considering referring to
>      > > the manifest file when the bundle gets mounted
>      > >
>      > > Could you give me a hint what forces you to do this kind of manual
>      > > handling? Maybe then we can think about a possible solution more
>      > > targeted.
>      > >
>      > > * For now, we have 4 partitions containing rfs.0, rfs.1, appfs.0 and
>      > > appfs.1. We would like to have a scheme where only the partition that
>      > > has been flashed with bundle data is  changed i.e lets say currently
>      > > we are using rfs.0 and appfs.0. Now if we flash a bundle containing
>      > > appfs, on next reboot, system should use rfs.0 and appfs.*1*.
>      > > Similarly for rfs.
>      > >
>      > > * To achieve this we started without a parent child relationship in
>      > > system.conf. So when we are executing from appfs.0 and rfs.0 and we
>      > > try to flash a bundle containing appfs, it would try to write to
>      > > appfs.0 which would already be in use and hence we would get errors.
>      > > So we introduced the parent child relationship
>      > >
>      > > * Now _with_ parent child relationship the issue above would be
>      > > resolved. Now when we boot from "rfs.0" and execute apps from
>      > > "appfs.0" and flash a appfs bundle it would write to appfs.1. On
>      > > reboot, as per our scheme we would boot from "rfs.0" and execute apps
>      > > from "appfs.1". Now when we try to do a rauc install it would write
>      > > to "appfs.1" as it would consider appfs.0 as active partition due to
>      > > parent-child relationship.
>      > >
>      > > * Hence as we were running into all these issues we felt that it
>      > > would help our cause if we had our own slot selection logic. Hence
>      > > the deviation to rauc info + rauc extract + rauc write-slot.
>      >
>      > Ok, I think I got your base motivation.
>      >
>      > What do you do to assure that the right appfs is used?
>      > Is that mechanism atomic? I.e. when you are interrupted between update
>      > of appfs, does it still boot the valid one?
>      >
> 
>     We have yet not reached a stage where we would start releasing out the
>     app bundles. Probably we would do it in a month or two.
>     Yes if we are interrupted between update of appfs we would not switch
>     to using the target partition and continue using the same appfs
>     partition

Where do you store the information about the active partition?
As you cannot place it in the rootfs...

>      > In general, as you might have read, our strict design case and
>      > recommendation is that you only install well-tested sets of all pieces
>      > of software. If you use it as an 'app store' maybe that is note a task
>      > for RAUC?
>      >
> 
>     Ohh. But we are able to achieve the appfs upgrades by mending rauc a
>     little bit. Isnt that ok? Would you be able to provide more details as
>     to why it might not be a good idea to use rauc for app-only upgrades?

It's not mainly about using RAUC, it's mainly about how to setup a reliable system
and a reliable redundancy boot!

Basically, we have 2 options:

1) Using the application together with the rootfs it was tested for (highly recommended)!

For this you need to assure that rootfs and appfs contain the exact same state
that you used for testing and the default behavior for RAUC is valid.

2) Using the application on any rootfs (NOT recommended)

This is what you probably do on consumer systems such as Android but what is
really not recommended if you want to deliver a system that is proven to work.

>     This would help us better design our update scheme. The core idea
>     behind all the experiments is we wish to maintain symmetry and across
>     reboots change only the partition that has been updated. So thats how
>     the idea of having a rfs.0 + appfs.1 combination in place of rfs.1 +
>     appfs.1 comes into the picture.
>     For now, we have 3 use-cases (that might increase)
>     - rfs only updates
>     - appfs only updates
>     - rfs + appfs updates

Note that if these are all special cases, you also have to test all these special
cases and the more they become the more your testing vector and thus effort will
grow!

RAUC is also explicitly not designed to handle these 3 cases differently.

You could have appfs and rootfs as two independent redundancy slots
(without parent-child relation) but then you will not be able to update
rootsf and appfs together as this would require you to switch both appfs
and rootfs which is always non-atomic and thus out of scope for RAUC.

>      > But, what speaks against having both rootfs and appfs updated together?
>      > Is that installation time / bundle size?
>      >
> 
>     Yes. Its mainly the bundle size. We would be updating our apps more
>     frequently as compared to rfs. Hence we would want to release as small
>     a bundle as we possibly can.

This sounds more or less like a requirement for (binary or file-based) delta updates.

File-based delta is on our wish list, binary delta is something we achieve (especially
for bundle-over-network cases) with our casync support [1]

>      > Note that RAUC will skip the actual update of the rootfs anyway if the
>      > slot's hash matches the hash of the image to install. Thus if you do not
>      > change the rootfs content across multiple instalations, it will skip
>      > the rootfs writing, anyway.
>      >
>      >
> 
>     Ok..
> 
>      > Another concept to thinks about is (if you insist on updating the
>      > appfs separately), if you do not move the switching point entirely to
>      > be in the rootfs. Then RAUC would incorporate with the appfs-selection
>      > logic you currently use. Might need some extension for using custom boot
>      > selection scripts, but this is on our list anyway.
>      >
> 
>     Yes. For now, we are planning to have that intelligence of
>     appfs-selection in our boot script.

Maybe you should be clear about this handling first before planning you
update solutions on it.

>      > The open question would be if you also intend to update your rootfs and
>      > if it is ok to never update the appfs and rootfs together, as this would
>      > not work with my approach.
>      >
>      >
> 
>     Yes. For now we have 3 different target use-cases
>     - rfs only : an update might happen once every 6 months or so
>     - appfs only : an update might happen monthly to start with and then
>     on a need basis
>     - rfs + appfs : probably once every 6 months or so

The crucial question is: Do you want to support writing the same appfs on different (non-tested)
variants of the rootfs!?

If not, meaning if you only want to install different appfs to a dedicated rootfs,
I would strongly recommend a solution that also references the target roofs SHA in
the manifest.

If bundle size is really a huge problem and if the (compressed!) rootfs in the bundle
is still too large for you, maybe having some kind of dummy slot description that
assures having the right rootfs for installation but no actual image for it contained
could be a solution.

Note that this still requires you to have the exact same rootfs on both redundant rootfs slots
and that an update of the appfs will always switch both rootfs and appfs!

This way you do also do not need an extra appfs-switching but only to assure that the
right appfs is mounted by the corresponding rootfs.

Similar discussions we also have in GitHub:

https://github.com/rauc/rauc/issues/360
https://github.com/rauc/rauc/pull/370

Best regards,

Enrico


[1] https://rauc.readthedocs.io/en/latest/advanced.html#rauc-casync-support

> 
> The information contained in this e-mail message (including any
> attachments) may be confidential, proprietary, privileged, or otherwise
> exempt from disclosure under applicable laws. It is intended to be
> conveyed only to the designated recipient(s). Any use, dissemination,
> distribution, printing, retaining or copying of this e-mail (including its
> attachments) by unintended recipient(s) is strictly prohibited and may
> be unlawful. If you are not an intended recipient of this e-mail, or believe
> that you have received this e-mail in error, please notify the sender
> immediately (by replying to this e-mail), delete any and all copies of
> this e-mail (including any attachments) from your system, and do not
> disclose the content of this e-mail to any other person. Thank you for your cooperation.
>
> /This e-mail message (including any attachments) may be confidential, proprietary, privileged, or otherwise exempt from disclosure under applicable laws. If you are not an intended recipient, please delete this message. Thank you./

I am still unsure if I am the intended recipient if you send such a mail
to a public mailing list.
I know this might be standard of your mailer / IT but really makes no sense on public
mailing list, apart from the fact that probably no one is allowed to answer you...

-- 
Pengutronix e.K.                           | Enrico Jörns                |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |


_______________________________________________
RAUC mailing list