[RAUC] Progress indication

[RAUC] Progress indication

[Middelschulte, Leif]

Hi,

I want to flash a µCtrl, as part of a "Install group" using RAUC and (if necessary) a custom Handler (?).

Looking at https://github.com/rauc/rauc/blob/master/test/install-content/custom_handler.sh I was wondering about the progress indication.
What's the supposed way to extend RAUC to handle some kind of binary bundled within the raucb and communicate the update progress back to the (already implemented) Update Controller?

Cheers,

Leif
_______________________________________________
RAUC mailing list

Re: [RAUC] Progress indication

[Enrico Joerns]

Hi Leif,

On 04/26/2017 06:03 PM, Middelschulte, Leif wrote:
> Hi,
>
> I want to flash a µCtrl, as part of a "Install group" using RAUC and (if necessary) a custom Handler (?).
>
> Looking at https://github.com/rauc/rauc/blob/master/test/install-content/custom_handler.sh I was wondering about the progress indication.
> What's the supposed way to extend RAUC to handle some kind of binary bundled within the raucb and communicate the update progress back to the (already implemented) Update Controller?

the `custom handler` you took a look at is something different than what 
you require. It is a way to fully customize the entire RAUC installation 
process using a script or a binary.

I guess this is not what you intend to do.

If I got it right, you only want to have a custom handling for a single 
slot class.
This is what RAUCs `hooks` [1] are for. Or, to be more detailed, what 
the per-slot `install` hook [2] is for.

Hooks are defined in your Bundles manifest and provided by a single 
`hook` script that must be placed inside the bundle and referred to in 
the bundle section `[hooks]`:

   [hooks]
   filename=my-hooks.sh

Then you have to add to your µCtrl-Slots image section the line

   [image.uctrl]
   ...
   hooks=install

This will make RAUC calling the hook skript instead of performing the 
default installation routine for this slot.

In the hook script you should check for being executed with the 
arguments matching your slot. An example for this is in the 
documentation. Slightly adapted to your needs it will look something 
like this:

   #!/bin/sh

   case "$1" in
           slot-install)
                   # only uctrl needs to be handled
                   test "$RAUC_SLOT_CLASS" = "uctrl" || exit 0

                   uctrl-update-routine "$RAUC_IMAGE_NAME"
                   ;;
           *)
                   exit 1
                   ;;
   esac

   exit 0


Does that help you?


Best regards

Enrico


[1] 
http://rauc.readthedocs.io/en/latest/using.html#bundle-based-customization-hooks
[2] http://rauc.readthedocs.io/en/latest/using.html#install-hook

-- 
Pengutronix e.K.                           | Enrico Jörns                |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |


_______________________________________________
RAUC mailing list

Re: [RAUC] Progress indication

[Middelschulte, Leif]

Hi Enrico,

firstofall: Thanks for your reply!

Am Donnerstag, den 04.05.2017, 12:29 +0200 schrieb Enrico Joerns:
> Hi Leif,
> 
> On 04/26/2017 06:03 PM, Middelschulte, Leif wrote:
> > Hi,
> > 
> > I want to flash a µCtrl, as part of a "Install group" using RAUC and (if necessary) a custom Handler (?).
> > 
> > Looking at https://github.com/rauc/rauc/blob/master/test/install-content/custom_handler.sh I was wondering about the progress indication.
> > What's the supposed way to extend RAUC to handle some kind of binary bundled within the raucb and communicate the update progress back to the (already implemented) Update Controller?
> 
> the `custom handler` you took a look at is something different than what 
> you require. It is a way to fully customize the entire RAUC installation 
> process using a script or a binary.
> 
> I guess this is not what you intend to do.
> 
> If I got it right, you only want to have a custom handling for a single 
> slot class.
> This is what RAUCs `hooks` [1] are for. Or, to be more detailed, what 
> the per-slot `install` hook [2] is for.
> 
> Hooks are defined in your Bundles manifest and provided by a single 
> `hook` script that must be placed inside the bundle and referred to in 
> the bundle section `[hooks]`:
> 
>    [hooks]
>    filename=my-hooks.sh
> 
> Then you have to add to your µCtrl-Slots image section the line
> 
>    [image.uctrl]
>    ...
>    hooks=install
> 
> This will make RAUC calling the hook skript instead of performing the 
> default installation routine for this slot.
> 
> In the hook script you should check for being executed with the 
> arguments matching your slot. An example for this is in the 
> documentation. Slightly adapted to your needs it will look something 
> like this:
> 
>    #!/bin/sh
> 
>    case "$1" in
>            slot-install)
>                    # only uctrl needs to be handled
>                    test "$RAUC_SLOT_CLASS" = "uctrl" || exit 0
> 
>                    uctrl-update-routine "$RAUC_IMAGE_NAME"
>                    ;;
>            *)
>                    exit 1
>                    ;;
>    esac
> 
>    exit 0
> 
> 
> Does that help you?
Yeah, that makes sense.

Now, about my second question: How does a custom hook script communicate its progress back to the install-handler (i.e. rauc service)?

I'd go modify rauc's update_handler.c, set the G_SUBPROCESS_FLAGS_STDOUT_PIPE flag for the spawned slot script-process and read back the progress and propagate it to the DBus interface using
r_context_send_progress.
Is that the right direction here?

> 
> 
> Best regards
> 
> Enrico
> 
> 
> [1] 
> http://rauc.readthedocs.io/en/latest/using.html#bundle-based-customization-hooks
> [2] http://rauc.readthedocs.io/en/latest/using.html#install-hook
> 

Best regards,

Leif
_______________________________________________
RAUC mailing list

Re: [RAUC] Progress indication

[Enrico Joerns]

Hi Leif,

On 05/04/2017 04:33 PM, Middelschulte, Leif wrote:
>
> Now, about my second question: How does a custom hook script communicate its progress back to the install-handler (i.e. rauc service)?
>
> I'd go modify rauc's update_handler.c, set the G_SUBPROCESS_FLAGS_STDOUT_PIPE flag for the spawned slot script-process and read back the progress and propagate it to the DBus interface using
> r_context_send_progress.
> Is that the right direction here?

in RAUC, the progress is generated by adding a 'step' for an action you 
perform.
When starting the action you call r_context_begin_step(), when 
terminating the action you call r_context_end_step(). It is important to 
know that steps have a nesting depth that allows visualizers to control 
the granularity of information they show.
A step can also be marked as either successful or failed.


An example is the "Determining slot states" step:

   r_context_begin_step("determine_slot_states", "Determining slot 
states", 0);
   [...]
   r_context_end_step("determine_slot_states", res);


Thus, this should be the interface to use to add new steps.


The current granularity we have does not go deeper than "Copying image". 
This will be called for both a default install handler as well as for an 
install hook.

If you ask for progress from the install hook, you intend to be more 
fine-grained with your messages?

Best regards, Enrico


-- 
Pengutronix e.K.                           | Enrico Jörns                |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |


_______________________________________________
RAUC mailing list

Re: [RAUC] Progress indication

[Middelschulte, Leif]

Hello Enrico,

Am Mittwoch, den 10.05.2017, 15:22 +0200 schrieb Enrico Joerns:
> Hi Leif,
> 
> On 05/04/2017 04:33 PM, Middelschulte, Leif wrote:
> > 
> > Now, about my second question: How does a custom hook script communicate its progress back to the install-handler (i.e. rauc service)?
> > 
> > I'd go modify rauc's update_handler.c, set the G_SUBPROCESS_FLAGS_STDOUT_PIPE flag for the spawned slot script-process and read back the progress and propagate it to the DBus interface using
> > r_context_send_progress.
> > Is that the right direction here?
> 
> in RAUC, the progress is generated by adding a 'step' for an action you 
> perform.
> When starting the action you call r_context_begin_step(), when 
> terminating the action you call r_context_end_step(). It is important to 
> know that steps have a nesting depth that allows visualizers to control 
> the granularity of information they show.
> A step can also be marked as either successful or failed.
> 
> 
> An example is the "Determining slot states" step:
> 
>    r_context_begin_step("determine_slot_states", "Determining slot 
> states", 0);
>    [...]
>    r_context_end_step("determine_slot_states", res);
> 
> 
> Thus, this should be the interface to use to add new steps.
> 
> 
> The current granularity we have does not go deeper than "Copying image". 
> This will be called for both a default install handler as well as for an 
> install hook.
> 
> If you ask for progress from the install hook, you intend to be more 
> fine-grained with your messages?
Yes, since some updates will take quite a while, we'd like to display an approximate progress (e.g. bytes written/byte to be written).

I saw that there's a "send_progress" callback one can use from *within* the installer. But currently it's not even used to indicate any progress of raw (stream) copies, right?

I'm trying to stick to upstream code as much as possible and, eventually, contribute what's necessary to communicate progress back.

If I understood the code right, I'd go and do the following:
1.) make my hook script write its percentual progress to stdout
2.) I'd fork RAUC Service's update_handler code to read back STDOUT from the spawned's process (i.e. hook.sh)
3.) use send_progress to publish the information to the "dumb" GUI

> 
> Best regards, Enrico
> 

Best regards,

Leif

P.S.: Is there an IRC, Slack or GITTER channel for RAUC discussions?
_______________________________________________
RAUC mailing list

Re: [RAUC] Progress indication

[Enrico Joerns]

Hi Leif,

On 05/11/2017 12:29 PM, Middelschulte, Leif wrote:
> Hello Enrico,
>
> Am Mittwoch, den 10.05.2017, 15:22 +0200 schrieb Enrico Joerns:
>> Hi Leif,
>>
>> On 05/04/2017 04:33 PM, Middelschulte, Leif wrote:
>>>
>>> Now, about my second question: How does a custom hook script
>>> communicate its progress back to the install-handler (i.e. rauc
>>> service)?
>>>
>>> I'd go modify rauc's update_handler.c, set the
>>> G_SUBPROCESS_FLAGS_STDOUT_PIPE flag for the spawned slot
>>> script-process and read back the progress and propagate it to the
>>> DBus interface using r_context_send_progress. Is that the right
>>> direction here?
>>
>> in RAUC, the progress is generated by adding a 'step' for an action
>> you perform. When starting the action you call
>> r_context_begin_step(), when terminating the action you call
>> r_context_end_step(). It is important to know that steps have a
>> nesting depth that allows visualizers to control the granularity of
>> information they show. A step can also be marked as either
>> successful or failed.
>>
>>
>> An example is the "Determining slot states" step:
>>
>> r_context_begin_step("determine_slot_states", "Determining slot
>> states", 0); [...] r_context_end_step("determine_slot_states",
>> res);
>>
>>
>> Thus, this should be the interface to use to add new steps.
>>
>>
>> The current granularity we have does not go deeper than "Copying
>> image". This will be called for both a default install handler as
>> well as for an install hook.
>>
>> If you ask for progress from the install hook, you intend to be
>> more fine-grained with your messages?
> Yes, since some updates will take quite a while, we'd like to display
> an approximate progress (e.g. bytes written/byte to be written).
>
> I saw that there's a "send_progress" callback one can use from
> *within* the installer.

Not sure if I understand you right, but this is what I described above, 
isn't it? A `git grep send_progress` only gives me 
`r_context_send_progress`.

> But currently it's not even used to indicate any progress of raw (stream) copies, right?

No, there is currently no progress indication for raw stream copies. For
this it would work, but most other copy operations performed by binaries
(such as nandwrite) do not provide any progress, thus we did not put too
much effort on it, yet.

> I'm trying to stick to upstream code as much as possible and,
> eventually, contribute what's necessary to communicate progress
> back.

That sounds like the perfect way for me :)

> If I understood the code right, I'd go and do the following:

> 1.) make my hook script write its percentual progress to stdout
> 2.) I'd fork RAUC Service's update_handler code to read back STDOUT from the
> spawned's process (i.e. hook.sh)
 > 3.) use send_progress to publish the information to the "dumb" GUI

Basically, yes, but what you should use for indicating progress (had to 
dig in the code for that, too ;) ) is `r_context_set_step_percentage()`. 
You can read from its documentation string that its designed for such 
purposes you have, where you have a step that you want to give a number 
of nonspecific substeps. Thus like a copy progress inside a `installing 
slot` step.

There you can provide percentage steps that will also propagate to the 
global progress, meaning that you will have something like

   50% Updating rootfs0...
   51% Updating rootfs0...
   52% Updating rootfs0...
   53% Updating rootfs0...
   ...

Unfortunately, the only place in the code where this is used, already is 
the "/progress/test_explicit_percentage" test in test/progress.c.

Hope that helps you.


Best regards, Enrico

-- 
Pengutronix e.K.                           | Enrico Jörns                |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-5080 |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |


_______________________________________________
RAUC mailing list