Win7x64 : Golden Image : partitions problems ... again

[Quazz]

@Sebastian-Roth Yes, the first and third e should be é but, the second e should just be a plain e!

Observe

if [[ $label =~ [Rr][Ee][Ss][Ee][Rr][Vv][Ee][Dd] || $label =~ [Rr][�~Ié][Ss][�~Ié][Rr][Vv][�~Ié] ]];

The word it should look for is Réservé

It will never match under these conditions and explains why the problem remains unsolved, imo.

[Jonathan Cool]

Hi !

You right @Quazz … the good word is “Réservé” and not “Résérvé”.

I’m back with the results of commands (on VM Golden Img) :
https://drive.google.com/file/d/13gj87dVq2SThjy5IYdbmtiyOCZuXXtDu/view?usp=sharing

my fog.postint :

#!/bin/bash
## This file serves as a starting point to call your custom pre-imaging/post init loading scripts.
## <SCRIPTNAME> should be changed to the script you're planning to use.
## Syntax of post init scripts are
#. ${postinitpath}<SCRIPTNAME>

sed -i -e "s#\[Rr\]\[Ee\]\[Ss\]\[Ee\]\[Rr\]\[Vv\]\[Ee\]\[Dd\]#RM-CM-\\\)servM-CM-\\\)#gi" /bin/fog.upload
sed -i -e "s#\[Rr\]\[Ee\]\[Ss\]\[Ee\]\[Rr\]\[Vv\]\[Ee\]\[Dd\]#RM-CM-\\\)servM-CM-\\\)#gi" /usr/share/fog/lib/funcs.sh```

[Jonathan Cool]

I think, this fix is not good : https://github.com/FOGProject/fos/blob/master/Buildroot/board/FOG/FOS/rootfs_overlay/usr/share/fog/lib/funcs.sh#L493

Maybe the good fix is : line 493 :

if [[ $label =~ [Rr][Ee][Ss][Ee][Rr][Vv][Ee][Dd] || $label =~ [Rr][Éé][Ss][Ee][Rr][Vv][Éé] ]]; then

[Quazz]

@Jonathan-Cool I created a pull request for this, but in the mean time you can of course edit those files on your own install already. Should do the trick, I reckon.

edit: Although, looking back, part of the problem is how blkid spits out the label when using é

blkid | grep /dev/sda1

Should spit it out.

I’ve tested a bit myself and the same thing happens on my French installs.

Will need further looking into.

I also noticed I can’t type in certain characters in FOS such as é, perhaps related, will look into it later.

[Sebastian Roth]

@Jonathan-Cool Arghhhh, I got it wrong again. My bad!

grep Vv /usr/share/fog/lib/funcs.sh
grep Vv /bin/fog.upload
blkid -po udev /dev/sda1 | grep LABEL

Keeping my fingers crossed I get it right this time…

@Quazz Thanks for finding the tripple é’s. Didn’t notice that. My guess is that it won’t help much because the label printed is not in the same character encoding as you noticed already. Thanks for the pull request. Shall I merge? I still hope we can come up with a way to pinpoint those partitions without labels. See my detailed comment here: https://github.com/FOGProject/fogproject/issues/195 (looking for files on the partitions might be way more reliable I hope)

[Jonathan Cool]

Hi,

Thank you both for you precious help …

Back with the results : https://drive.google.com/file/d/1hJXQ-H7C4gZPvAUlPlVz9CzmFmBeVLK2/view?usp=sharing

@Quazz : How i can fix my funcs.sh ? i can hack it with my fog.postinit i think ? With sed tool …

[Sebastian Roth]

@Jonathan-Cool Holy crap (sorry)! Thanks heaps for testing both blkidcommands, the one Quazz posted and the one I did. I would have never expected those to return such different results!! As we use the blkid -po udev syntax in the scripts (ref) we are back in the race. Mind trying this script:

#!/bin/bash
## This file serves as a starting point to call your custom pre-imaging/post init loading scripts.
## <SCRIPTNAME> should be changed to the script you're planning to use.
## Syntax of post init scripts are
#. ${postinitpath}<SCRIPTNAME>

sed -i -e "s#\[Rr\]\[Ee\]\[Ss\]\[Ee\]\[Rr\]\[Vv\]\[Ee\]\[Dd\]#\[Rr\]\[Éé\]\[Ss\]\[Ee\]\[Rr\]\[Vv\]\[Éé\]#gi" /bin/fog.upload
sed -i -e "s#\[Rr\]\[Ee\]\[Ss\]\[Ee\]\[Rr\]\[Vv\]\[Ee\]\[Dd\]#\[Rr\]\[Éé\]\[Ss\]\[Ee\]\[Rr\]\[Vv\]\[Éé\]#gi" /usr/share/fog/lib/funcs.sh

Again, boot up the client in debug mode and run:

grep Vv /usr/share/fog/lib/funcs.sh
grep Vv /bin/fog.upload
blkid -po udev /dev/sda1 | grep LABEL

If those match you can go ahead and capture the image (issue command fog on the console and step through).

[Quazz]

@Sebastian-Roth Yes, your pull request seems much better, although I do wonder if there won’t be any problems with a file based approach, what if someone creates a similar structure on their other partitions for some reason?

Btw, the whole label thing is very strange with the encoding because even when you search for the exact label as blkid prints it out, it still doesn’t match.

The ID_FS_LABEL as Jonathan has already provided seems to have the correct encoding, though.

Very strange indeed, but for the time being this could be what we are looking for.

[Quazz]

@Sebastian-Roth Perhaps we could use something like

parted -l /dev/sda | grep boot | awk '{print $1}'

(making use of the partition flags to find the correct partition, since flags should always be in English in FOS, this should work across the board)

Which will return the partition number for the boot partition (for both linux and windows even)

All these tools are already present in the inits.

Can also use similar commands to find other special cases

Microsoft reserved partition (Windows 10 UEFI installations, typically third partition)

parted -l /dev/sda | grep msftres | awk '{print $1}'

Hidden “Basic data partition” (Windows 10 UEFI installations, typically first partition)

parted -l /dev/sda | grep hidden | awk '{print $1}'

Linux swap

parted -l /dev/sda | grep swap | awk '{print $1}'

Linux LVM

parted -l /dev/sda | grep lvm | awk '{print $1}'

The only edge cases coming to mind are Windows XP and such maybe. They might set the boot flag on the only partition.

But it shouldn’t be hard to detect if there’s only one partition in which case skip the remaining logic because it would be pointless to check.

[Jonathan Cool]

@Sebastian-Roth said in Win7x64 : Golden Image : partitions problems ... again:

@Jonathan-Cool Holy crap (sorry)! Thanks heaps for testing both blkidcommands, the one Quazz posted and the one I did. I would have never expected those to return such different results!! As we use the blkid -po udev syntax in the scripts (ref) we are back in the race. Mind trying this script:

#!/bin/bash
## This file serves as a starting point to call your custom pre-imaging/post init loading scripts.
## <SCRIPTNAME> should be changed to the script you're planning to use.
## Syntax of post init scripts are
#. ${postinitpath}<SCRIPTNAME>

sed -i -e "s#\[Rr\]\[Ee\]\[Ss\]\[Ee\]\[Rr\]\[Vv\]\[Ee\]\[Dd\]#\[Rr\]\[Éé\]\[Ss\]\[Ee\]\[Rr\]\[Vv\]\[Éé\]#gi" /bin/fog.upload
sed -i -e "s#\[Rr\]\[Ee\]\[Ss\]\[Ee\]\[Rr\]\[Vv\]\[Ee\]\[Dd\]#\[Rr\]\[Éé\]\[Ss\]\[Ee\]\[Rr\]\[Vv\]\[Éé\]#gi" /usr/share/fog/lib/funcs.sh

Again, boot up the client in debug mode and run:

grep Vv /usr/share/fog/lib/funcs.sh
grep Vv /bin/fog.upload
blkid -po udev /dev/sda1 | grep LABEL

If those match you can go ahead and capture the image (issue command fog on the console and step through).

Hm, it’s seem not match …

my fog.postinit :

#!/bin/bash
## This file serves as a starting point to call your custom pre-imaging/post init loading scripts.
## <SCRIPTNAME> should be changed to the script you're planning to use.
## Syntax of post init scripts are
#. ${postinitpath}<SCRIPTNAME>

sed -i -e "s#[Rr][Ee][Ss][Ee][Rr][Vv][Ee][Dd]#[Rr][Éé][Ss][Ee][Rr][Vv][Éé]#gi" /bin/fog.upload
sed -i -e "s#[Rr][Ee][Ss][Ee][Rr][Vv][Ee][Dd]#[Rr][Éé][Ss][Ee][Rr][Vv][Éé]#gi" /usr/share/fog/lib/funcs.sh

The results on my VM - Debug Upload Task : https://drive.google.com/file/d/1vAlGXD-PKZgv_Kwve9NIsSkWqobHyOR2/view?usp=sharing

[Jonathan Cool]

Holy shit, i forgot to run “fog” and ctrl+c … i will be back soon with the real results …

[Jonathan Cool]

Back with the results … : https://drive.google.com/file/d/1Kf9DscMwdhU7VCRf5Y-L7rL86qXQc1wP/view?usp=sharing

[Quazz]

@Jonathan-Cool Looks like the sed didn’t quite work as intended.

Try this:

sed -i -e  "s#\\[Rr\\]\\[Ee\\]\\[Ss\\]\\[Ee\\]\\[Rr\\]\\[Vv\\]\\[Ee\\]\\[Dd\\]#[Rr][Éé][Ss][Ee][Rr][Vv][Éé]#gi" /bin/fog.upload
sed -i -e  "s#\\[Rr\\]\\[Ee\\]\\[Ss\\]\\[Ee\\]\\[Rr\\]\\[Vv\\]\\[Ee\\]\\[Dd\\]#[Rr][Éé][Ss][Ee][Rr][Vv][Éé]#gi" /usr/share/fog/lib/funcs.sh

edit: hold up, I see what’s going on here. The forum is removing the backslashes that are supposed to escape the brackets

edit2: there we go, this one should work, I think

[Jonathan Cool]

I’m back with bad news …

my fog.postinit :

#!/bin/bash
## ....
#. ${postinitpath}<SCRIPTNAME>

sed -i -e  "s#\[Rr\]\[Ee\]\[Ss\]\[Ee\]\[Rr\]\[Vv\]\[Ee\]\[Dd\]#[Rr][Éé][Ss][Ee][Rr][Vv][Éé]#gi" /bin/fog.upload
sed -i -e  "s#\[Rr\]\[Ee\]\[Ss\]\[Ee\]\[Rr\]\[Vv\]\[Ee\]\[Dd\]#[Rr][Éé][Ss][Ee][Rr][Vv][Éé]#gi" /usr/share/fog/lib/funcs.sh

Upload Debug Task : Seem to be good this time … https://drive.google.com/file/d/1cv2Lerb5MGFQlwkvHO1jKcfoa8580KWv/view?usp=sharing

After a deploy on o7050, same problem : Reserved Partition : 78Gb …

Maybe i can try a debug download task to see what the hell is going on ?

[george1421]

@Jonathan-Cool I would do a debug deploy. Then at the command prompt ensure that both fog.upload and funcs.sh have been properly patched.

[Quazz]

@Jonathan-Cool Ok, so the label should match now at the very least.

Did you recapture the image since implementing the latest sed?

If so, what do your d1 files look like for this image?

[Jonathan Cool]

Hi,
I’m back … i typed the same commands on client during a DL Debug Task … : same results : (with SSH)

[Fri Sep 28 root@fogclient ~]# fog
 * Running post init scripts.........................Done
 * Press [Enter] key to continue
^C
[Fri Sep 28 root@fogclient ~]# grep Vv /usr/share/fog/lib/funcs.sh 
            if [[ $label =~ [Rr][Ee][Cc][Oo][Vv][Ee][Rr][Yy] ]]; then
            if [[ $label =~ [Rr][Éé][Ss][Ee][Rr][Vv][Éé] || $label =~ [Rr][Éé][Ss][Éé][Rr][Vv][Éé] ]]; then
[Fri Sep 28 root@fogclient ~]# grep Vv /bin/fog.upload 
                            *[Rr][Ee][Cc][Oo][Vv][Ee][Rr][Yy]*|*[Rr][Éé][Ss][Ee][Rr][Vv][Éé]*)
[Fri Sep 28 root@fogclient ~]# blkid -po udev /dev/sda1 | grep LABEL
ID_FS_LABEL=Réservé_au_système
ID_FS_LABEL_ENC=Réservé\x20au\x20système
[Fri Sep 28 root@fogclient ~]# 

The patch seem to be applied as you can see … " if [[ $label =~ [Rr][Éé][Ss][Ee][Rr][Vv][Éé]"

@Quazz Sorry, i’m not sure if i understood the question about this file … i take a screenshot of the folder of my Golden Img :
https://drive.google.com/file/d/1iN8YbEsgOsCSu622R9qG4JzRE9xjXo1B/view?usp=sharinga
But, yes, i recapture the image before lastest sed

[Quazz]

@Jonathan-Cool Can you share the contents of d1.fixed_size_partitions d1.partitions and d1.minimum.partitions

[Jonathan Cool]

Yes i can !

d1.fixed_size_partitions :

:2

d1.partitions :

label: dos
label-id: 0xc91a5cff
device: /dev/sda
unit: sectors

/dev/sda1 : start=        2048, size=      204800, type=7, bootable
/dev/sda2 : start=      206848, size=   335335424, type=7

d1.minimum.partitions :

label: dos
label-id: 0xc91a5cff
device: /dev/sda
unit: sectors

/dev/sda1 : start=        2048, size=       52316, type=7, bootable
/dev/sda2 : start=      206848, size=   335335424, type=7

What do you think about that ?

[Quazz]

@Jonathan-Cool It’s very strange. The regex should definitely match now, yet it marks the wrong partition as fixed_size for some reason.

I’ve looked over the code and can’t immediatly find anything desperately wrong or incorrect for the time being.

The strange thing, imo, is that it marks the second partition as fixed, whereas we’d expect it to mark none as fixed if it were to fail (unless I’m missing something in the code, but don’t think so)