Adding needed repository... Failed!

Sebastian Roth

@Nexqs I have not found out what’s causing the issue but looking at the PCAP. There is a lot of noise from the webbrowser. Should have told you to close that before doing the capture. But I still can get the gist. I see three DNS requests to keyserver.ubuntu.com five seconds (!!!) later followed by a HTTP request to retrieve the key. To me this sounds strange. Why would it do the DNS request that often before initiating the HTTP connection? Did you run the add-apt-repository command only once or did it fail and you can it three times?

What happens of you try to download the key using curl (or wget)?

user@fogserver:~$ curl "http://keyserver.ubuntu.com/pks/lookup?op=get&options=mr&search=0x14AA40EC0831756756D7F66C4F4EA0AAE5267A6C"
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: SKS 1.1.6
Comment: Hostname: keyserver.ubuntu.com

mI0ESX35nAEEALKDCUDVXvmW9n+T/+3G1DnTpoWh9/1xNaz/RrUH6fQKhHr568F8hfnZP/2C
GYVYkW9hxP9LVW9IDvzcmnhgIwK+ddeaPZqh3T/FM4OTA7Q78HSvR81mJpf2iMLm/Zvh89Zs
mP2sIgZuARiaHo8lxoTSLtmKXsM3FsJVlusyewHfABEBAAG0H0xhdW5jaHBhZCBQUEEgZm9y
IE9uZMWZZWogU3Vyw72ItgQTAQIAIAUCSX35nAIbAwYLCQgHAwIEFQIIAwQWAgMBAh4BAheA
AAoJEE9OoKrlJnpsQjYD/jW1NlIFAlT6EvF2xfVbkhERii9MapjaUsSso4XLCEmZdEGX54GQ
01svXnrivwnd/kmhKvyxCqiNLDY/dOaK8MK//bDI6mqdKmG8XbP2vsdsxhifNC+GH/OwaDPv
n1TyYB653kwyruCGFjEnCreZTcRUu2oBQyolORDl+BmF4DjL
=R5tk
-----END PGP PUBLIC KEY BLOCK-----

Should look like this.

Nexqs

@Sebastian-Roth I did once this commands : LANG='en_US.UTF-8' LC_ALL='en_US.UTF-8' sudo add-apt-repository -y ppa:ondrej/php after typing the commands sudo tcpdump -w add-repo.pcap
Soz i let the browser open !
Ill try to get the key via curl as you shown me : curl “http://keyserver.ubuntu.com/pks/lookup?op=get&options=mr&search=0x14AA40EC0831756756D7F66C4F4EA0AAE5267A6C”

Sebastian Roth

@nexqs said in Adding needed repository... Failed!:

Ill try to get the key via curl as you shown me

You might need to install curl first, nor sure if it is by default in Ubuntu?!

No worries about the browser… Should have told you.

Nexqs

@Sebastian-Roth I just did the cmd with an wget :

fog@fogserver:~$ wget "http://keyserver.ubuntu.com/pks/lookup?op=get&options=mr&search=0x14AA40EC0831756756D7F66C4F4EA0AAE5267A6C"
--2018-03-28 17:56:04--  http://keyserver.ubuntu.com/pks/lookup?op=get&options=mr&search=0x14AA40EC0831756756D7F66C4F4EA0AAE5267A6C
Resolving keyserver.ubuntu.com (keyserver.ubuntu.com)... 91.189.90.55, 91.189.89.49
Connecting to keyserver.ubuntu.com (keyserver.ubuntu.com)|91.189.90.55|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 626 [application/pgp-keys]
Saving to: ‘lookup?op=get&options=mr&search=0x14AA40EC0831756756D7F66C4F4EA0AAE5267A6C’

lookup?op=get&optio 100%[===================>]     626  --.-KB/s    in 0s      

2018-03-28 17:56:20 (43.4 MB/s) - ‘lookup?op=get&options=mr&search=0x14AA40EC0831756756D7F66C4F4EA0AAE5267A6C’ saved [626/626]

fog@fogserver:~$

Sebastian Roth

@Nexqs Hope you don’t feel annoyed by all the questions and stuff I make you try out. Could you please run wget "http://keyserver.ubuntu.com/pks/lookup?op=get&options=mr&search=0x14AA40EC0831756756D7F66C4F4EA0AAE5267A6C" ; cat "lookup?op=get&options=mr&search=0x14AA40EC0831756756D7F66C4F4EA0AAE5267A6C" just to make sure you get the same key downloaded that I had.

I will setup an Ubuntu VM and try things out as well. I am sure we can figure this out. I still have a feeling that this might be some kind of network issue on your side. Not sure though.

Sebastian Roth

@Nexqs Ok, I got up a test VM and did some testing. It works great on my side. Taking a packet dump on my machine I noticed that two DNS requests to launchpad.net seem to be part of the add-apt-repository call. I see those in your PCAP as well (after I filtered through all the other stuff more thoroughly). Now the interesting thing I notice is that the requests from your machine seem to take some time. We are talking milliseconds here but having looked into many packet dumps I clearly notice this as a possible pointer for explaining what’s wrong. While requests in my case come three milliseconds after the other yours seem to take 400 ms at times - which is hell of a lot even in networks. It could trigger some unexpected behavior. It’s not always like this but I do see this and wonder what’s wrong. As you capture right on the server to me it would simply translate to a high load on the server. Is there other (network/calculation) “stuff” happening at the same time? Is this running on a physical server or within some kind of VM environment?

What else do I see. It takes five (!!!) seconds before it sends the initiating TCP SYN packet to start the HTTP connection after it got the DNS answer. Something is definitely wrong here. From what I see in the PCAP it’s not the network but the host itself. Please run cat /proc/loadavg and post results here.

Nexqs

@sebastian-roth SO if i understand everything, its my internet connection that is slow ?

fog@fogserver:~$ cat /proc/loadavg
0.23 0.50 0.25 4/392 2005
fog@fogserver:~$ 
fog@fogserver:~$ wget “http://keyserver.ubuntu.com/pks/lookup?op=get&options=mr&search=0x14AA40EC0831756756D7F66C4F4EA0AAE5267A6C” 
[1] 2019
[2] 2020
fog@fogserver:~$ “http://keyserver.ubuntu.com/pks/lookup?op=get: Scheme missing.

[1]-  Exit 1                  wget “http://keyserver.ubuntu.com/pks/lookup?op=get
[2]+  Done                    options=mr
fog@fogserver:~$ cat “lookup?op=get&options=mr&search=0x14AA40EC0831756756D7F66C4F4EA0AAE5267A6C”
[1] 2021
[2] 2022
fog@fogserver:~$ cat: '“lookup?op=get': No such file or directory

Sebastian Roth

@Nexqs No from the PCAP it looks like your server is slow! Is FOG installed on hardware or in a virtual machine?

Sorry for the command hickup. It did not mask the stuff as command when I posted it and that broke it when you copy & pasted it. Please try again with this:

wget -O key.txt "http://keyserver.ubuntu.com/pks/lookup?op=get&options=mr&search=0x14AA40EC0831756756D7F66C4F4EA0AAE5267A6C"
cat key.txt

Nexqs

@Sebastian-Roth Ubuntu is installed on a VM (with 3 Go ramfor the vm : i have 8go on my laptop, an ssd and i3). It did the same thing with my VM on my desktop (i7, 6go ram for the vm and 16go in total and a ssd).
Here is the key :

fog@fogserver:~$ wget -O key.txt "http://keyserver.ubuntu.com/pks/lookup?op=get&options=mr&search=0x14AA40EC0831756756D7F66C4F4EA0AAE5267A6C"
--2018-04-01 14:41:46--  http://keyserver.ubuntu.com/pks/lookup?op=get&options=mr&search=0x14AA40EC0831756756D7F66C4F4EA0AAE5267A6C
Resolving keyserver.ubuntu.com (keyserver.ubuntu.com)... 91.189.90.55, 91.189.89.49
Connecting to keyserver.ubuntu.com (keyserver.ubuntu.com)|91.189.90.55|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 626 [application/pgp-keys]
Saving to: ‘key.txt’

key.txt             100%[===================>]     626  --.-KB/s    in 0s      

2018-04-01 14:42:02 (39.4 MB/s) - ‘key.txt’ saved [626/626]

fog@fogserver:~$ cat key.txt
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: SKS 1.1.6
Comment: Hostname: keyserver.ubuntu.com

mI0ESX35nAEEALKDCUDVXvmW9n+T/+3G1DnTpoWh9/1xNaz/RrUH6fQKhHr568F8hfnZP/2C
GYVYkW9hxP9LVW9IDvzcmnhgIwK+ddeaPZqh3T/FM4OTA7Q78HSvR81mJpf2iMLm/Zvh89Zs
mP2sIgZuARiaHo8lxoTSLtmKXsM3FsJVlusyewHfABEBAAG0H0xhdW5jaHBhZCBQUEEgZm9y
IE9uZMWZZWogU3Vyw72ItgQTAQIAIAUCSX35nAIbAwYLCQgHAwIEFQIIAwQWAgMBAh4BAheA
AAoJEE9OoKrlJnpsQjYD/jW1NlIFAlT6EvF2xfVbkhERii9MapjaUsSso4XLCEmZdEGX54GQ
01svXnrivwnd/kmhKvyxCqiNLDY/dOaK8MK//bDI6mqdKmG8XbP2vsdsxhifNC+GH/OwaDPv
n1TyYB653kwyruCGFjEnCreZTcRUu2oBQyolORDl+BmF4DjL
=R5tk
-----END PGP PUBLIC KEY BLOCK-----
fog@fogserver:~$

Sebastian Roth

@Nexqs I am at a loss here. This seems to be such a strange issue. I have seen a lot of crazy things happen but this one I can’t get my head around. Why would apt-add-repository tell us “timed out” if it comes back adding the key anyway.

Are you keen to just modify the installer script to skip the check and go on? We can give you instructions on how to do this.

Nexqs

@Sebastian-Roth Yes sure ! strange issue …

Sebastian Roth

@Nexqs Sorry for my late reply. Please try this: Edit /opt/fogproject/lib/common/functions.sh, jump to line 623 and add a hash tag as comment at the beginning to disable, make it look like this:

...
            esac
            ;;
    esac
    #errorStat $?
    dots "Preparing Package Manager"
...

Now rerun the installer as usual.

Arrowtron

Hi there,

I am getting the same issue as well, please assist

Currently running FOG 1.5.0, trying to upgrade to 1.5.2 through PUTTY

0_1523498264074_bb171c07-b7e9-411b-b91e-ce0b482d7901-image.png

I am not running through a Proxy, i have upgraded to 1.5.0 a few weeks ago i think

Arrowtron

@arrowtron

Update:

@Sebastian-Roth Post about editing \root\fogproject\lib\common\functions.sh has fixed the issue

Thanks!

benc

I just ran into this issue:

 * Adding needed repository....................................Failed!

Fresh install of Ubuntu server 18.04 LTS using network installer on an old IBM X3650 server. The only software I selected to install during Ubuntu setup was the SSH server. Downloaded and extracted the tarball from https://github.com/FOGProject/fogproject/archive/1.5.5.tar.gz. Could not get past the error until I ran this:

sudo apt-get install software-properties-common

It looks like the FOG install script expected that package to already be installed. Hope this helps someone.

Ben

Sebastian Roth

@benc Thanks for letting us know. As far as I can see we do exactly this within the installer. See here: https://github.com/FOGProject/fogproject/blob/master/lib/common/functions.sh#L534

As we have been informed about this being an issue within Ubuntu twice in the last days I am wondering if there is anything Ubuntu changed to make this fail?!

@Wayne-Workman The installer test seems absolutely fine but possibly that’s because you have the package software-properties-common pre-installed? Can you check on that? I have had a look at the logs from your installer tests but the package is not mentioned and that’s why I guess you have it pre-installed…

Sebastian Roth

@benc I just pushed a fix to the dev-branch - download here: https://github.com/FOGProject/fogproject/archive/dev-branch.tar.gz

Wayne Workman

@Sebastian-Roth said in Adding needed repository... Failed!:

@Wayne-Workman The installer test seems absolutely fine but possibly that’s because you have the package software-properties-common pre-installed? Can you check on that? I have had a look at the logs from your installer tests but the package is not mentioned and that’s why I guess you have it pre-installed…

Yes, it’s pre-installed. This is on Ubuntu 18.04 server, minimal installation.

root@ip-10-0-0-165:~# dpkg -l | grep software-properties-common
ii  software-properties-common     0.96.24.32.7                      all          manage the repositories that you install software from (common)

Adding needed repository... Failed!

226

11.6k

17.1k

154.6k