How to rescan /images/ to repopulate FOG Web UI>List all Images page

DBCountMan

I have secondary FOG server. I imported images from the main server and while I also copied all the images from the main, or tried to, not all of them made it over, the secondary server has much less drive capacity. So now I have a bunch of invalid and dup entries on the secondary FOG web ui image list. Is there a way to delete all of the images from the list and make the server rescan /images to repopulate that List all Images list?

Junkhacker

@brakcounty there is no way to “rescan” the images to rebuild the database entries, but here’s some python i used to list all images that exist as file folders but aren’t in the database

import mysql.connector
import os

mydb = mysql.connector.connect(
    host="localhost",
    database="fog",
    username="<valid database username>",
    password='<valid password>'
    )

def Diff(li1, li2):
    return (list(list(set(li1)-set(li2)) + list(set(li2)-set(li1))))

mycursor = mydb.cursor()
mycursor.execute("SELECT imagePath FROM images")
myresult = mycursor.fetchall()

dbimagelist = []

#print("########images in db")
for x in myresult:
    #print(x[0])
    dbimagelist.append(x[0])

#print("########directorys in /images")
#for x in os.listdir('/images'):
    #print (x)

#print("########diff")
for x in Diff(os.listdir('/images'),dbimagelist):
    print (x)

you may need to edit it for your environment

Sebastian Roth

@brakcounty As an addition to the nice script @Junkhacker posted I can tell you that manually creating the image definition in the FOG web UI as well. You just need to make sure names/paths match exactly the ones that you have in /images (be aware this is case sensitive!). As well you need to know the correct image type (e.g. non/resizable) and compression (Gzip/Zstd). The later two can be figured out by looking at the image files as well or just trying out till deploy works. Let us know if you need help finding out the correct settings for each image.

DBCountMan

@junkhacker Wow thanks! I will try that!

@Sebastian-Roth So as long as the “Image path” is correct, the name of the image in the FOG UI doesn’t really matter. I’ll go through and clean them up. Thanks again!

UPDATE: I saved that code into a file called checkimages.py and ran python3 checkimages.py as root and got this error:
Traceback (most recent call last):
File “checkimages.py”, line 1, in <module>
import mysql.connector
ModuleNotFoundError: No module named ‘mysql’
I entered my sql password in the script as well.

Junkhacker

@brakcounty use pip to install the mysql connector module for python

DBCountMan

@junkhacker So I can’t remember what password I set for the mysql db. But I have a new question related to the topic. I captured an image, uploaded to the main fog server. I copied that image folder from /images to the /images folder on my secondary FOG server. I created a new image in FOG web ui and pointed it to the folder I just copied. When I try to deploy the image from the secondary FOG server, an error comes up saying that image store could not be found. I’m guessing its because the image doesn’t really exist in the db even though I created it in the web UI.

Junkhacker

@brakcounty find the password in the config file fog uses to connect to the database.

make sure the directory you told it to use is correct. compare it to the other working images. and remember, it is case sensitive. are there any unusual characters in the path? did you put in the full path instead of the folder name by mistake? does the image show up on the list of images when you browse there? (perhaps you forgot to save)

DBCountMan

@junkhacker yup I had an underscore where it shouldn’t have been. bang.

I found the sql creds and placed them into your script. Ran it and got:

python3.8 checkimages.py 
Traceback (most recent call last):
  File "/home/administrator/.local/lib/python3.8/site-packages/mysql/connector/abstracts.py", line 301, in config
    DEFAULT_CONFIGURATION[key]
KeyError: 'username'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "checkimages.py", line 4, in <module>
    mydb = mysql.connector.connect(
  File "/home/administrator/.local/lib/python3.8/site-packages/mysql/connector/__init__.py", line 179, in connect
    return MySQLConnection(*args, **kwargs)
  File "/home/administrator/.local/lib/python3.8/site-packages/mysql/connector/connection.py", line 95, in __init__
    self.connect(**kwargs)
  File "/home/administrator/.local/lib/python3.8/site-packages/mysql/connector/abstracts.py", line 713, in connect
    self.config(**kwargs)
  File "/home/administrator/.local/lib/python3.8/site-packages/mysql/connector/abstracts.py", line 303, in config
    raise AttributeError("Unsupported argument '{0}'".format(key))
AttributeError: Unsupported argument 'username'

Junkhacker

@brakcounty from a bit of googling, it looks like maybe something changed in the module. where i have “username” you should replace with “user”

DBCountMan

@junkhacker Cool thanks. I only know some coding and scripting but nothing about python. I replaced “username” with “user” as you instructed and got a new error this time:

Traceback (most recent call last):
  File "checkimages.py", line 4, in <module>
    mydb = mysql.connector.connect(
  File "/home/administrator/.local/lib/python3.8/site-packages/mysql/connector/__init__.py", line 179, in connect
    return MySQLConnection(*args, **kwargs)
  File "/home/administrator/.local/lib/python3.8/site-packages/mysql/connector/connection.py", line 95, in __init__
    self.connect(**kwargs)
  File "/home/administrator/.local/lib/python3.8/site-packages/mysql/connector/abstracts.py", line 716, in connect
    self._open_connection()
  File "/home/administrator/.local/lib/python3.8/site-packages/mysql/connector/connection.py", line 208, in _open_connection
    self._do_auth(self._user, self._password,
  File "/home/administrator/.local/lib/python3.8/site-packages/mysql/connector/connection.py", line 137, in _do_auth
    packet = self._protocol.make_auth(
  File "/home/administrator/.local/lib/python3.8/site-packages/mysql/connector/protocol.py", line 99, in make_auth
    packet += self._auth_response(client_flags, username, password,
  File "/home/administrator/.local/lib/python3.8/site-packages/mysql/connector/protocol.py", line 58, in _auth_response
    auth = get_auth_plugin(auth_plugin)(
  File "/home/administrator/.local/lib/python3.8/site-packages/mysql/connector/authentication.py", line 190, in get_auth_plugin
    raise errors.NotSupportedError(
mysql.connector.errors.NotSupportedError: Authentication plugin 'caching_sha2_password' is not supported

Does python think the “password” is a hash and not an actual password?

Junkhacker

@brakcounty i don’t know what’s going on, but make sure you don’t have any typos and that you’re running the script with python3

DBCountMan

This post is deleted!

DBCountMan

Got it. Script works. Thanks!

How to rescan /images/ to repopulate FOG Web UI>List all Images page

227

12.0k

17.3k

155.2k