• Recent
    • Unsolved
    • Tags
    • Popular
    • Users
    • Groups
    • Search
    • Register
    • Login

    Windows server as FOG Storage Node - proof of concept blog

    Scheduled Pinned Locked Moved
    Tutorials
    4
    17
    10.6k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • george1421G
      george1421 Moderator
      last edited by

      (place holder)

      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

      1 Reply Last reply Reply Quote 1
      • george1421G
        george1421 Moderator
        last edited by george1421

        Well my first round of tests have failed, but also did add some light onto what I suspected. The storage node needs a web server running because the master storage node makes some calls to the storage node for image deployment. The following is a call I trapped on the IIS server.

        /fog/management/index.php node=client&sub=wakeEmUp&mac=00:50:56:94:f5:21
        

        and

        /fog/status/getimages.php path=%2Fimages
        

        At this time I don’t think the windows server (or a synology nas) will function as a storage node out of the box. I could load apache or the php plugin for IIS and get it to work. But I’m not sure if its worth the effort.

        I can setup nfs on windows, I can setup ftp and the images will replicate to the windows server. But without php and the rest of the storage node code, I’m at a standstill.

        As it stands now, this post https://wiki.fogproject.org/wiki/index.php/Windows_Storage_Node will not work either and should be deprecated.

        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

        1 Reply Last reply Reply Quote 1
        • Wayne WorkmanW
          Wayne Workman @george1421
          last edited by

          @george1421 said:

          Determine if the FTP service is really required for a storage node if we will only capture to the master node

          If you want your images to be replicated to the node automatically, it’s absolutely required.

          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
          Daily Clean Installation Results:
          https://fogtesting.fogproject.us/
          FOG Reporting:
          https://fog-external-reporting-results.fogproject.us/

          1 Reply Last reply Reply Quote 0
          • george1421G
            george1421 Moderator
            last edited by george1421

            Tom added some new code checks (30-Mar-16) that will have FOG revert back to the 1.2.0 way of checking for images if the storage node doesn’t respond properly to the http query. This opens the door again to proving if a windows storage node is possible.

            Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

            Tom ElliottT 1 Reply Last reply Reply Quote 1
            • Tom ElliottT
              Tom Elliott @george1421
              last edited by

              @george1421 just a slight correction. It is not using 1.2.0 method of checking. That is way gone and the new method is faster and simpler. Each node is aware of the images they have available to them now where 1.2.0 just checked if it existed on the node it was to pull from. This would cause problems in 1.2.0 because the master was checked at task creation. The client would load up all the way and then if it was pointed at a node that didn’t have the image would fail quite dramatically. Now boot will only boot to a node that has the image on it.

              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

              Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

              Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

              1 Reply Last reply Reply Quote 1
              • Wayne WorkmanW
                Wayne Workman
                last edited by

                #wiki worthy

                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                Daily Clean Installation Results:
                https://fogtesting.fogproject.us/
                FOG Reporting:
                https://fog-external-reporting-results.fogproject.us/

                1 Reply Last reply Reply Quote 0
                • george1421G
                  george1421 Moderator
                  last edited by george1421

                  Part 1b

                  Here is a rough outline on how to do this same thing with Windows 2008 R2. I hope to be able to test this as soon as I get my test environment rebuilt.

                  The following executed correctly on Windows 2008 R2 with Windows Managment Framework installed which installs PowerShell 3.0 I’m actually not using very many powershell commands here, basically just to add windows roles and features. Powershell on Windows 2008 is a bit lacking as compared to 2012.

                  Set-ExecutionPolicy Unrestricted -Scope CurrentUser
                  Import-Module ServerManager
                  
                  Add-WindowsFeature FS-FileServer
                  Add-WindowsFeature FS-NFS-Services
                  
                  mkdir c:\share
                  mkdir c:\share\tftpboot
                  mkdir c:\share\images
                  mkdir c:\share\snapins
                  mkdir c:\share\snapins\ssl
                  
                  net localgroup fog_users /add
                  net user fog_user "mi5ty_cl0ud" /add /EXPIRES:NEVER /PASSWORDCHG:NO /active:YES /Y
                  net localgroup fog_users fog_user /add
                  icacls c:\share /grant "fog_users:M"
                  
                  *****
                  
                  nfsshare images=c:\share\images -o rw sec=sys root unmapped=yes
                  netsh advfirewall firewall set rule group="Server for NFS" new enable=Yes
                  
                  *************
                  
                  On FOG server
                  mount -t nfs 192.168.1.205:/images /mnt
                  mkdir /mnt/dev
                  touch /mnt/.mntcheck
                  touch /mnt/dev/.mntcheck
                  umount /mnt
                  
                  ****************
                  
                  # On windows server
                  Add-WindowsFeature Web-WebServer,Web-Common-Http,Web-FTP-Server,Web-FTP-Service,Web-FTP-Ext,Web-Mgmt-Console
                  
                  Import-Module WebAdministration
                  New-WebFtpSite -Name "FOGFtpSite" -Port 21 -PhysicalPath "c:\share" -IPAddress "192.168.1.205" 
                  
                  Set-ItemProperty "IIS:\Sites\FOGFtpSite" -Name ftpServer.security.ssl.controlChannelPolicy -Value 0
                  Set-ItemProperty "IIS:\Sites\FOGFtpSite" -Name ftpServer.security.ssl.dataChannelPolicy -Value 0
                  Set-ItemProperty "IIS:\Sites\FOGFtpSite" -Name ftpServer.security.authentication.basicAuthentication.enabled -Value $true
                  Set-ItemProperty "IIS:\Sites\FOGFtpSite" -Name ftpserver.userisolation.mode -Value 4
                  Add-WebConfiguration "/system.ftpServer/security/authorization" -value @{accessType="Allow";roles="fog_users";permissions="Read,Write";users=""} -PSPath IIS:\ -location "FOGFtpSite"
                  Restart-WebItem "IIS:\Sites\FOGFtpSite"
                  
                  *********************************
                  
                  New-Item "IIS:\Sites\Default Web Site\fog" -type Directory
                  New-Item "IIS:\Sites\Default Web Site\fog\service" -type Directory
                  New-Item "IIS:\Sites\Default Web Site\fog\service\ipxe" -type Directory
                  
                  ********************************
                  
                  In IIS management for the default web site must update mime types to add “.*” (dot star without the quotes) and with a type of “application/octet-stream”
                  
                  ********************************
                  
                  #Now that IIS is all setup and ready you will need to copy all of the files from /var/www/html/fog/service/ipxe to the IIS server in the windows path IIS:\Sites\Default Web Site\fog\services\ipxe
                  #Lets do this with nfs. Create the NFS share on the Windows server
                  
                  nfsshare fogipxe=C:\inetpub\wwwroot\fog\service\ipxe -o rw sec=sys root unmapped=yes
                  
                  ****************
                  
                  On FOG server
                  mount -t nfs 192.168.1.205:/fogipxe /mnt
                  cp /var/www/html/fog/service/ipxe/* /mnt
                  umount /mnt
                  
                  ****************
                  
                  # On Windows server, remove the nfs share
                  nfsshare fogipxe /delete
                  
                  (Note: you need to "replace all child objects permission with inheritable permissions from this object" on C:\inetpub\wwwroot\fog\service\ipxe or the bg download test will fail
                  # need to figure out the icacls command on this one
                  
                  Install the TFTP service as outlined in the OP
                  
                  

                  Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                  1 Reply Last reply Reply Quote 1
                  • george1421G
                    george1421 Moderator
                    last edited by george1421

                    Part 2

                    For the remainder of this thread I’ll continue with the Windows 2008 R2 server as the windows storage node. If time permits I’ll circle back and attempt the same thing with 2012, but based on what I’ve done so far I have confidence that 2012 will work equally as well as 2008.

                    There is one bit of error data I found running through a QC check before moving onto the next part. I created the FTP site using the powershell command: New-WebFtpSite -Name "FOGFtpSite" -Port 21 -PhysicalPath "c:\share" -IPAddress "192.168.1.205" While testing I found that the -PhysicalPath switch did not update the ftp server setting causing the ftp server to report that the home folder could not be located. I went into the mmc console for IIS and manually set the homedrive for the ftp server. Once that was done I was able to login via FTP.

                    On to the QC checks before moving on to the FOG setup of this. For these tests we’ll use a windows 7 workstation.

                    First we need to test to see if the FTP server is working and we can login using the fog_user account we setup.

                    C:\>ftp 192.168.1.205
                    Connected to 192.168.1.205.
                    220 Microsoft FTP Service
                    User (192.168.1.205:(none)): fog_user
                    331 Password required for fog_user.
                    Password:
                    230 User logged in.
                    

                    Just issue a dir command to see if we can see the files we created.

                    ftp> dir
                    200 PORT command successful.
                    125 Data connection already open; Transfer starting.
                    05-22-16  08:39PM       <DIR>          images
                    05-22-16  07:41PM       <DIR>          snapins
                    05-24-16  02:42PM       <DIR>          tftpboot
                    226 Transfer complete.
                    ftp: 144 bytes received in 0.00Seconds 144000.00Kbytes/sec.
                    quit
                    

                    That completes the FTP test

                    The next step is to test the tftp server

                    C:\>tftp 192.168.1.205 get default.ipxe
                    Transfer successful: 427 bytes in 1 second, 427 bytes/s
                    

                    If your tests matches the above then we can move onto the FOG part of the setup.

                    My dev environment was created fresh with the latest trunk version of FOG 1.2.0. The following is the complete steps I went through to setup FOG to communicate with the windows storage node.

                    Fog Configuration -> Plugin System
                    FOG_PLUGINSYS_ENABLED = checked

                    Plugin Managment
                    Activate and install Location management plugin

                    Storage Management -> Add Storage node
                    Name: WinStoreNode
                    IP Address: 192.168.1.205
                    Web Root: /fog
                    Max Clients: 10
                    Is Master Node: (unchecked)
                    Replication Bandwidth: 0
                    Storage Group: default
                    Image Path: /images
                    FTP Path: /images
                    Snapin Path: /snapins
                    SSL Path: /snapins/ssl
                    Bitrate: (blank)
                    Interface: eth0
                    Is Enabled: (checked)
                    Is Graph enabled: (unchecked)
                    Managment Username: fog_user
                    Managment Password: mi5ty_cl0ud

                    Location Managment -> Create New Location
                    Location Name: winsn
                    Storage Group: Default
                    Storage Node: WinStoreNode
                    Use inits and kernels from this node: (checked)
                    Press the Add button

                    I next copied a host image from my production server to the dev environment. Once all of the files were in place, I went back to the FOG GUI and added an Image reference that defined the image I copied over from my production server.

                    And finally I restarted the FOGImageReplication service. Inspecting the /opt/fog/log directory you should see the replication service start up with a transfer log specifically for the windows storage node we just created (i.e. fogreplicator.log.transfer.WinStoreNode.log). If you see this long file, hop over to your windows storage node and inspect the directory c:\share\images to see if the image files have started to copy over from the FOG server.

                    Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                    V 1 Reply Last reply Reply Quote 1
                    • george1421G
                      george1421 Moderator
                      last edited by george1421

                      Part 3 (Final)

                      Today I went in and registered one of my VMs on the dev fog server. I ran through the full registration and allocated this host to the “winsn” location.

                      I pxe booted the vm and saw that it initially connected to the dev fog server and then the dev fog server redirected the client to pull the bzImage and init.xz from the Windows storage node (!!getting excited level 20% !!).

                      The vm downloaded both bzImage and the init from the windows storage node cleanly (excitement level 50%).

                      I saw the FOS kernel boot and started the initial prep work for imaging (excitement level 80%).

                      Partclone did its prep work and started downloading the image (excitement level 90%). I quickly checked the windows storage node and ran netstat -an and this was the the line I was searching for

                      TCP    192.168.1.205:2049     192.168.1.6:747      ESTABLISHED
                      

                      This told me that the windows storage node was communicating with an external (dhcp) device over the NFS protocol!!

                      Imaging completed and the workstation started booting windows (time for a smoke and a beer) [full disclosure I don’t smoke so I’ll just take two beers instead].

                      So it IS possible to setup a FOG / MS Windows Storage node. In the end I’m still not seeing the value in it, but it IS DOABLE.

                      !!Success!! and the end of this POC project…

                      Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                      Wayne WorkmanW 1 Reply Last reply Reply Quote 1
                      • Wayne WorkmanW
                        Wayne Workman @george1421
                        last edited by

                        @george1421 you never cease to amaze.

                        Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!
                        Daily Clean Installation Results:
                        https://fogtesting.fogproject.us/
                        FOG Reporting:
                        https://fog-external-reporting-results.fogproject.us/

                        george1421G 1 Reply Last reply Reply Quote 1
                        • george1421G
                          george1421 Moderator @Wayne Workman
                          last edited by

                          @Wayne-Workman

                          Right now this post is a jumbled up mess. I’m going to break down my POC environment and build it again. I’ll verify the settings are correct and post a concise kb one for 2008 and one for 2012. But in theory it does work.

                          Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                          1 Reply Last reply Reply Quote 0
                          • V
                            vkenny @george1421
                            last edited by

                            @george1421 Where is “Location Management” is that relating to an older version of FOG that no longer has this or am I completely missing something?

                            Tom ElliottT 1 Reply Last reply Reply Quote 0
                            • Tom ElliottT
                              Tom Elliott @vkenny
                              last edited by

                              @vkenny Location Management comes from a Plugin that’s known as the “Location Plugin”.

                              To install the Location Plugin first goto:

                              FOG Configuration Page->FOG Settings->Plugin Settings->FOG_PLUGIN_ENABLE
                              Check the box, press save. You’ll see a new menu item appear in the Main menu system. Click on this new menu item.

                              You will be presented with a list of plugins.
                              Click “Location” icon
                              Click “Activate Plugins”
                              Click “Location” icon
                              Click “Install plugin”

                              You will see the new Location menu item appear in your menu.

                              Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG! Get in contact with me (chat bubble in the top right corner) if you want to join in.

                              Web GUI issue? Please check apache error (debian/ubuntu: /var/log/apache2/error.log, centos/fedora/rhel: /var/log/httpd/error_log) and php-fpm log (/var/log/php*-fpm.log)

                              Please support FOG if you like it: https://wiki.fogproject.org/wiki/index.php/Support_FOG

                              george1421G 1 Reply Last reply Reply Quote 0
                              • george1421G
                                george1421 Moderator @Tom Elliott
                                last edited by george1421

                                @Tom-Elliott I’m currently spinning up a new FOG 1.4.0 server to test multicasting across subnets (and usb FOS booting it now appears). I’ll divert that setup to test FOG with a windows 2012 server setup as a storage node. I don’t have a centos template on this dev box so its going to take me some time to get up to speed. I do have a windows 2012 template so that one shouldn’t take too long.

                                I would still expect it to take until this evening before I can get to testing with my day job and everything…

                                Please help us build the FOG community with everyone involved. It's not just about coding - way more we need people to test things, update documentation and most importantly work on uniting the community of people enjoying and working on FOG!

                                1 Reply Last reply Reply Quote 0
                                • 1 / 1
                                • First post
                                  Last post

                                214

                                Online

                                12.0k

                                Users

                                17.3k

                                Topics

                                155.2k

                                Posts
                                Copyright © 2012-2024 FOG Project