A few weeks ago, I decided to wipe clean my entire lab setup running Proxmox 6.2. I wanted to connect the latest version of Proxmox VE 8.0-2 using iSCSI LUNs from the TrueNAS® system I have with me. I thought it would be fun to have the configuration steps and the process documented. This is my journal on how to provision a TrueNAS® CORE iSCSI LUN to Proxmox storage. This iSCSI volume in Proxmox is where the VMs will be installed into.
Here is a simplified network diagram of my setup but it will be expanded to a Proxmox cluster in the future with the shared storage.
Proxmox and TrueNAS network setup
Preparing the iSCSI LUN provisioning
The iSCSI LUN (logical unit number) is provisioned as a logical disk volume to the Proxmox node, where the initiator-target relationship and connection are established.
This part assumes that a zvol has been created from the zpool. At the same time, the IQN (iSCSI Qualified Name) should be known to the TrueNAS® storage as it establishes the connection between Proxmox (iSCSI initiator) and TrueNAS (iSCSI target).
The IQN for Proxmox can be found by viewing the content of the /etc/iscsi/initiatorname.iscsi within the Proxmox shell as shown in the screenshot below.
Where to find the Proxmox iSCSI IQN
The green box shows the IQN number of the Proxmox node that starts with iqn.year-month.com.domain:generated-hostname. This will be used during the iSCSI target portal configuration in the TrueNAS® webGUI.
Provisioning and “sharing” the iSCSI LUN
TrueNAS® uses the term “Block Shares” in its navigation panel. Technically it is not exactly sharing but most users who aren’t enterprise-minded are not familiar with the term iSCSI LUN provisioning anyways. The best way to create the iSCSI LUN is using the Wizard, a button on the top right corner of the WebGUI.
Here are the following screenshots in creating and provisioning the iSCSI LUN.
Select Block Shares (iSCSI)
(1) Give the iSCSI LUN a name (2) Select the path to the zvol
(1) Create a new portal. (2) Provide the target portal IP address with port 3260 (Do not change this number)
(1) Link the iSCSI initiator to the target Portal. Provide the Proxmox IQN 2) Partition the network to just the allowed hosts of the subnet for basic security segmentation.
Confirm the configuration details. Submit to initiate the creation of iSCSI LUN and iSCSI target portal
We can confirm that the iSCSI LUN has been created by checking the Targets tab in the webGUI.
Confirming the iSCSI LUN has been created and is provisioned through the iSCSI target portal
Connecting the Proxmox storage to TrueNAS iSCSI LUN
The iSCSI LUN is ready to be received at the Proxmox side. It is created with the Proxmox GUI as an iSCSI volume.
Datacenter > Storage > Add > iSCSI
Configuring the details of iSCSI volume in Proxmox
Proxmox iSCSI volume created
The iSCSI volume is shown in the navigation panel on the left
Creating the VM on the iSCSI volume
When both the Proxmox and the TrueNAS® are both configured and ready, it is time to use the iSCSI volume as the central storage repository for the VMs. Here is one example of creating the VM.
Setting the disk in VM creation to the iSCSI volume
The VM disk (vDisk) for the Virtual Machine
Installing Ubuntu in the VM
Once the VM has been created, the next logical step is to install a guest OS. In my case, the quick and easy way is to install Ubuntu since I have the ISO already in the OS image repository.
Installing Ubuntu in the VM as a guest OS
That’s it!
Setting this up for the Enterprise
While this is fun and good for a home lab, putting this together for the datacenter takes another level of skillset. The architecture design and workload sizing become a vital component in deploying Proxmox and TrueNAS® in a business resilient environment. One of the most important points to consider is Availability. It can be viewed as systems availability and also data availability.
Systems availability is to enable the high availability infrastructure components of the Proxmox compute layer, the networking and the HA storage with multipath I/O. The diagram below would be something most enterprise organizations should aspire to.
A Proxmox 3-node cluster with a TrueNAS HA Enterprise storage
Going into this level would require further understanding of the Proxmox Cluster Filesystem (pmxcfs), something I understand well in theory. Anyway, I hope to have a chance to explore this when I have the right dual controller TrueNAS® HA system in place. But until then, this is something that is easy to do from an administration and operations point of view.
I wish all readers happy hacking!
Pingback: Random Short Take #89 | PenguinPunk.net
Hi cfheoh,
I was following your guide and prior creating the iscsi share I did create the zvol – 5TB on one of my zpool. The problem is when I create the VM, it wants to use the whole 5TB as opposed to allow me to set up the VM disk size during the VM creation wizard.
Thoughts?
Hello Gabriel
Thanks for sending me your experience. The zvol will take up the size of the zpool if there is no quota set at the zvol level. When the zvol is presented to the VM, it will of course, see the entire zvol capacity. Set a quota for the zvol, and see how it turns out.
All the best. 😉
Hi cfheoh,
I discovered that after adding the iscsi target on proxmox, add and LVM on top, it makes it behave as a local storage and then I can manually change the vm disk to my liking during the vm creation as I normally do when I use the local storage. I hope that makes sense.
Cheers.
HI Gabriel,
That is absolutely brilliant. The zvol (a logical structure within TrueNAS) is presented as an iSCSI LUN which is then provisioned on the TCP/IP-Ethernet network to the Proxmox as a datastore. The Proxmox datastore is then “sliced and diced” to a be virtual disk to the VM running in the Proxmox host.
Hope this satisfies your requirements. Keep it up! All the best!
BTW, mind telling me where you are at? Which country you are from? I am in Malaysia. 😉 You can find me on LinkedIn to be connected professionally.
Thank you and best regards
is TrueNAS a virtual-software Environment running of VM on Proxmox, or a hardware attached to the Proxmox host?
Hello Shaine
In my case, a separate TrueNAS system is provisioning an iSCSI LUN to the Proxmox host, and the VMs can basically carve its own virtual disks from the datastore (from the iSCSI LUN) in Proxmox.
I know of many users (especially the less enterprise types) who create a TrueNAS VM inside Proxmox. This is absolutely okay too, but less efficient from the OpenZFS point-of-view. If you have a Proxmox host with multiple physical drives that are not attached to Proxmox, you can enable PCI passthrough to all the TrueNAS VM to access the physical drives in the Proxmox server. OpenZFS would love it because the storage drives layer is less obfuscated, and OpenZFS would have better control and reliability in an enterprise setting.
All the best and thank you for your support.