Proxmox storage with TrueNAS iSCSI volumes

A few weeks ago, I decided to wipe clean my entire lab setup running Proxmox 6.2. I wanted to connect the latest version of Proxmox VE 8.0-2 using iSCSI LUNs from the TrueNAS® system I have with me. I thought it would be fun to have the configuration steps and the process documented. This is my journal on how to provision a TrueNAS® CORE iSCSI LUN to Proxmox storage. This iSCSI volume in Proxmox is where the VMs will be installed into.

Here is a simplified network diagram of my setup but it will be expanded to a Proxmox cluster in the future with the shared storage.

Proxmox and TrueNAS network setup

Preparing the iSCSI LUN provisioning

The iSCSI LUN (logical unit number) is provisioned as a logical disk volume to the Proxmox node, where the initiator-target relationship and connection are established.

This part assumes that a zvol has been created from the zpool. At the same time, the IQN (iSCSI Qualified Name) should be known to the TrueNAS® storage as it establishes the connection between Proxmox (iSCSI initiator) and TrueNAS (iSCSI target).

The IQN for Proxmox can be found by viewing the content of the /etc/iscsi/initiatorname.iscsi within the Proxmox shell as shown in the screenshot below.

Where to find the Proxmox iSCSI IQN

The green box shows the IQN number of the Proxmox node that starts with iqn.year-month.com.domain:generated-hostname. This will be used during the iSCSI target portal configuration in the TrueNAS® webGUI.

Provisioning and “sharing” the iSCSI LUN 

TrueNAS® uses the term “Block Shares” in its navigation panel. Technically it is not exactly sharing but most users who aren’t enterprise-minded are not familiar with the term iSCSI LUN provisioning anyways. The best way to create the iSCSI LUN is using the Wizard, a button on the top right corner of the WebGUI.

Here are the following screenshots in creating and provisioning the iSCSI LUN.

Select Block Shares (iSCSI)

(1) Give the iSCSI LUN a name (2) Select the path to the zvol

(1) Create a new portal. (2) Provide the target portal IP address with port 3260 (Do not change this number)

(1) Link the iSCSI initiator to the target Portal. Provide the Proxmox IQN 2) Partition the network to just the allowed hosts of the subnet for basic security segmentation.

Confirm the configuration details. Submit to initiate the creation of iSCSI LUN and iSCSI target portal

We can confirm that the iSCSI LUN has been created by checking the Targets tab in the webGUI.

Confirming the iSCSI LUN has been created and is provisioned through the iSCSI target portal

Connecting the Proxmox storage to TrueNAS iSCSI LUN

The iSCSI LUN is ready to be received at the Proxmox side. It is created with the Proxmox GUI as an iSCSI volume.

Datacenter > Storage > Add > iSCSI

Configuring the details of iSCSI volume in Proxmox

Proxmox iSCSI volume created

The iSCSI volume is shown in the navigation panel on the left

Creating the VM on the iSCSI volume

When both the Proxmox and the TrueNAS® are both configured and ready, it is time to use the iSCSI volume as the central storage repository for the VMs. Here is one example of creating the VM.

Setting the disk in VM creation to the iSCSI volume

The VM disk (vDisk) for the Virtual Machine

Installing Ubuntu in the VM

Once the VM has been created, the next logical step is to install a guest OS. In my case, the quick and easy way is to install Ubuntu since I have the ISO already in the OS image repository.

Installing Ubuntu in the VM as a guest OS

That’s it!

Setting this up for the Enterprise

While this is fun and good for a home lab, putting this together for the datacenter takes another level of skillset. The architecture design and workload sizing become a vital component in deploying Proxmox and TrueNAS® in a business resilient environment. One of the most important points to consider is Availability. It can be viewed as systems availability and also data availability.

Systems availability is to enable the high availability infrastructure components of the Proxmox compute layer, the networking and the HA storage with multipath I/O. The diagram below would be something most enterprise organizations should aspire to.

A Proxmox 3-node cluster with a TrueNAS HA Enterprise storage

Going into this level would require further understanding of the Proxmox Cluster Filesystem (pmxcfs), something I understand well in theory. Anyway, I hope to have a chance to explore this when I have the right dual controller TrueNAS® HA system in place. But until then, this is something that is easy to do from an administration and operations point of view.

I wish all readers happy hacking!

Tagged , , , , , , , , , . Bookmark the permalink.

About cfheoh

I am a technology blogger with 30 years of IT experience. I write heavily on technologies related to storage networking and data management because those are my areas of interest and expertise. I introduce technologies with the objectives to get readers to know the facts and use that knowledge to cut through the marketing hypes, FUD (fear, uncertainty and doubt) and other fancy stuff. Only then, there will be progress. I am involved in SNIA (Storage Networking Industry Association) and between 2013-2015, I was SNIA South Asia & SNIA Malaysia non-voting representation to SNIA Technical Council. I currently employed at iXsystems as their General Manager for Asia Pacific Japan.

7 Responses to Proxmox storage with TrueNAS iSCSI volumes

  1. Pingback: Random Short Take #89 | PenguinPunk.net

  2. Gabriel says:

    Hi cfheoh,
    I was following your guide and prior creating the iscsi share I did create the zvol – 5TB on one of my zpool. The problem is when I create the VM, it wants to use the whole 5TB as opposed to allow me to set up the VM disk size during the VM creation wizard.
    Thoughts?

    • cfheoh says:

      Hello Gabriel

      Thanks for sending me your experience. The zvol will take up the size of the zpool if there is no quota set at the zvol level. When the zvol is presented to the VM, it will of course, see the entire zvol capacity. Set a quota for the zvol, and see how it turns out.

      All the best. 😉

  3. Gabriel says:

    Hi cfheoh,
    I discovered that after adding the iscsi target on proxmox, add and LVM on top, it makes it behave as a local storage and then I can manually change the vm disk to my liking during the vm creation as I normally do when I use the local storage. I hope that makes sense.
    Cheers.

    • cfheoh says:

      HI Gabriel,

      That is absolutely brilliant. The zvol (a logical structure within TrueNAS) is presented as an iSCSI LUN which is then provisioned on the TCP/IP-Ethernet network to the Proxmox as a datastore. The Proxmox datastore is then “sliced and diced” to a be virtual disk to the VM running in the Proxmox host.

      Hope this satisfies your requirements. Keep it up! All the best!

      BTW, mind telling me where you are at? Which country you are from? I am in Malaysia. 😉 You can find me on LinkedIn to be connected professionally.

      Thank you and best regards

  4. Shaine says:

    is TrueNAS a virtual-software Environment running of VM on Proxmox, or a hardware attached to the Proxmox host?

    • cfheoh says:

      Hello Shaine

      In my case, a separate TrueNAS system is provisioning an iSCSI LUN to the Proxmox host, and the VMs can basically carve its own virtual disks from the datastore (from the iSCSI LUN) in Proxmox.

      I know of many users (especially the less enterprise types) who create a TrueNAS VM inside Proxmox. This is absolutely okay too, but less efficient from the OpenZFS point-of-view. If you have a Proxmox host with multiple physical drives that are not attached to Proxmox, you can enable PCI passthrough to all the TrueNAS VM to access the physical drives in the Proxmox server. OpenZFS would love it because the storage drives layer is less obfuscated, and OpenZFS would have better control and reliability in an enterprise setting.

      All the best and thank you for your support.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.