HP DL380G8 Proxmox and shadow Gaming

Introduction

Subject: use a Windows 10 VM under Proxmox to play a streamed game - in my case Skyrim and Baldur's gate 3

we will cover:

DL380G8p installation
DL380G8p upgrade
Using a Nvidia RTX3060ti graphic card in a DL380G8
Streaming

-

DL380G8p installation

BIOS Configuration

Acces to BIOS : BIOS Setup/RBSU (press F9).

Config Serveur HP (P/N 653200-B21)

System option : Intel VT

System option : Hyper Threading : enable

System option : Processor Core Disable : 6

System option : Intel turbo boot : enable

System option : smart memory enable

System option : Intel VT-d :enable

System option : SATA :embedded SATA config : Enable AHCI

System option : SATA :Drvie write cache : Enable

PCI devices=>Embedded SATA=>Disable

PCI devices=>FlexiLOM1 : ethernet 1G 4 port 331FLR =>Disable

PCI devices=> NP NC364T : enable

Advanced Options=>SRV_IOV:enable

Advanced Options=>Video Options=>(3rd option) Embedded Video Primary, Optional Video Secondary

Advanced Options=>Advanced Performance =>133MH support : Disable (memoire en 1333MHz)

Advanced Options=>Advanced Performance =>ACPI split :enable

Notes: silencing

https://www.reddit.com/r/homelab/comments/ba56lf/silencing_a_hp_dl380_gen8/

Ilo4

Reset Password

Connect the server to a local network.

Access the server’s local console.

Power the system on.

Press the F8 key to access Ilo Configuration mode.

Select the Administrator account.

Choose the forgotten password option.

Enter the new password of your choice.

Press Enter to confirm the new password.

Install Proxmox 8.1

We do a simple install from USB key or ISO via iLo Card

https://pve.proxmox.com/wiki/Prepare_Installation_Media

Upgrade Firmware

as root :

Firmware Name	Firmware Version	source	Installation cmd
HP Ethernet 1Gb 4-port 331FLR Adapter	17.4.41=> 20.23.41	https://support.hpe.com/connect/s/softwaredetails?softwareId=MTX_21384dc0325349feb3af83a8ef	alien firmware-nic-broadcom-2.31.0-1.1.x86_64.rpm --scripts dpkg -i firmware-nic-broadcom_2.31.0-2.1_amd64.deb cd /usr/lib/x86_64-linux-gnu/firmware-nic-broadcom-2.31.0-1.1/ chmod +x setup ./setup
iLO	v 2.82 2 Mars 2023	https://support.hpe.com/connect/s/softwaredetails?language=fr&softwareId=MTX_47590c9ba0974132ba3946e21d	rpm2cpio firmware-ilo4-2.81-1.1.i386.rpm \| cpio -id cd /root/usr/lib/i386-linux-gnu/firmware-ilo4-2.81-1.1 ./hpsetup
Intelligent Platform Abstraction Data	5.43
Intelligent Provisioning	1.63.192
Power Management Controller Firmware	3.3
Power Management Controller Firmware Bootloader	2.7
Redundant System ROM	P70 07/01/2015	hp-firmwaresystem-p70-2015.07.01-1.1.i386.rpm http://repos.storage37.net/hpsum/baseline/gen8.1/swpackages/hp-firmware-system-p70-2015.07.01-1.1.i386.rpm
SAS Programmable Logic Device	Version 0x0C
Server Platform Services (SPS) Firmware	2.1.7.E7.4
Smart Array P420i Controlle	8.32 2017-11-02	https://support.hpe.com/hpesc/public/swd/detail?swItemId=MTX_42b6aa58956a438aa85bd73d0f	Extract the contents of the rpm using the command: rpm2cpio <rpm_filename>.rpm \| cpio -id. This extracts the contents of the rpm to 'usr' folder in the current directory Navigate to the 'usr/lib/x86_64-linux-gnu/firmware-smartarray-46a4d957a7-' location and run ./setup to update the firmware. The actual name of the 'firmware-smartarray-46a4d957a7-' folder is based on the version of the rpm installed in the step above The results of the firmware upgrade are saved to the Component.log file, located in the /var/cpq/ directory
System Programmable Logic Device Version	0x32
System ROM	P70 05/24/2019	https://support.hpe.com/connect/s/softwaredetails?softwareId=MTX_a300316c0840482a976d843730&language=fr SP99438.exe https://support.hpe.com/connect/s/softwaredetails?language=fr&softwareId=MTX_f999a22b47af43ca897c0835be firmware-system-p70-2019.05.24-1.1.i386.rpm http://132.187.1.14/transfer/SPP_Gen8.1/firmware-system-p70-2019.05.24-1.1.i386.rpm https://support.hpe.com/connect/s/softwaredetails?language=en_US&softwareId=MTX_f999a22b47af43ca897c0835be	rpm2cpio firmware-system-p70-2019.05.24-1.1.i386.rpm \| cpio -idmv cd usr/lib/i386-linux-gnu/firmware-system-p70-2019.05.24-1.1 ./hpsetup
System ROM Bootblock	03/05/2013
Physical disk (3a,3b,4b) EG0600FBDBU	HPDA(C) (24 oct. 2016)	https://support.hpe.com/connect/s/softwaredetails?language=fr&softwareId=MTX_3ce712a9596a433db7d1976930&tab=revisionHistory hp-firmware-hdd-f1eafd9715-HPDA-3.1.x86_64.rpm (4.1 MB)
Physical disk (4a) EG0600FCVBK
Physical disk (1a&2a&1b)EG0600FBLSH	HDP7=>HPD8(B) 24/10/2016	https://support.hpe.com/connect/s/softwaredetails?language=fr&softwareId=MTX_c5e1962178d24aa197f14f6154 hp-firmware-hdd-815e5e101b-HPD8-2.1.x86_64.rpm
Physical disk (2b) EG0600FBVFP		https://support.hpe.com/connect/s/softwaredetails?language=fr&softwareId=MTX_0de208f0a8444cc9a5ab5352cc
HPE Ethernet 10Gb 2-port 530FLR-SFP+ Adapter	2.18.44	hp-firmware-nic-qlogic-nx2-2.18.44-1.1.x86_64.rpm https://support.hpe.com/connect/s/softwaredetails?language=fr&softwareId=MTX_c739aea95c1847f6a9e7c3b94c

Upgrade CPU Intel 26xxv2

Upgrade to support th use of newer CPU

Ref: https://www.youtube.com/watch?v=GXattxVpSAI

Ref : https://support.hpe.com/hpesc/public/docDisplay?docId=c03911173

we need to upgrade the BIOS :

1- Download the ISO, boot it, upgrade, reboot (around 10 minutes)

https://downloads.hpe.com/pub/softlib2/software1/cd/p355461430/v96920/hp-ivb_update-1.80-2.iso

2- fisrt boot is long on particular on step 114 (30% boot)

Proxmox

Helper

from : Proxmox Helper

remove nag screen : past this in terminal

bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/misc/post-pve-install.sh)"

add Intell drivers:

bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/misc/microcode.sh)"

HP SmartArray

Replace HP ACU to configure Array from OS

Add sources:

echo "deb http://downloads.linux.hpe.com/SDR/repo/mcp stretch/current non-free" > /etc/apt/sources.list.d/hp-mcp.list

wget -q -O - http://downloads.linux.hpe.com/SDR/hpPublicKey1024.pub | apt-key add -

wget -q -O - http://downloads.linux.hpe.com/SDR/hpPublicKey2048.pub | apt-key add -

wget -q -O - http://downloads.linux.hpe.com/SDR/hpPublicKey2048_key1.pub | apt-key add -

wget -q -O - http://downloads.linux.hpe.com/SDR/hpePublicKey2048_key1.pub | apt-key add -

apt update

Install :
apt install ssacli

Exemple :https://gist.github.com/mrpeardotnet/a9ce41da99936c0175600f484fa20d03

ssacli ctrl all show

Smart Array P420i in Slot 0 (Embedded) (sn: 00143xxxxxx)

ssacli ctrl all show status

Smart Array P420i in Slot 0 (Embedded)

Controller Status: OK

Cache Status: Permanently Disabled

Battery/Capacitor Status: Failed (Replace Batteries/Capacitors)

Desactivation cache from card:

Smart Array P420i in Slot 0 (Embedded)

Controller Status: OK

Cache Status: Not Configured

Battery/Capacitor Status: Recharging

root@pve:~# ssacli "ctrl slot=0 show" |grep Cache

Cache Serial Number: PBKUA0BRH3XCEB

Wait for Cache Room: Disabled

Cache Board Present: True

Cache Status: Not Configured

Drive Write Cache: Enabled

Total Cache Size: 0.5

Total Cache Memory Available: 0.3

Battery Backed Cache Size: 0.3

No-Battery Write Cache: Disabled

Cache Backup Power Source: Capacitors

Cache Module Temperature (C): 36

Cache Activation

ctrl slot=0 ld all show detail

Smart Array P420i in Slot 0 (Embedded)

Array A

Logical Drive: 1

Size: 256.00 GB

Fault Tolerance: 5

Heads: 255

Sectors Per Track: 32

Cylinders: 65535

Strip Size: 512 KB

Full Stripe Size: 3584 KB

Status: OK

Unrecoverable Media Errors: None

Caching: Disabled

Parity Initialization Status: Queued

Unique Identifier: 600508xxxxxx

Disk Name: /dev/sda

Mount Points: None

Logical Drive Label: A504B2Cxxxxxxxx

Drive Type: Data

LD Acceleration Method: All disabled

ctrl slot=0 ld all modify arrayaccelerator=enable

=> ctrl slot=0 ld all show detail

Smart Array P420i in Slot 0 (Embedded)

Array A

Logical Drive: 1

Size: 256.00 GB

Fault Tolerance: 5

Heads: 255

Sectors Per Track: 32

Cylinders: 65535

Strip Size: 512 KB

Full Stripe Size: 3584 KB

Status: OK

Unrecoverable Media Errors: None

Caching: Enabled

Parity Initialization Status: Queued

Unique Identifier: 600508B1001C77A8E9390A307E6D3DA7

Disk Name: /dev/sda

Mount Points: None

Logical Drive Label: A504B2CA001438025BB5A000A6F

Drive Type: Data

LD Acceleration Method: Controller Cache

Convert from RAID 0 to To RAID1

1 disk assigned to Logical drive 1 and unused :

root@pve:~# ssacli ctrl slot=0 pd all show

Smart Array P420i in Slot 0 (Embedded)

Array B

physicaldrive 1I:0:1 (port 1I:box 0:bay 1, SAS HDD, 6 TB, OK)

Unassigned

physicaldrive 1I:0:4 (port 1I:box 0:bay 4, SAS HDD, 6 TB, OK)

Change rebuild priority and add a disk in Logical drive.

ssacli ctrl slot=0 modify rebuildpriority=high

ssacli ctrl slot=0 ld 2 add drives=allunassigned

ssacli ctrl slot=0 pd all show

Smart Array P420i in Slot 0 (Embedded)

Array B

physicaldrive 1I:0:1 (port 1I:box 0:bay 1, SAS HDD, 6 TB, OK)

physicaldrive 1I:0:4 (port 1I:box 0:bay 4, SAS HDD, 6 TB, OK)

root@pve:~# ssacli ctrl slot=0 ld 2 show

Smart Array P420i in Slot 0 (Embedded)

Array B

Logical Drive: 2

Size: 5.46 TB

Fault Tolerance: 0

Heads: 255

Sectors Per Track: 32

Cylinders: 65535

Strip Size: 256 KB

Full Stripe Size: 512 KB

Status: Transforming, 0.05% complete

Caching: Enabled

Unique Identifier: 600508xxxxxxxxxxxxxxxxxxxx

Disk Name: /dev/sdb

Mount Points: None

Drive Type: Data

LD Acceleration Method: Controller Cache

Change RAID structure from Raid0 à RAID1, becasue half space used

ssacli ctrl slot=0 ld 2 modify raid=1

Proxmox : PCI Passthrough

change: /etc/default/grub

The Iommu is to activate passThru, the 2nd to split PCIe Canals

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on pcie_acs_override=downstream,multifunction"

and

update-grub

echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf

Check graphic card :

lspci -n -s 04:00

04:00.0 0300: 10de:1d01 (rev a1)

04:00.1 0403: 10de:0fb8 (rev a1)

We have the video part and the audio part, we pass everything

echo "options vfio-pci ids=10de:1d01,10de:0fb8" > /etc/modprobe.d/vfio.conf

Proxmox PCI passthrough an RMRR issue

With PVE kernel before 6.5.10-1 we have to use a relaxable RMRR kernel.

https://github.com/Aterfax/relax-intel-rmrr/releases

PCI Passthroug with non RTX card

We have to extract the ROM from graphic card ad copy it to /usr/share/kvm/

add add it to the VM

hostpci0: 01:00,pcie=1,romfile=<GTX1030>.rom