You should expect this to take time, as you are asking the portfilter to check all its rules against the firewall rules, which just is checking all the rules against itself.

Example:

I don't know how many rules you are dealing with, but on my standalone system:

($total = (Get-NetFirewallRule).count) 783

($portfilter = Get-NetFirewallPortFilter).Count 783

This means that is code is running 1566 times (on my system), because you are asking the filter each one of its own rules against all 783 portfilter rules against the 783 firewall rules to create your object. ForLoops are just slow, and with 1566 passes, in my case, well you should gather how much this will add up.

If you did this for just one firewall rule, you'd get something like:

Measure-Command {
$RuleCount = 0
$testcount = 0
($total = (Get-NetFirewallRule).count)
($portfilter = Get-NetFirewallPortFilter).Count

ForEach($Rule in (Get-NetFirewallRule | Select -First 1))
{    
  $portfilter = Get-NetFirewallPortFilter | 
  ForEach-Object{
  $testcount++
  $testcount

  [pscustomobject]@{
        DisplayName = $Rule.DisplayName
        Profile = $Rule.Profile 
        Action = $Rule.Action
        Direction = $Rule.Direction
        Protocol = $_.Protocol
        LocalPort = $_.LocalPort
        RemotePort = $_.RemotePort
        IcmpType = $_.IcmpType
        DynamicTarget = $_.DynamicTarget
      }
      return
  }  

  $RuleCount++
  $perc=Int

  Write-Progress -Activity 'My Important Activityssss' -PercentComplete $perc -Status $perc
}
}

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 2 ********* * times the total needed passes 
Milliseconds      : 414
Ticks             : 24149617
TotalDays         : 2.79509456018519E-05
TotalHours        : 0.000670822694444444
TotalMinutes      : 0.0402493616666667
TotalSeconds      : 2.4149617  **************
TotalMilliseconds : 2414.9617

If we tweak your code a bit more to show more information / progress, say like this...

Clear-Host
$total = (Get-NetFirewallRule).count
$total1 = (Get-NetFirewallPortFilter).Count

$RuleCount = 0

ForEach($Rule in (Get-NetFirewallRule | Select -First 3))
{    
    Write-host "Processing firewall rule $($Rule.Name)" -ForegroundColor Cyan


    $RuleCount++
    $perc = Int
    Write-Progress -Activity 'My Important firewall rules' -PercentComplete $perc -Status $perc -Id 1

    $testcount = 0

    Get-NetFirewallPortFilter | 
    ForEach-Object {
        Write-host "Processing port rule $($_.Name)" -ForegroundColor Yellow

        $testcount++
        $perc1 = Int
        Write-Progress -Activity 'My Important Port rules' -PercentComplete $perc1 -Status $perc1 -Id 2
    } 
    Write-Warning -Message "$testcount "
}


Processing firewall rule vm-monitoring-dcom
Processing port rule 
...
WARNING: 783 
Processing firewall rule vm-monitoring-icmpv4
Processing port rule 
...
WARNING: 783 
Processing firewall rule vm-monitoring-icmpv6
Processing port rule 
...
WARNING: 783 

… it should further illustrate what I am trying to say.

Then you have the limitations of your system itself, processor speed, memory resources / speed, any other processes you are running on your machine.

See this similar Q&A discussion:

How can I speed up PowerShell to get firewall rules on windows 10?

#Using a registry approach

param
( 
    [switch]$Local, 
    [switch]$GPO 
) 

# If no switches are set the script will default to local firewall rules 
if (!($Local) -and !($Gpo)) 
{ $Local = $true } 

$RegistryKeys = @() 

if ($Local) {$RegistryKeys += 'Registry::HKLM\System\CurrentControlSet\Services\SharedAccess\Parameters\FirewallPolicy\FirewallRules'} 
if ($GPO) {$RegistryKeys += 'Registry::HKLM\Software\Policies\Microsoft\WindowsFirewall\FirewallRules'} 

Foreach ($Key in $RegistryKeys) 
{ 
    if (Test-Path -Path $Key) 
    { 
        (Get-ItemProperty -Path $Key).PSObject.Members | 
        Where-Object {
        (@('PSPath','PSParentPath','PSChildName') -notcontains $_.Name) -and 
        ($_.MemberType -eq 'NoteProperty') -and 
        ($_.TypeNameOfValue -eq 'System.String')} | 
         ForEach-Object { 

            # Prepare hashtable 
            $HashProps = @{ 
                NameOfRule = $_.Name 
                RuleVersion = ($_.Value -split '\|')[0] 
                Action = $null 
                Active = $null 
                Dir = $null 
                Protocol = $null 
                LPort = $null 
                App = $null 
                Name = $null 
                Desc = $null 
                EmbedCtxt = $null 
                Profile = $null 
                RA4 = $null 
                RA6 = $null 
                Svc = $null 
                RPort = $null 
                ICMP6 = $null 
                Edge = $null 
                LA4 = $null 
                LA6 = $null 
                ICMP4 = $null 
                LPort2_10 = $null 
                RPort2_10 = $null 
            } 

            # Determine if this is a local or a group policy rule and display this in the hashtable 
            if ($Key -match 'HKLM\\System\\CurrentControlSet') 
            {  $HashProps.RuleType = 'Local' } 
            else 
            {  $HashProps.RuleType = 'GPO' } 

            # Iterate through the value of the registry key and fill PSObject with the relevant data 
            ForEach ($FireWallRule in ($_.Value -split '\|')) 
            { 
                switch (($FireWallRule -split '=')[0]) 
                { 
                    'Action' {$HashProps.Action = ($FireWallRule -split '=')[1]} 
                    'Active' {$HashProps.Active = ($FireWallRule -split '=')[1]} 
                    'Dir' {$HashProps.Dir = ($FireWallRule -split '=')[1]} 
                    'Protocol' {$HashProps.Protocol = ($FireWallRule -split '=')[1]} 
                    'LPort' {$HashProps.LPort = ($FireWallRule -split '=')[1]} 
                    'App' {$HashProps.App = ($FireWallRule -split '=')[1]} 
                    'Name' {$HashProps.Name = ($FireWallRule -split '=')[1]} 
                    'Desc' {$HashProps.Desc = ($FireWallRule -split '=')[1]} 
                    'EmbedCtxt' {$HashProps.EmbedCtxt = ($FireWallRule -split '=')[1]} 
                    'Profile' {$HashProps.Profile = ($FireWallRule -split '=')[1]} 
                    'RA4' {[array]$HashProps.RA4 += ($FireWallRule -split '=')[1]} 
                    'RA6' {[array]$HashProps.RA6 += ($FireWallRule -split '=')[1]} 
                    'Svc' {$HashProps.Svc = ($FireWallRule -split '=')[1]} 
                    'RPort' {$HashProps.RPort = ($FireWallRule -split '=')[1]} 
                    'ICMP6' {$HashProps.ICMP6 = ($FireWallRule -split '=')[1]} 
                    'Edge' {$HashProps.Edge = ($FireWallRule -split '=')[1]} 
                    'LA4' {[array]$HashProps.LA4 += ($FireWallRule -split '=')[1]} 
                    'LA6' {[array]$HashProps.LA6 += ($FireWallRule -split '=')[1]} 
                    'ICMP4' {$HashProps.ICMP4 = ($FireWallRule -split '=')[1]} 
                    'LPort2_10' {$HashProps.LPort2_10 = ($FireWallRule -split '=')[1]} 
                    'RPort2_10' {$HashProps.RPort2_10 = ($FireWallRule -split '=')[1]} 
                    Default {} 
                } 
            } 

            # Create and output object using the properties defined in the hashtable 
            New-Object -TypeName 'PSCustomObject' -Property $HashProps
        } 
    } 
}

and the link from that post as to:

2.2.2.19 Firewall Rule and the Firewall Rule Grammar Rule