LogFileParser – Classes and Enums

Hi together,

i created a LogFileParser (continuation from here), which can handle till now 4 different LogFileTypes:
SCCM-Logs
DISM
CBS
Upgrade-Logs

It´s written for Powershell v5, because it makes use of classes and i wanted to show you, how you can work with enums and classes.

Here is how you can make use of it: (can be found in Examples.ps1)

1.JPG

3.JPG

4.JPG

1.JPG

Download: https://github.com/ddneves/LogFileParser

Its code is written with Powershell classes which can be easily extended with additional helper functions or new LogFileTypes.

I integrated also a helper function to retrieve a range of cells before and after defined cell rownumbers. I used it to gather all rows, which contain “*error*” and all the 20(x) rows before and after for a whole file.

#requires -Version 5

<#	
        .NOTES
        ===========================================================================
        Created on:   	07.08.2016
        Created by:   	David das Neves
        Version:        0.1
        Project:        LogAnalyzer
        Filename:       LogFileParser.ps1
        ===========================================================================
        .DESCRIPTION
        Parses Logfiles into the class ParsedLogFile, which is integrated in the class LogFileParser
#> 


## Enumeration of the LogFileTypes
enum LogFileTypes { 
    SCCM
    CBS   
    Upgrade
    DISM
}
   
## Class of the LogParser which can open n logfiles
class LogFileParser
{
    #region Props

    # Generic List of all parsed LogFiles of Type "ParsedLogfile" (Class)
    [System.Collections.Generic.List``1[ParsedLogFile]] $ParsedLogFiles

    # FilePath
    [String] $LogFilePath
    
    #endregion

    #region Funcs

    # Constructor
    LogFileParser($LogFilePath)
    {            
        # Constructor Code
        if (Test-Path $LogFilePath)
        {       
            $allLogFiles = Get-ChildItem $LogFilePath -Filter *.log -Recurse
            $this.ParsedLogFiles = New-Object -TypeName System.Collections.Generic.List``1[ParsedLogFile]
            $this.LogFilePath = $LogFilePath
            foreach ($file in $allLogFiles)
            {                
                $fileType = Get-LogFileType $file.Name  
                $this.ParsedLogFiles.Add([ParsedLogFile]::new($file.FullName, $fileType))   
            }            
        }
        else
        {
            Write-Error -Message 'Path was not reachable. Please verify your Path.'
        }    
    }
    #endregion
}

class ParsedLogFile
{
    #region Props

    #Hidden variable for the keys in the logfile.
    hidden [string[]] $ColumnNames

    # FilePath of the parsed log.
    [string] $LogFilePath
   
    #The parsed logging data.
    $ParsedLogData

    #LogFileType for this file
    [LogFileTypes] $LogFileType

    #endregion

    #region Funcs

    ## standard constructor
    ## LogFileType SCCM is set
    ParsedLogFile($LogFilePath)
    {        
        $this.LogFileType = [LogFileTypes]::SCCM
        $this.LogFilePath = $LogFilePath
        $this.LogFileType = $this.LogFileType
        $this.Init()
    }

    ## Constructr with LogFileType
    ParsedLogFile($LogFilePath, $LogFileType)
    {     
        $this.LogFilePath = $LogFilePath
        $this.LogFileType = [LogFileTypes]$LogFileType
        $this.Init()
    }

    ## Initialization of class and log
    hidden Init()
    {        
        if (Test-Path -Path $this.LogFilePath)
        {            
            # Constructor Code
            $this.LogFilePath = $this.LogFilePath
            Write-Host -Object "Parsing LogFile $($this.LogFilePath) with LogfileType $($this.LogFileType)."
            $actualParsedLog = Get-RegExParsedLogfile -Path $this.LogFilePath -LogFileType $this.LogFileType
            Write-Host -Object 'Parsing done.'
            $this.ParsedLogData = $actualParsedLog.Log
            $this.ColumnNames = $actualParsedLog.Keys
        }
        else
        {
            Write-Error -Message "Path was not reachable. Please verify your Path: $($this.LogFilePath)."
        }   
    }

    # Returns the column Keys
    [string[]] GetColumnNames()
    {
        return $this.ColumnNames
    }

    # Returns lines with errors
    [int[]] GetLinesWithErrors()
    {
        $LinesWithErrors = ($this.ParsedLogData).Where{
            $_.Entry -like '*error*'
        }
        return $LinesWithErrors
    }

    # Returns lines with errors
    [int[]] GetLinesWithErrorsHeuristic()
    {
        $LinesWithErrors = (($this.ParsedLogData).Where{
                $_.Entry -like '*error*'
        }).RowNum
        $RowList = Get-RowNumbersInRange $LinesWithErrors
        $ShowingRows = ($this.ParsedLogData).Where{
            $_.RowNum -in $RowList
        }
        return $LinesWithErrors
    }

    # Returns lines with errors
    # Overload with Range
    [int[]] GetLinesWithErrorsHeuristic([int]$Range)
    {
        $LinesWithErrors = (($this.ParsedLogData).Where{
                $_.Entry -like '*error*'
        }).RowNum
        $RowList = Get-RowNumbersInRange $LinesWithErrors -Range $Range
        $ShowingRows = ($this.ParsedLogData).Where{
            $_.RowNum -in $RowList
        }
        return $LinesWithErrors
    }
    
    # Returns lines with errors
    [int[]] GetRowNumbersWithErrors()
    {
        $LinesWithErrors = (($this.ParsedLogData).Where{
                $_.Entry -like '*error*'
        }).RowNum
        return $LinesWithErrors  
    }
    #endregion
}



function Get-RowNumbersInRange
{
    <#
            .Synopsis
            Get-RowNumbersInRange
            .DESCRIPTION
            Heuristic method, which returns a list of all transmitted rowlines addiing a number of lines n ($Range) before and after.
            .EXAMPLE
            $rowsWithErrors = 21,345,456
            $allRowsToSHow = Get-RowNumbersInRange -RowNumbers $rowsWithErrors -Range 10 
    #>
    [CmdletBinding()]    
    Param
    (
        #Previous calculated set of rowNumbers.
        [Parameter(Mandatory = $true,
                ValueFromPipelineByPropertyName = $true,
        Position = 0)]
        $RowNumbers,

        [Parameter(Mandatory = $false,                   
        Position = 1)]
        [int]$Range = 20
    )
    Begin
    { }
    Process
    {
        $allShowingRowNumbers = New-Object -TypeName System.Collections.Generic.List``1[Int]
        foreach ($rowNum in $RowNumbers)
        {
            $min = $rowNum - $Range
            $max = $rowNum + $Range

            for ($x = $min; $x -lt $max; $x += 1) 
            {
                $allShowingRowNumbers.Add($x)
            }
        }        
        $allShowingRowNumbers
    }
    End
    { }
}


function Get-LogFileType
{
    <#
            .Synopsis
            Get-LogFileType
            .DESCRIPTION
            Returns the type of the transmitted LogFile.
            .EXAMPLE       
            $LogFileType = Get-LogFileType -LogFileName 'dism.log'
    #>
    [CmdletBinding()]    
    Param
    (
        #Name of the logFile
        [Parameter(Mandatory = $true,
                ValueFromPipelineByPropertyName = $true,
        Position = 0)]
        $LogFileName
    )
    Begin
    { }
    Process
    {
        switch ($LogFileName)
        {
            #DISM
            {
                $_ -like 'dism*'
            }     
            {
                [LogFileTypes]::DISM 
                break
            }

            #Upgrade
            {
                $_ -like 'setupact*'
            } 
            {
                [LogFileTypes]::Upgrade
                break
            }
            {
                $_ -like 'setuperr*'
            } 
            {
                [LogFileTypes]::Upgrade
                break
            }

            #CBS
            {
                $_ -like 'cbs*'
            }      
            {
                [LogFileTypes]::CBS
                break
            }
                  
            #SCCM
            default                
            {
                [LogFileTypes]::SCCM
            }
        }
    }
    End
    { }
}

function Get-RegExParsedLogfile
{
    <#
            .SYNOPSIS
            Returns a ordered hashtable list for a log by using Regex.
            .DESCRIPTION
            The Regular Expression splits a single line of the log file into named keys.
            This is used for a whole log file and a ordered hashtable list is returned.
            .EXAMPLE
            $parsedLogFile = Get-RegExParsedLogfile -Path 'c:\windows\CCM\ccmexec.log' -LogFileType SCCM | Out-GridView
            .EXAMPLE
            Get-RegExParsedLogfile -Path 'c:\windows\logs\cbs\cbs.log' -LogFileType CBS | Out-GridView
            .EXAMPLE
            cls 
            $parsedLogFile = Get-RegExParsedLogfile -Path 'c:\windows\logs\cbs\cbs.log' 
            $parsedLogFile.Log.Line | Where-Object { $_ -like '*error*' }
            The Logfile is written into the hastable with the integrated key "Line".
            You can filter these with where.
            .EXAMPLE
            $rx = '(?<Date>\d{4}-\d{2}-\d{2})\s+(?<Time>(\d{2}:)+\d{2}),\s+(?<Type>\w+)\s+(?<Component>\w+)\s+(?<Message>.*)$'
            $parsedLogFile = Get-RegExParsedLogfile -Path 'c:\windows\logs\cbs\cbs.log' -RegexString $rx
            $parsedLogFile.Keys    
            .EXAMPLE
            $rx='<!\[LOG\[(?<Entry>.*)]LOG]!><time="(?<Time>.*)\.\d{3}-\d{3}"\s+date="(?<Date>.*)"\s+component="(?<Component>.*)"\s+context="(?<Context>.*)"\s+type="(?<Type>.*)"\s+thread="(?<Thread>.*)"\s+file="(?<File>.*):(?<CodeLine>\d*)">' 
            $parsedLogFile = Get-RegExParsedLogfile -Path 'c:\windows\CCM\ccmexec.log' -RegexString $rx
    #>
    [CmdletBinding()]
    param
    (
        #Contains the log file destination.
        [Parameter(Mandatory = $true, Position = 0)]
        [System.String]
        $Path = 'c:\windows\logs\cbs\cbs.log',
        
        #Contains the RegEx with named keys
        [Parameter(Mandatory = $false, Position = 1)]
        [System.String]
        $RegexString = '(?<Line>.*)$',
        
        #ValidateSet of the differenct preconfigured LogFileTypes               
        [Parameter(Mandatory = $false, Position = 2)]
        [LogFileTypes]$LogFileType = 'SCCM',

        #Filter
        [Parameter(Mandatory = $false, Position = 3)]
        [System.String]
        $GatherOnlyLinesWhichContain = '' 
    )
    
    $t = (Get-Content -Path $Path -ReadCount 1000).Split([System.Environment]::NewLine)

    if ($GatherOnlyLineWhichContain)
    {
        $t = $t| Select-String $GatherOnlyLinesWhichContain
    }    

    [regex]$rx = $RegexString

    # for each LogFileType a different Regex-String is used to parse the log.
    switch ($LogFileType)
    {
        'SCCM'       
        { 
            $rx = '<!\[LOG\[(?<Entry>.*)]LOG]!><time="(?<Time>.*)\.\d{3}-\d{3}"\s+date="(?<Date>.*)"\s+component="(?<Component>.*)"\s+context="(?<Context>.*)"\s+type="(?<Type>.*)"\s+thread="(?<Thread>.*)"\s+file="(?<File>.*):(?<CodeLine>\d*)">' 
            break
        }
        'CBS'        
        { 
            $rx = '(?<Date>\d{4}-\d{2}-\d{2})\s+(?<Time>(\d{2}:)+\d{2}),\s+(?<Type>\w+)\s+(?<Component>\w+)\s+(?<Message>.*)$'
            break
        }
        'Upgrade'
        {
            $rx = '(?<Date>\d{4}-\d{2}-\d{2})\s+(?<Time>(\d{2}:)+\d{2}),\s+(?<Type>\w+)\s{1,17}(\[(?<ErrorCode>\w*)\])?(?<Component>\s\w+)?\s+(?<Message>.*)'
            break
        }
        'DISM'
        {            
            $rx = '(?<Date>\d{4}-\d{2}-\d{2})\s+(?<Time>(\d{2}:)+\d{2}),\s+(?<Type>\w+)\s{1,18}(?<Component>\w+)?\s+(?<Message>.*)'
            break
        }
        
        default      
        {
            Write-Error -Message 'Not Type has been set or found.'
        }
    }        
        
  
    [string[]]$names = 'RowNum'  
    $names += $rx.GetGroupNames() | Where-Object -FilterScript {
        $_ -match '\w{2}'
    } 
    
    [long]$rowNum = 0   
    $data = $t | ForEach-Object -Process  {
        $rx.Matches($_) | ForEach-Object -Process {
            $match = $_
            $names | ForEach-Object -Begin {
                $hash = [Ordered]@{}
               # $thisDate = $null
            } -Process {
                if ($_ -eq 'RowNum')
                {
                    $rowNum += 1
                    $hash.Add($_, $rowNum) 
                }
                elseif ($_ -eq 'Thread')
                {                    
                    $hash.Add($_, [int]($match.groups["$_"].Value)) 
                }
                else
                {
                    $hash.Add($_, $match.groups["$_"].Value)
                }                
            } -End {
                $thisDate=[datetime]($hash.Date + ' ' + $hash.Time)
                $hash.Add('DateTime', $thisDate)                
                [PSCustomObject]$hash
            }
        }
    }    
    $object = New-Object -TypeName PSObject
    $object | Add-Member -MemberType NoteProperty -Name Keys -Value $names
    $object | Add-Member -MemberType NoteProperty -Name Log -Value $data 
    $object    
}

 

I hope, you like it.

 

Best regards,

David

Advertisements

PSGUI v0.2 – Next Round – Powershell vs. XAML – Fight!

PSGUI_v0.2.JPG

Hi together,

today i have pretty good news for you!

Long time ago i started the project PSGUI, but i had not much time and power to create a stable version, because i had been working and training hard for Microsoft in the last months.

But – luckily i had again some time to get progress in this nice project and here is the next version!

Download:
https://github.com/ddneves/PSGUI

HOWTO:

  • Download as Zip
  • Unblock in NTFS Properties
  • install.bat
  • Link on Desktop [or] Link in folder [or] Powershell.exe – Start-PSGUIManager
  • have fun – till now

 

But what was actually PSGUI about? Below a short summary – detailed instructions will follow.


Continue reading

Who’s afraid of PowerShell security?

Hello together,

many people are afraid of using Powershell remoting, because they think it is very unsafe.

Just take a moment and read this:

The improvements in WMF 5.0 (or WMF 4.0 with KB3000850) make PowerShell the worst tool of choice for a hacker when you enable script block logging and system-wide transcription. Hackers will leave fingerprints everywhere, unlike popular CMD utilities. For this reason, PowerShell should be the only tool you allow for remote administration. These features allow you to answer the classic questions who, what, when, where, and how for activities on your servers.

Technet Link



PowerShell Remoting Security Considerations

New security documentation from the PowerShell team. This is a start, and it will continue to be updated. Give this link to your InfoSec people who need more information.

https://msdn.microsoft.com/en-us/powershell/scripting/setup/winrmsecurity

PowerShell ♥ the Blue Team

Whitepaper by Lee Holmes “Scripting Security and Protection Advances in Windows 10” (PowerShell 5).

Give this to your InfoSec people, your manager, and your grandmother. Then implement it.

https://blogs.msdn.microsoft.com/powershell/2015/06/09/powershell-the-blue-team/


Best regards,

David

PSConfEU – Interviews – Material

Hi all,

some time passed since my last update, but now i am arriving with a whole bunch of demo scripts for everyone.

I have been at the PSConfEU and had two speaking slots there:

  • PS-Repo Server
  • Deep Dive XAML-GUIs in PS

My good friend Sebstian Klenk, Technical Evangelist Microsoft, did two interviews in german with me and summed everything up in his blog.

There you can get also the material of my sessions and more – i recommend the demo Scripts of the GUI Session-  but keep in mind that not all of the examples were written by me but i personally did not want to keep them out.

Interview/Blog: PS-Repo-Server

Material – PS-Repo-Server

Interview/Blog: Deep Dive XAML-GUIs in PS

Material Deep Dive XAML-GUIs in PS

Greetings,

David

PS – External Utilities – german console with umlauts

Hi all,

i am using a german console with german words. Many of you know that we use “very special characters” also called umlauts: “ÄÜÖ”.

The interesting part is – if you try to use external tools as the following and they normally would return any values with umlauts this will not work out from Powershell; Let´s say for this example you created a shared directory named ‘ÄÜÖ’:

net.exe VIEW \\localhost\ /all

Here you would get something like this by using it in Powershell:


Freigabename  Typ     Verwendet als  Kommentar         

-------------------------------------------------------------------------------
ADMIN$        Platte                 Remoteverwaltung
Bilder        Platte
C$            Platte                 Standardfreigabe
IPC$          IPC                    Remote-IPC
print$        Platte                 Druckertreiber
Users         Platte
Ž™š           Platte
Der Befehl wurde erfolgreich ausgefhrt.

This error is caused by not matching encodings. Therefore these have to be set previously for the console itself. But here comes again another tricky part. Till now – you would get an error by setting the consoles encoding without having used the cmd.exe out of powershell before, because the handle to the console has to be created previously. Therefore the way to get this to work would be like this:

cmd /c ''
if ([Console]::OutputEncoding.CodePage -ne 852 )
{
    [Console]::OutputEncoding = [System.Text.Encoding]::GetEncoding(852)
}
net.exe VIEW \\localhost\ /all

#getting only the specific line
net.exe VIEW \\localhost\ /all | Select-String 'Ö'
net.exe VIEW \\localhost\ /all | Where-Object { $_ -like '*Ö*' }

With following result:

Freigabename  Typ     Verwendet als  Kommentar         

-------------------------------------------------------------------------------
ADMIN$        Platte                 Remoteverwaltung  
Bilder        Platte                                   
C$            Platte                 Standardfreigabe  
IPC$          IPC                    Remote-IPC        
print$        Platte                 Druckertreiber    
Users         Platte                                   
ÄÖÜ           Platte                                   
Der Befehl wurde erfolgreich ausgeführt.

I am pretty sure that german readers may need this information some time.

Best regards,
David

PS – CleanUp HDD

Hello together,

today i want to show you a simple but effective method to cleanup your HDD. This will also remove the installation data of previous installations and may prompt therefore for authorization. Run it with admin rights.

CleanUp.png

More information here

Set-Location 'HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\VolumeCaches\'

foreach ($item in $(Get-ChildItem).PSPath)
{
    if (-not (Get-ItemProperty -Path $item -Name 'StateFlags1234'))
    {
        New-ItemProperty -Path $item -Name 'StateFlags1234' -Value 2
    }
}

cleanmgr.exe /sagerun:1234 

Greetings,

David

How to become a Powershell Pro?

PowershellPro

Hello together,

today i want to share my experiences of the last months with you where i worked nearly every day hours over hours on my Powershell skills. I have read a lot of books, watched douzens webinars, written many blog and forum posts and also thousands of lines powershell code. But not every material or action was reasonable at some learning point.

So i worked out – somewhat like – a learning plan. I will list up the best material  you should work on at a specific learning point. (as for my own personal opinion)
But keep in mind – this could vary for you. Persons are different as also their learning types. Some persons need to hear and see the things to capture the input better. Others learn best by reading and some need to interact as by writing.
But one thing is for sure – you will need a lot of practice and you will need to write scripts by yourself!

Continue reading