Pipeline objects need to be referenced with $_, not $, as shown below:
Get-Childitem -Path C:\folder1 -Recurse | Where-Object {$_.Name -ilike "*tempspecssuite*"} | Remove-Item -Force -WhatIf
Which can also be referenced with $PSItem or set to a custom variable with the -PipelineVariable common parameter. You can find out more in about_pipelines,about_objects and about_automatic_variables.
You could also simplify the above by making use of the -Filter parameter from Get-ChildItem, which also accepts wildcards:
Get-ChildItem -Path C:\folder1 -Directory -Filter tempspecssuite_* -Recurse | Remove-Item -Force -Recurse -WhatIf
Which allows Get-ChildItem to filter files while they are getting retrieved, rather than filtering with Where-Object afterwards.
You can also limit the filtering with the -Directory switch, since we only care about deleting directories.
$.Nameshould be$_.Name