Access Denied While Deleting .exe File

Summary Of Problem:

I have a console app that, after copying many folders and files over to a new location, a local drive, it then deletes certain files/folders. One of these filetypes it deletes is .exe files. When trying to delete said files it gives me a denied access error.(This also occurs while trying to delete other kinds of files and folders as well)

Other Notes:

I saw several questions, such as Unable to delete .exe file through c#. However the process was never running on my local machine nor on the source it was copied from. I am both a local administrator and a domain administrator on our domain, and I had ownership over all folders and files in the directories I'm working with. But it still denies me access when trying to delete the file from code. I am however able to delete such files/folders manually.(click delete key)

Problem:

As stated above I am looking for a way to get past this issue denying me access to delete these "Illegal" file types and delete them from my code.

My Code:(Updated)

#region Delete_Illegal_Items
    public static void RemoveIllegalItems()
    {
        Console.Clear();
        DirectoryInfo Libraries = new DirectoryInfo(Library.DestinationMain);
        try
        {
            foreach (var Lib in Libraries.GetDirectories())
            {
                Console.WriteLine("Working On {0}.", Lib.Name);
                Parallel.Invoke(
                        () =>
                        {
                            RemoveBadFiles(Lib);
                        },

                        () =>
                        {
                            DeleteEmptyFolders(Lib);
                        }
                    );
            }
        }
        catch (AggregateException e)
        {
            Console.WriteLine("There Was An Unusual Error During Initialization Of Library Correction:\n{0}", e.InnerException.ToString());
        }
    }

    private static string[] BadFiles = { 
                                        @".hta",
                                        @".exe",
                                        @".lnk",
                                        @".tmp",
                                        @".config",
                                        @".ashx",
                                        @".hta.",
                                        @".hta::$DATA",
                                        @".zip",
                                        @".asmx",
                                        @".json",
                                        @".soap",
                                        @".svc",
                                        @".xamlx",
                                        @".msi",
                                        @".ops",
                                        @".pif",
                                        @".shtm",
                                        @".shtml",
                                        @"smt",
                                        @".vb",
                                        @".vbe",
                                        @".vbs",
                                        @".ds_store",
                                        @".db",
                                        @".ini",
                                        @".tiff"
                                      };
    private static void RemoveBadFiles(DirectoryInfo directory)
    {
        DirectoryInfo[] dirs = null;
        FileInfo[] files = null;
        if (directory != null)
        {
            files = directory.GetFiles();
        }

        try
        {
            dirs = directory.GetDirectories();
        }
        catch (IOException) { }
        catch (Exception e)
        {
            Console.WriteLine("\nError During Enumeration Of Items To Delete:\n{0}", e.Message);
        }

        if (files != null)
        {
            foreach (var file in files)
            {
                try
                {
                    if (file.IsReadOnly)
                    {
                        file.IsReadOnly = false;
                    }

                    if (BadFiles.Contains(Path.GetExtension(file.FullName)))
                    {
                        File.Delete(file.FullName);
                    }
                }
                catch (Exception e)
                {
                    Console.WriteLine("\nError During Removal Or Illegal Files:\n" + e.Message);
                }
            }
        }

        if (dirs != null)
        {
            foreach (var dir in dirs)
            {
                switch (dir.Name)
                {
                    case ".TemporaryItems":
                        {
                            try
                            {
                                Directory.Delete(dir.FullName);
                            }
                            catch { }
                            break;
                        }
                    case "AI_RecycleBin":
                        {
                            try
                            {
                                Directory.Delete(dir.FullName);
                            }
                            catch { }
                            break;
                        }
                    case ".ToRemove":
                        {
                            try
                            {
                                Directory.Delete(dir.FullName);
                            }
                            catch { }
                            break;
                        }
                    default:
                        {
                            break;
                        }
                }
                RemoveBadFiles(dir);
            }
        }
    }

    private static void DeleteEmptyFolders(DirectoryInfo directory)
    {
        Program Main = new Program();
        try
        {
            DirectoryInfo[] dirs = directory.GetDirectories();

            foreach (var subDirectory in dirs)
            {
                int sum = Library.CountLibrary(subDirectory.FullName);

                if (sum == 0)
                {
                    Directory.Delete(subDirectory.FullName);
                }

                DeleteEmptyFolders(subDirectory);
            }
        }
        catch { }
    }
    #endregion

Any help would be greatly appreciated. If this question has been directly answered elsewhere please feel free to mention it in the comment. As I stated above I have been looking through past question regarding this issue and have not found a solution as of yet. Otherwise thank you for your help.