Fixing Recursive Folder Issue

Resolving a program that creates endless recursive folders

Node.js applications are generally efficient, but sometimes a bug can lead to unexpected consequences. Recently, I encountered a situation where a Node.js program started creating a near-infinite number of recursive folders on my Windows machine. It quickly spiraled out of control, and manually deleting these folders wasn’t an option due to their sheer number. In this article, I’ll walk you through the problem, how I identified the issue, and the C++ solution I used to clean up the folders.

Problem: Near-Infinite Recursive Folders

The issue began when a Node.js script, due to faulty logic, created recursive directories. Each iteration of the script added a new folder inside the previous one. This quickly grew into thousands of nested directories, becoming impossible to delete manually through Windows Explorer or Command Prompt due to the excessive depth of the folder structure.

Long Path Support in Windows

Windows has a long-standing history of limitations with file path lengths. The default maximum length for paths is 260 characters, known as "MAX_PATH." By default, attempts to create or manipulate paths longer than this would fail. However, starting with Windows 10 (Anniversary Update), it became possible to enable long file path support by modifying the Group Policy or Windows Registry.

Enabling Long File Path Support

To enable support for longer paths, I did the following:

1. Opened Group Policy Editor (gpedit.msc)

Navigated to "Computer Configuration > Administrative Templates > System > Filesystem" and enabled the "Enable Win32 long paths" policy.

2. Edited the Windows Registry

As an alternative, I edited the Registry directly by setting the value of "LongPathsEnabled" in `HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem` to `1`. This allowed support for file paths longer than 260 characters.

Why Path Length is Important

Despite enabling long file path support, I still encountered limitations with other tools. The core issue was that many standard Windows utilities (such as Command Prompt and PowerShell) still had trouble handling long paths reliably, even with the new support. That's why a custom C++ solution, which directly interacts with the WIN32 API, became necessary.

Steps Attempted Before Writing the C++ Program

Initially, I tried using built-in tools like Windows Command Prompt and PowerShell, thinking I could delete the directories with a simple recursive command. Below are the tools I tried to use before concluding that I needed a custom solution:

Windows Command Prompt

The Windows Command Prompt is a simpler tool that lacks robust recursion and error-handling mechanisms. Commands like "rmdir" are not built to handle such complex directory structures and will often terminate with errors related to file path length or file locks.

rmdir /s "C:\Your\Directory\Path"

Using the "rmdir /s" command, I attempted to remove the entire folder structure recursively. While this works well with normal directories, it failed in this case due to the extreme depth and length of the directory paths. Windows has limitations on path lengths, and this caused the command to fail with errors like "The file name is too long." or "The system cannot find the path specified." or just having the command shell close abrubtly.

PowerShell

PowerShell is usually a robust tool for file management, but when dealing with an extremely large number of nested directories, it often fails due to the path length limitation and the recursion depth.

I then tried using PowerShell with the "Remove-Item -Recurse -Force" command. PowerShell is a more powerful tool for managing files and directories, but it, too, failed due to the long file paths and the number of folders. PowerShell inherits the same file path limitations as the Command Prompt unless configured differently, so this option was insufficient.

rm -r -force "C:\Your\Directory\Path"

Though PowerShell had limitations in my case, a potential workaround could involve using the following script, which attempts to bypass the long path limitation by enabling long paths and splitting the process into smaller recursive chunks:

Remove-Item -LiteralPath "C:\Your\Directory\Path" -Recurse -Force

This script could work for some, but in extreme cases like mine, where paths were excessively long and directories numbered in the thousands, it still fell short.

Bash Shell in VS Code

While Linux systems generally handle file systems more gracefully, using Bash on WSL still relies on Windows' underlying file system. This means that although the Linux environment is more forgiving with paths, it inherits the limitations from Windows.

Linux systems handle long paths and recursive operations much more gracefully than Windows. I attempted to use Bash through VS Code to navigate and delete the directories. However, even though Linux could theoretically handle this depth of recursion, Bash failed due to Windows' underlying file system still imposing limits on path lengths.

Robocopy

Robocopy is a robust file copy utility built into Windows that can handle complex file operations. I attempted to use Robocopy to mirror an empty directory structure over the runaway directory, effectively overwriting it. However, Robocopy also failed due to the path length limitations, as it couldn't traverse the deep directory structure to delete the folders. I also tried to purge the directory using Robocopy, but it was unable to handle the excessive depth of the folder structure.

Conclusion

Ultimately, after trying all the built-in tools and existing solutions, I decided to write a C++ program that used the WIN32 API to delete directories recursively.

Writing a C++ Program

Windows C++ Program to Rename and Delete Recursive Folders

Solution: Windows C++ Program to Rename and Delete Recursive Folders

Writing the custom C++ solution allowed me to bypass the limitations of standard Windows tools. The WIN32 API offers direct access to the underlying file system and can handle long paths effectively, provided the paths are prefixed with "\\\\?\\". This special prefix enables the API to handle file paths beyond the normal MAX_PATH limitation. By using the WIN32 API, I was able to traverse and delete directories without running into the typical errors caused by long paths.

This program first collected all directories and files and then removed them from the deepest level up to the root, ensuring safe cleanup without getting errors for trying to delete a folder that has sub-folders.

#include <iostream>
#include <windows.h>
#include <string>
#include <vector>
#include <fstream>

void LogOperation(const std::wstring& message, const std::wstring& logFileName = L"operation_log.txt") {
  std::wofstream logFile(logFileName, std::ios::app);
  if (logFile.is_open()) {
    logFile << message << std::endl;
  }
}

void DeleteDirectoryRecursively(const std::wstring& directory) {
  std::vector<std::wstring> directoriesToDelete;
  std::vector<std::wstring> allDirectories;

  directoriesToDelete.push_back(directory);

  while (!directoriesToDelete.empty()) {
    std::wstring currentDir = directoriesToDelete.back();
    directoriesToDelete.pop_back();

    WIN32_FIND_DATA findFileData;
    std::wstring searchPath = currentDir + L"\\*";
    HANDLE hFind = FindFirstFile(searchPath.c_str(), &findFileData);

    if (hFind != INVALID_HANDLE_VALUE) {
      do {
        const std::wstring fileOrDirName = findFileData.cFileName;
        if (fileOrDirName != L"." && fileOrDirName != L"..") {
          const std::wstring fullPath = currentDir + L"\\" + fileOrDirName;
          if (findFileData.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) {
            directoriesToDelete.push_back(fullPath);
            allDirectories.push_back(fullPath);
          } else {
            SetFileAttributes(fullPath.c_str(), FILE_ATTRIBUTE_NORMAL);
            DeleteFile(fullPath.c_str());
          }
        }
      } while (FindNextFile(hFind, &findFileData) != 0);
      FindClose(hFind);
    }
  }

  std::reverse(allDirectories.begin(), allDirectories.end());

  for (const std::wstring& dir : allDirectories) {
    RemoveDirectory(dir.c_str());
  }

  RemoveDirectory(directory.c_str());
}

int main() {
  const std::wstring baseDirectory = L"\\\\?\\C:\\AzureDevOps\\runaway-node";
  LogOperation(L"Starting directory delete process.");
  DeleteDirectoryRecursively(baseDirectory);
  LogOperation(L"Directory delete process completed.");
  return 0;
}

Explanation of the Code

This C++ code starts by scanning the directory provided (in this case, "C:\\AzureDevOps\\runaway-node") and collecting a list of all subdirectories. It does this recursively, storing all directories for later deletion. After collecting all the directories, it deletes them from the deepest folder up to the root directory.