Fixing a Runaway Node.js Recursive Folder Issue

Resolving a program that creates endless recursive folders

Node.js applications are generally efficient, but sometimes a bug can lead to unexpected consequences. This article explains how to handle a situation where a Node.js program creates thousands of nested directories that are nearly impossible to delete with standard tools.

The Problem: Near-Infinite Recursive Folders

The issue began when a Node.js script, due to faulty logic, created recursive directories. Each iteration of the script added a new folder inside the previous one. This quickly grew into thousands of nested directories, becoming impossible to delete manually through Windows Explorer or Command Prompt due to the excessive depth of the folder structure.

Long Path Support in Windows

Windows has a long-standing history of limitations with file path lengths. The default maximum length for paths is 260 characters, known as "MAX_PATH." By default, attempts to create or manipulate paths longer than this would fail. However, starting with Windows 10 (Anniversary Update), it became possible to enable long file path support by modifying the Group Policy or Windows Registry.

Enabling Long File Path Support

To enable support for longer paths, I did the following:

Group Policy Editor Method

Navigated to "Computer Configuration > Administrative Templates > System > Filesystem" and enabled the "Enable Win32 long paths" policy.

Registry Edit Method

As an alternative, I edited the Registry directly by setting the value of "LongPathsEnabled" in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem to 1 . This allowed support for file paths longer than 260 characters.

Why Path Length is Important

Despite enabling long file path support, I still encountered limitations with other tools. The core issue was that many standard Windows utilities (such as Command Prompt and PowerShell) still had trouble handling long paths reliably, even with the new support. That's why a custom C++ solution, which directly interacts with the WIN32 API, became necessary.

Steps Attempted Before Writing the C++ Program

Initially, I tried using built-in tools like Windows Command Prompt and PowerShell, thinking I could delete the directories with a simple recursive command. Below are the tools I tried to use before concluding that I needed a custom solution:

Windows Command Prompt

The Windows Command Prompt is a simpler tool that lacks robust recursion and error-handling mechanisms. Commands like "rmdir" are not built to handle such complex directory structures and will often terminate with errors related to file path length or file locks.

rmdir /s "C:\Your\Directory\Path"

Using the "rmdir /s" command, I attempted to remove the entire folder structure recursively. While this works well with normal directories, it failed in this case due to the extreme depth and length of the directory paths. Windows has limitations on path lengths, and this caused the command to fail with errors like "The file name is too long." or "The system cannot find the path specified." or just having the command shell close abruptly.

PowerShell

PowerShell is usually a robust tool for file management, but when dealing with an extremely large number of nested directories, it often fails due to the path length limitation and the recursion depth.

I then tried using PowerShell with the "Remove-Item -Recurse -Force" command. PowerShell is a more powerful tool for managing files and directories, but it, too, failed due to the long file paths and the number of folders. PowerShell inherits the same file path limitations as the Command Prompt unless configured differently, so this option was insufficient.

rm -r -force "C:\Your\Directory\Path"

Though PowerShell had limitations in my case, a potential workaround could involve using the following script, which attempts to bypass the long path limitation by enabling long paths and splitting the process into smaller recursive chunks:

Remove-Item -LiteralPath "C:\Your\Directory\Path" -Recurse -Force

This script could work for some, but in extreme cases like mine, where paths were excessively long and directories numbered in the thousands, it still fell short.

Bash Shell in VS Code

While Linux systems generally handle file systems more gracefully, using Bash on WSL still relies on Windows' underlying file system. This means that although the Linux environment is more forgiving with paths, it inherits the limitations from Windows.

Linux systems handle long paths and recursive operations much more gracefully than Windows. I attempted to use Bash through VS Code to navigate and delete the directories. However, even though Linux could theoretically handle this depth of recursion, Bash failed due to Windows' underlying file system still imposing limits on path lengths.

Robocopy

Robocopy is a robust file copy utility built into Windows that can handle complex file operations. I attempted to use Robocopy to mirror an empty directory structure over the runaway directory, effectively overwriting it. However, Robocopy also failed due to the path length limitations, as it couldn't traverse the deep directory structure to delete the folders. I also tried to purge the directory using Robocopy, but it was unable to handle the excessive depth of the folder structure.

Conclusion: Ultimately, after trying all the built-in tools and existing solutions, I decided to write a C++ program that used the WIN32 API to delete directories recursively.

Writing a C++ Program Solution

Writing the custom C++ solution allowed me to bypass the limitations of standard Windows tools. The WIN32 API offers direct access to the underlying file system and can handle long paths effectively, provided the paths are prefixed with "\\\\?\\". This special prefix enables the API to handle file paths beyond the normal MAX_PATH limitation. By using the WIN32 API, I was able to traverse and delete directories without running into the typical errors caused by long paths.

This program first collected all directories and files and then removed them from the deepest level up to the root, ensuring safe cleanup without getting errors for trying to delete a folder that has sub-folders.

#include <iostream>
#include <windows.h>
#include <string>
#include <vector>
#include <fstream>

void LogOperation(const std::wstring& message, const std::wstring& logFileName = L"operation_log.txt") {
  std::wofstream logFile(logFileName, std::ios::app);
  if (logFile.is_open()) {
    logFile << message << std::endl;
  }
}

void DeleteDirectoryRecursively(const std::wstring& directory) {
  std::vector<std::wstring> directoriesToDelete;
  std::vector<std::wstring> allDirectories;

  directoriesToDelete.push_back(directory);

  while (!directoriesToDelete.empty()) {
    std::wstring currentDir = directoriesToDelete.back();
    directoriesToDelete.pop_back();

    WIN32_FIND_DATA findFileData;
    std::wstring searchPath = currentDir + L"\\*";
    HANDLE hFind = FindFirstFile(searchPath.c_str(), &findFileData);

    if (hFind != INVALID_HANDLE_VALUE) {
      do {
        const std::wstring fileOrDirName = findFileData.cFileName;
        if (fileOrDirName != L"." && fileOrDirName != L"..") {
          const std::wstring fullPath = currentDir + L"\\" + fileOrDirName;
          if (findFileData.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) {
            directoriesToDelete.push_back(fullPath);
            allDirectories.push_back(fullPath);
          } else {
            SetFileAttributes(fullPath.c_str(), FILE_ATTRIBUTE_NORMAL);
            DeleteFile(fullPath.c_str());
          }
        }
      } while (FindNextFile(hFind, &findFileData) != 0);
      FindClose(hFind);
    }
  }

  std::reverse(allDirectories.begin(), allDirectories.end());

  for (const std::wstring& dir : allDirectories) {
    RemoveDirectory(dir.c_str());
  }

  RemoveDirectory(directory.c_str());
}

int main() {
  const std::wstring baseDirectory = L"\\\\?\\C:\\AzureDevOps\\runaway-node";
  LogOperation(L"Starting directory delete process.");
  DeleteDirectoryRecursively(baseDirectory);
  LogOperation(L"Directory delete process completed.");
  return 0;
}

Explanation of the C++ Code

This C++ code starts by scanning the directory provided (in this case, "C:\\AzureDevOps\\runaway-node") and collecting a list of all subdirectories. It does this recursively, storing all directories for later deletion. After collecting all the directories, it deletes them from the deepest folder up to the root directory.

Key Components of the Solution:

  1. Special Path Prefix: The code uses "\\\\?\\" prefix before the path, which tells Windows to disable path parsing and bypass the 260-character limit.
  2. Breadth-First Directory Collection: The program uses a breadth-first approach to collect all directories that need to be deleted.
  3. Reverse Deletion Order: After collecting all directories, it reverses the order so that it deletes from the deepest nested folder first.
  4. File Attribute Handling: The code sets file attributes to normal before deletion, which helps overcome issues with read-only or hidden files.
  5. Logging: The program includes a logging function to track the progress and any potential issues during the deletion process.

The Result

This C++ solution successfully deleted the thousands of nested directories that were created by the runaway Node.js script. It handled the long paths without issues and completed the cleanup in a reasonable amount of time. By working directly with the WIN32 API, the program was able to overcome the limitations of standard Windows tools and utilities.

Key Takeaways

When dealing with extremely deep directory structures in Windows, standard tools often fail due to path length limitations, even with modern settings enabled.

Prevention is Better Than Cure

Always test file system operations in your Node.js scripts, especially when they involve recursive directory creation or manipulation. Add appropriate validation and safeguards to prevent runaway processes.

For extreme cases where standard utilities fail, writing a custom solution using the WIN32 API with the "\\\\?\\" path prefix is the most reliable approach to handle long paths and deep directory structures in Windows.

Back to Table of Contents