fix: URI handling for URIs with special/blank characters [IDE-1203][IDE-1235] (#706)

* fix: improve path and URI conversion for cross-platform compatibility

This commit enhances the path and URI conversion functionality to ensure
reliable file path handling across different operating systems.
Specifically improves Windows path handling with proper URI encoding/decoding
and adds comprehensive tests to verify the bidirectional conversion between
paths and URIs works correctly on both POSIX and Windows systems.

* fix: more test cases and use ASCII strings

* fix: toLanguageServerURL

* fix: toLanguageServerURL tests

* fix: improve language server initialization and URI handling [IDE-1203]

- Increase language server initialization timeout from 20L to 2000L
- Add proper error handling when adding content roots
- Run content root addition asynchronously to prevent UI freezes
- Improve error logging with more descriptive messages
- Add user notification when language server fails to initialize
- Replace size > 0 check with isNotEmpty() for better readability
- Fix code formatting and parameter organization

* fix: tests in ReferenceChooserDialogTest.kt

* fix: error handling in fromUriToPath

Co-authored-by: windsurf-bot[bot] <189301087+windsurf-bot[bot]@users.noreply.github.com>

* fix: windsurf suggestion

* fix: test setup for ReferenceChooserDialogTest.kt

* chore: revert timeout

* docs: CHANGELOG.md

* fix: normalize path before using it to persist folderConfig

* fix: Improve folder config path normalization and add tests [IDE-1203]

Ensures that FolderConfigSettings consistently handles folder paths
by normalizing and absolutizing them. The 'folderPath' property
of FolderConfig objects managed by FolderConfigSettings will
now always reflect this normalized, absolute path.

This resolves an issue where the stored folderPath in FolderConfig
objects did not always represent the fully normalized and absolute
path used as the key in the settings map, leading to potential
inconsistencies and failing tests for path normalization.

Key changes include:
- Added FolderConfigSettingsTest.kt with comprehensive unit tests
  for path normalization, covering various scenarios including
  paths with '.', '..', and equivalent path representations.
- Converted tests to JUnit 4 syntax as per project standards.
- Updated FolderConfigSettings:
  - 'addFolderConfig' now stores a copy of the FolderConfig with
    its 'folderPath' correctly normalized and absolutized.
  - 'createEmpty' now directly instantiates FolderConfig with the
    normalized path, improving clarity and efficiency.
- Fixed a compile error in ReferenceChooserDialogTest.kt by
  refactoring a direct private field access to use a public method.

* chore: update initialization messaging in summary [IDE-1235]

* refactor: ReferenceChooserDialogTest.kt setup

* refactor: add tests, normalize more paths

* refactor: ensure file separator suffix in folder configs

* fix: tests

---------

Co-authored-by: Abdelrahman Shawki Hassan <[email protected]>
(cherry picked from commit 9ba9809)

Author: bastiandoetsch

.windsurfrules

@@ -6,6 +6,7 @@ always write and update test cases. iterate until they pass.
 use existing mocks, don't write new ones.
 if you use mocks, use mockk to generate them.
 always run the tests after editing.
+use junit4 syntax
 
 ** security **
 determine the absolute path of the project directory. you can do that e.g. by executing pwd on the shell within the directory.

CHANGELOG.md

@@ -1,7 +1,8 @@
 # Snyk Security Changelog
 ## [2.13.1]
 ### Fixed
-- fixed not initialized exception in error handling during language server startup 
+- fixed not initialized exception in error handling during language server startup
+- fixed handling of special characters in filepaths
 
 ## [2.13.0]
 

src/main/kotlin/io/snyk/plugin/Utils.kt

@@ -421,12 +421,17 @@ fun String.toVirtualFile(): VirtualFile {
     return if (!this.startsWith("file:")) {
         StandardFileSystems.local().refreshAndFindFileByPath(this) ?: throw FileNotFoundException(this)
     } else {
-        val filePath = Paths.get(this.toFilePathString())
+        val filePath = fromUriToPath()
         VirtualFileManager.getInstance().refreshAndFindFileByNioPath(filePath)
             ?: throw FileNotFoundException(this)
     }
 }
 
+fun String.fromUriToPath(): Path {
+    val filePath = Paths.get(URI.create(this))
+    return filePath.normalize()
+}
+
 fun String.toVirtualFileOrNull(): VirtualFile? {
     return try {
         this.toVirtualFile()
@@ -436,77 +441,11 @@ fun String.toVirtualFileOrNull(): VirtualFile? {
 }
 
 fun VirtualFile.toLanguageServerURI(): String {
-    return this.url.toFileURIString()
-}
-
-/**
- * Normalizes a string that represents a file path or URI.
- *
- * This should be called on a string that represents an absolute path to a local file, or a file uri for a local file.
- * Relative paths and files on network shares are not currently supported.
- *
- * We deliberately avoid use of the Path, File and URI libraries as these will make decisions on how paths are handled
- * based on the underlying operating system. This approach provides consistency.
- *
- * @param forceUseForwardSlashes Whether to force the use of forward slashses as path separators, even for files on
- * Windows. Unix systems will always use forward slashes.
- * @return The normalized string.
- */
-private fun String.toNormalizedFilePath(forceUseForwardSlashes: Boolean): String {
-    val fileScheme = "file:"
-    val windowsSeparator = "\\"
-    val unixSeparator = "/"
-
-    // Strip the scheme and standardise separators on Unix for now.
-    val normalizedPath = this.removePrefix(fileScheme).replace(windowsSeparator, unixSeparator)
-    var targetSeparator = unixSeparator
-
-    // Split the path into parts, filtering out any blanks or references to the current directory.
-    val parts = normalizedPath.split(unixSeparator).filter { it.isNotBlank() && it != "." }.mapIndexed { idx, value ->
-        if (idx == 0) {
-            // Since we only support local files, we can use the first element of the path can tell us whether we
-            // are dealing with a Windows or unix file.
-            if (value.startsWithWindowsDriveLetter()) {
-                // Change to using Windows separators (if allowed) and capitalize the drive letter.
-                if (!forceUseForwardSlashes) targetSeparator = windowsSeparator
-                value.uppercase()
-            } else {
-                // On a Unix system, start with a slash representing root.
-                unixSeparator + value
-            }
-        } else value
-    }
-
-    // Removing any references to the parent directory (we have already removed references to the current directory).
-    val stack = mutableListOf<String>()
-    for (part in parts) {
-        if (part == "..") {
-            if (stack.isNotEmpty()) stack.removeAt(stack.size - 1)
-        } else stack.add(part)
-    }
-    return stack.joinToString(targetSeparator)
-}
-
-/**
- * Converts a string representing a file path to a normalised form. @see io.snyk.plugin.UtilsKt.toNormalizedFilePath
- */
-fun String.toFilePathString(): String {
-    return this.toNormalizedFilePath(forceUseForwardSlashes = false)
+    return this.path.fromPathToUriString()
 }
 
-/**
- * Converts a string representing a file path to a normalised form. @see io.snyk.plugin.UtilsKt.toNormalizedFilePath
- */
-fun String.toFileURIString(): String {
-    var pathString = this.toNormalizedFilePath(forceUseForwardSlashes = true)
-
-    // If we are handling a Windows path it may not have a leading slash, so add one.
-    if (pathString.startsWithWindowsDriveLetter()) {
-        pathString = "/$pathString"
-    }
-
-    // Add a file scheme. We use two slashes as standard.
-    return "file://$pathString"
+fun String.fromPathToUriString(): String {
+    return Paths.get(this).normalize().toUri().toASCIIString()
 }
 
 private fun String.startsWithWindowsDriveLetter(): Boolean {
@@ -517,7 +456,7 @@ fun VirtualFile.getDocument(): Document? = runReadAction { FileDocumentManager.g
 
 fun Project.getContentRootPaths(): SortedSet<Path> {
     return getContentRootVirtualFiles()
-        .mapNotNull { it.path.toNioPathOrNull() }
+        .mapNotNull { it.path.toNioPathOrNull()?.normalize() }
         .toSortedSet()
 }
 

src/main/kotlin/io/snyk/plugin/services/SnykTaskQueueService.kt

@@ -22,6 +22,7 @@ import io.snyk.plugin.pluginSettings
 import io.snyk.plugin.refreshAnnotationsForOpenFiles
 import io.snyk.plugin.ui.SnykBalloonNotificationHelper
 import org.jetbrains.annotations.TestOnly
+import org.jetbrains.concurrency.runAsync
 import snyk.common.lsp.LanguageServerWrapper
 import snyk.common.lsp.ScanState
 import snyk.trust.confirmScanningAndSetWorkspaceTrustedStateIfNeeded
@@ -56,7 +57,13 @@ class SnykTaskQueueService(val project: Project) {
 
         // wait for modules to be loaded and indexed so we can add all relevant content roots
         DumbService.getInstance(project).runWhenSmart {
-            languageServerWrapper.addContentRoots(project)
+            runAsync {
+                try {
+                    languageServerWrapper.addContentRoots(project)
+                } catch (e: RuntimeException) {
+                    logger.error("unable to add content roots for project $project", e)
+                }
+            }
         }
     }
 

src/main/kotlin/io/snyk/plugin/ui/SnykSettingsDialog.kt

@@ -185,7 +185,7 @@ class SnykSettingsDialog(
             // TODO: check for concrete project roots, and if we received a message for them
             // this is an edge case, when a project is opened after ls initialization and
             // preferences dialog is opened before ls sends the additional parameters
-            additionalParametersTextField.isEnabled = LanguageServerWrapper.getInstance().folderConfigsRefreshed.isNotEmpty()
+            additionalParametersTextField.isEnabled = LanguageServerWrapper.getInstance().getFolderConfigsRefreshed().isNotEmpty()
             additionalParametersTextField.text = getAdditionalParams(project)
             scanOnSaveCheckbox.isSelected = applicationSettings.scanOnSave
             cliReleaseChannelDropDown.selectedItem = applicationSettings.cliReleaseChannel

src/main/kotlin/snyk/common/lsp/LanguageServerWrapper.kt

@@ -12,14 +12,15 @@ import com.intellij.openapi.project.ProjectManager
 import com.intellij.openapi.util.Disposer
 import com.intellij.openapi.util.io.toNioPathOrNull
 import com.intellij.openapi.vfs.VirtualFile
+import io.snyk.plugin.fromUriToPath
 import io.snyk.plugin.getCliFile
 import io.snyk.plugin.getContentRootVirtualFiles
 import io.snyk.plugin.getSnykTaskQueueService
 import io.snyk.plugin.getWaitForResultsTimeout
 import io.snyk.plugin.pluginSettings
 import io.snyk.plugin.runInBackground
-import io.snyk.plugin.toFilePathString
 import io.snyk.plugin.toLanguageServerURI
+import io.snyk.plugin.ui.SnykBalloonNotificationHelper
 import io.snyk.plugin.ui.toolwindow.SnykPluginDisposable
 import org.eclipse.lsp4j.ClientCapabilities
 import org.eclipse.lsp4j.ClientInfo
@@ -64,9 +65,9 @@ import snyk.common.lsp.commands.COMMAND_WORKSPACE_FOLDER_SCAN
 import snyk.common.lsp.commands.SNYK_GENERATE_ISSUE_DESCRIPTION
 import snyk.common.lsp.progress.ProgressManager
 import snyk.common.lsp.settings.FolderConfigSettings
+import snyk.common.lsp.settings.IssueViewOptions
 import snyk.common.lsp.settings.LanguageServerSettings
 import snyk.common.lsp.settings.SeverityFilter
-import snyk.common.lsp.settings.IssueViewOptions
 import snyk.common.removeTrailingSlashesIfPresent
 import snyk.pluginInfo
 import snyk.trust.WorkspaceTrustService
@@ -97,7 +98,7 @@ class LanguageServerWrapper(
 
     // internal for test set up
     internal val configuredWorkspaceFolders: MutableSet<WorkspaceFolder> = Collections.synchronizedSet(mutableSetOf())
-    internal var folderConfigsRefreshed: MutableMap<String, Boolean> = ConcurrentHashMap()
+    private var folderConfigsRefreshed: MutableMap<String, Boolean> = ConcurrentHashMap()
     private var disposed = false
         get() {
             return ApplicationManager.getApplication().isDisposed || field
@@ -199,10 +200,10 @@ class LanguageServerWrapper(
                 LanguageServerRestartListener.getInstance()
                 refreshFeatureFlags()
             } else {
-                logger.warn("Language Server initialization did not succeed")
+                logger.error("Snyk Language Server process launch failed.")
             }
         } catch (e: Exception) {
-            logger.warn(e)
+            logger.error("Initialization of Snyk Language Server failed", e)
             if (processIsAlive()) process.destroyForcibly()
             isInitialized = false
         }
@@ -336,7 +337,7 @@ class LanguageServerWrapper(
                 added.filter { !configuredWorkspaceFolders.contains(it) },
                 removed.filter { configuredWorkspaceFolders.contains(it) },
             )
-            if (params.event.added.size > 0 || params.event.removed.size > 0) {
+            if (params.event.added.isNotEmpty() || params.event.removed.isNotEmpty()) {
                 languageServer.workspaceService.didChangeWorkspaceFolders(params)
                 configuredWorkspaceFolders.removeAll(removed)
                 configuredWorkspaceFolders.addAll(added)
@@ -485,11 +486,12 @@ class LanguageServerWrapper(
         // the folderConfigs in language server
         val folderConfigs = configuredWorkspaceFolders
             .filter {
-                val folderPath = it.uri.toFilePathString()
+                val folderPath = it.uri.fromUriToPath().toString()
                 folderConfigsRefreshed[folderPath] == true
             }.map {
-                val folderPath = it.uri.toFilePathString()
-                service<FolderConfigSettings>().getFolderConfig(folderPath) }
+                val folderPath = it.uri.fromUriToPath().toString()
+                service<FolderConfigSettings>().getFolderConfig(folderPath)
+            }
             .toList()
 
         return LanguageServerSettings(
@@ -621,7 +623,13 @@ class LanguageServerWrapper(
 
     fun addContentRoots(project: Project) {
         if (disposed || project.isDisposed) return
-        ensureLanguageServerInitialized()
+        if (!ensureLanguageServerInitialized()) {
+            SnykBalloonNotificationHelper.showWarn(
+                "Unable to initialize the Snyk Language Server. The plugin will be non-functional.",
+                project
+            )
+            return
+        }
         ensureLanguageServerProtocolVersion(project)
         updateConfiguration(false)
         val added = getWorkspaceFoldersFromRoots(project)
@@ -665,7 +673,13 @@ class LanguageServerWrapper(
         executeCommand(param)
     }
 
-    fun sendSubmitIgnoreRequestCommand(workflow: String, issueId: String, ignoreType: String, ignoreReason: String, ignoreExpirationDate: String) {
+    fun sendSubmitIgnoreRequestCommand(
+        workflow: String,
+        issueId: String,
+        ignoreType: String,
+        ignoreReason: String,
+        ignoreExpirationDate: String
+    ) {
         if (!ensureLanguageServerInitialized()) throw RuntimeException("couldn't initialize language server")
         try {
             val param = ExecuteCommandParams()
@@ -753,6 +767,15 @@ class LanguageServerWrapper(
         shutdown()
     }
 
+    fun getFolderConfigsRefreshed(): Map<String?, Boolean?> {
+        return Collections.unmodifiableMap(this.folderConfigsRefreshed)
+    }
+
+    fun updateFolderConfigRefresh(folderPath: String, refreshed: Boolean) {
+        val path = Paths.get(folderPath).normalize().toAbsolutePath().toString()
+        this.folderConfigsRefreshed[path] = refreshed
+    }
+
 
     companion object {
         private var instance: LanguageServerWrapper? = null

src/main/kotlin/snyk/common/lsp/SnykLanguageClient.kt

@@ -199,7 +199,7 @@ class SnykLanguageClient :
             val service = service<FolderConfigSettings>()
             service.addAll(folderConfigs)
             folderConfigs.forEach {
-                LanguageServerWrapper.getInstance().folderConfigsRefreshed[it.folderPath] = true
+                LanguageServerWrapper.getInstance().updateFolderConfigRefresh(it.folderPath, true)
             }
         }
     }

src/main/kotlin/snyk/common/lsp/settings/FolderConfigSettings.kt

@@ -2,11 +2,14 @@ package snyk.common.lsp.settings
 
 import com.intellij.openapi.components.Service
 import com.intellij.openapi.project.Project
+import io.snyk.plugin.fromUriToPath
 import io.snyk.plugin.getContentRootPaths
-import io.snyk.plugin.toFilePathString
+import io.snyk.plugin.suffixIfNot
 import org.jetbrains.annotations.NotNull
 import snyk.common.lsp.FolderConfig
 import snyk.common.lsp.LanguageServerWrapper
+import java.io.File
+import java.nio.file.Paths
 import java.util.concurrent.ConcurrentHashMap
 import java.util.stream.Collectors
 
@@ -17,19 +20,35 @@ class FolderConfigSettings {
 
     @Suppress("UselessCallOnNotNull", "USELESS_ELVIS", "UNNECESSARY_SAFE_CALL", "RedundantSuppression")
     fun addFolderConfig(@NotNull folderConfig: FolderConfig) {
-        if (folderConfig?.folderPath.isNullOrBlank() ?: true) return
-        configs[folderConfig.folderPath] = folderConfig
+        if (folderConfig.folderPath.isNullOrBlank()) return
+        val normalizedAbsolutePath = normalizePath(folderConfig.folderPath)
+
+        val configToStore = folderConfig.copy(folderPath = normalizedAbsolutePath)
+        configs[normalizedAbsolutePath] = configToStore
+    }
+
+    private fun normalizePath(folderPath: String): String {
+        val normalizedAbsolutePath =
+            Paths.get(folderPath)
+                .normalize()
+                .toAbsolutePath()
+                .toString()
+                .suffixIfNot(File.separator)
+        return normalizedAbsolutePath
     }
 
     internal fun getFolderConfig(folderPath: String): FolderConfig {
-        val folderConfig = configs[folderPath] ?: createEmpty(folderPath)
+        val normalizedPath = normalizePath(folderPath)
+        val folderConfig = configs[normalizedPath] ?: createEmpty(normalizedPath)
         return folderConfig
     }
 
-    private fun createEmpty(folderPath: String): FolderConfig {
-        val folderConfig = FolderConfig(folderPath = folderPath, baseBranch = "main")
-        addFolderConfig(folderConfig)
-        return folderConfig
+    private fun createEmpty(normalizedAbsolutePath: String): FolderConfig {
+        val newConfig = FolderConfig(folderPath = normalizedAbsolutePath, baseBranch = "main")
+        // Directly add to map, as addFolderConfig would re-normalize and copy, which is redundant here
+        // since normalizedAbsolutePath is already what we want for the key and the object's path.
+        configs[normalizedAbsolutePath] = newConfig
+        return newConfig
     }
 
     fun getAll(): Map<String, FolderConfig> {
@@ -58,7 +77,7 @@ class FolderConfigSettings {
         val additionalParameters = LanguageServerWrapper.getInstance().getWorkspaceFoldersFromRoots(project)
             .asSequence()
             .filter { LanguageServerWrapper.getInstance().configuredWorkspaceFolders.contains(it) }
-            .map { getFolderConfig(it.uri.toFilePathString()) }
+            .map { getFolderConfig(it.uri.fromUriToPath().toString()) }
             .filter { it.additionalParameters?.isNotEmpty() ?: false }
             .map { it.additionalParameters?.joinToString(" ") }
             .joinToString(" ")

src/main/kotlin/snyk/trust/TrustedProjects.kt

@@ -4,7 +4,6 @@ package snyk.trust
 
 import com.intellij.openapi.application.ApplicationManager
 import com.intellij.openapi.application.invokeAndWaitIfNeeded
-import com.intellij.openapi.application.runInEdt
 import com.intellij.openapi.components.service
 import com.intellij.openapi.diagnostic.Logger
 import com.intellij.openapi.project.Project

src/main/resources/html/ScanSummaryInit.html

@@ -39,13 +39,10 @@
   ${ideStyle}
 </head>
 <body>
-  <div class="snx-header">
-    <h1 class="snx-title snx-h1">Snyk Security is loading...</h1>
-  </div>
   <div class="snx-summary">
     <div class="snx-status">
       <span class="snx-loader"></span>
-      <p class="snx-message">Waiting for the Snyk CLI to be downloaded and the Language Server to be initialized. </p>
+      <p class="snx-message">Snyk Security is loading...</p>
     </div>
   </div>
 </body>