{- An automated benchmark built around the simple experiment described in: > https://neilmitchell.blogspot.com/2020/05/fixing-space-leaks-in-ghcide.html As an example project, it unpacks Cabal-3.2.0.0 in the local filesystem and loads the module 'Distribution.Simple'. The rationale for this choice is: - It's convenient to download with `cabal unpack Cabal-3.2.0.0` - It has very few dependencies, and all are already needed to build ghcide - Distribution.Simple has 235 transitive module dependencies, so non trivial The experiments are sequences of lsp commands scripted using lsp-test. A more refined approach would be to record and replay real IDE interactions, once the replay functionality is available in lsp-test. A more declarative approach would be to reuse ide-debug-driver: > https://github.com/digital-asset/daml/blob/master/compiler/damlc/ide-debug-driver/README.md The result of an experiment is a total duration in seconds after a preset number of iterations. There is ample room for improvement: - Statistical analysis to detect outliers and auto infer the number of iterations needed - GC stats analysis (currently -S is printed as part of the experiment) - Analysis of performance over the commit history of the project How to run: 1. `cabal exec cabal run ghcide-bench -- -- ghcide-bench-options` 1. `stack build ghcide:ghcide-bench && stack exec ghcide-bench -- -- ghcide-bench-options` Note that the package database influences the response times of certain actions, e.g. code actions, and therefore the two methods above do not necessarily produce the same results. -} {-# LANGUAGE ImplicitParams #-} import Control.Exception.Safe import Control.Monad import Experiments import Options.Applicative import System.IO optsP :: Parser (Config, Bool) optsP = (,) <$> configP <*> switch (long "no-clean") main :: IO () main = do hSetBuffering stdout LineBuffering hSetBuffering stderr LineBuffering (config, noClean) <- execParser $ info (optsP <**> helper) fullDesc let ?config = config hPrint stderr config output "starting test" SetupResult{..} <- setup runBenchmarks experiments `finally` unless noClean cleanUp