IpeDB
The IpeDB package contains a library for building and using GHC IPE and cost-centre databases, as well as two command-line tools, ipedb and ccdb, that provide a command-line interface to the library.
Working with IPE data using ipedb
What is IPE data?
Broadly speaking, Info Tables are the runtime representation for Haskell symbols—constructors, functions, etc. An IPE, or Info Table Provenance Entry—the T is silent—contains provenance information for its Info Table, such as the symbol's name and its source location. This can be useful for analysing heap profiles generated with -hi or stack profiles generated by ghc-stack-profiler, which by themselves only contain Info Table IDs.
An IPE database is a lookup table from Info Table ID to Info Table. An Info Table ID is a stable identifier that can be used to look up the Info Table. Since GHC 10, these IDs are determined statically at compile-time, which means that an IPE database can only be reused with the same binary.
⚠️ Warning:
IPE data is not portable across builds.
Prior to GHC 10, an Info Table ID was the actual address of the Info Table in memory. Since these were computed by the linker, they would change for every execution, which meant that IPE databases were not portable across executions of the same binary. For binaries built using those older GHC versions, you can only use an IPE database built from an eventlog to analyse that eventlog.
Stable Info Table IDs were introduced in merge request !14465.
How to build a binary with IPE data?
To build a package with IPE data, you must pass -finfo-table-map and (optionally, but recommended) -fdistinct-constructor-tables to GHC.
You can add the following to your cabal.project file to instruct Cabal to build all packages with these options:
package *
ghc-options:
-finfo-table-map
-fdistinct-constructor-tables
Unfortunately, this won't add IPE data to boot libraries that ship with GHC or to GHC builtins. To get this additional information, you must build GHC with the ipe flavour, see GHC Build flavours.
How to build an IPE database with ipedb?
The ipedb tool extracts IPE data from a binary's eventlog. This is produced when the binary is run with the -l RTS option. This means your binary must be built with support for RTS options. You can add the following to your *.cabal file:
executable my-exe
-- ...
ghc-options: -rtsopts
-- ...
You can obtain an eventlog by running your binary with the -l RTS option.
my-exe +RTS -l -olmy.eventlog -RTS
The -ol RTS option controls the output filename. If this is not provided, it defaults to your program's name with the .eventlog extension. If you built your program with GHC >=10, you can pass -l-aI to generate an eventlog that only contains IPE events.
Your program does not need to run for long to generate IPE events, as these are emitted during RTS initialisation. If your program has a --version or --help option, that is more than sufficient.
To generate an IPE database, you run ipedb index with the eventlog and an output file:
ipedb index my.eventlog -o my.ipedb
Optionally, you can specify the table format with the --table-format=<fmt> flag.
The current implementation supports three formats:
- lsm: This format creates a directory that contains an
lsm-tree snapshot. This format is the fastest to write, since it hard links to the runtime database files, but requires that the session root and the target path are on the same filesystem.
- tar: This format creates a tar archive of an lsm export. This requires copying the database files.
- tgz: This format creates a compressed tar export. This format is the slowest to write, since it requires compressing the database files, but the it results in significantly smaller files.
The default table format is tgz.
How to query an IPE database with ipedb?
To query an IPE database, you run ipedb query with the IPE database and a series of Info Table IDs:
ipedb query my.ipedb [IPE_ID...]
For example, if we create an IPE database from the eventlog in test/data/oddball.eventlog, and run ipedb query with 0x100000000, 0x350000050f, and 0x1e000006c9, we get:
0x100000000: Just (InfoProv {ipName = "I#_Main_1_con_info", ipClosureDesc = 3, ipTyDesc = "Int", ipLabel = "main", ipModule = "Main", ipSrcLoc = SrcLoc {srcFilePath = Just "app/Main.hs", srcRange = Just (Range'MultiLine {line = 13, column = 1, endLine = 16, endColumn = 11})}})
0x350000050f: Just (InfoProv {ipName = "(,)_GHC.Internal.Data.Data_327_con_info", ipClosureDesc = 4, ipTyDesc = "(,)", ipLabel = "gmapMo.k", ipModule = "GHC.Internal.Data.Data", ipSrcLoc = SrcLoc {srcFilePath = Just "libraries/ghc-internal/src/GHC/Internal/Data/Data.hs", srcRange = Just (Range'OneLine {line = 439, column = 30, endColumn = 44})}})
0x1e000006c9: Just (InfoProv {ipName = "block_caBN_info", ipClosureDesc = 53, ipTyDesc = "", ipLabel = "decodeFloat", ipModule = "Data.Complex", ipSrcLoc = SrcLoc {srcFilePath = Just "libraries/ghc-internal/src/GHC/Internal/Float.hs", srcRange = Just (Range'MultiLine {line = 651, column = 5, endLine = 652, endColumn = 52})}})
If you wish to list all IPE data in an IPE database, you can run ipedb list with the IPE database:
ipedb list my.ipedb
Be warned that this prints a lot of information.
Working with cost-centre data using ccdb
What is cost-centre data?
Cost Centres are virtual symbols that correspond closely to the Haskell source symbols. A Haskell binary built with GHC's profiling maintains a virtual cost-centre stack at runtime. The cost-centre stack contains entries for all functions annotated by the user with a {-# SCC myFunctionName #-} pragma or by some annotation strategy (e.g., -fprof-auto or -fprof-late). The recommended annotation strategy is -fprof-late, see Late Cost Centre Profiling.
Cost Centres are useful when analysing heap profiles generated with -hc or stack profiles generated by -p, which by themselves only contain Cost Centre IDs.
A cost-centre database is a lookup table from Cost Centre ID to Cost Centre. A Cost Centre ID is a stable identifier that can be used to look up the Cost Centre. When building with the GHC option -fobject-determinism, the cost-centre data for a binary is deterministic. This means that if -fobject-determinism is passed to GHC and the GHC version, package source, and Cabal build plan are all identical, then the generated cost-centre data should be identical.
How to build a binary with cost-centre data?
For a detailed overview of how to build a binary with cost-centre data, see the GHC users guide. At the time of writing, our recommendation is to add the following to your cabal.project file to instruct Cabal to build all packages with late cost-centre profiling:
profiling: True
package *
profiling-detail: late
How to build a cost-centre database with ccdb?
The ccdb command-line interface follows the same conventions as ipedb. Build your binary with cost-centre data and follow the instructions under How to build an IPE database with ipedb? using ccdb instead of ipedb.
How to query a cost-centre database with ccdb?
The ccdb command-line interface follows the same conventions as ipedb. Build your binary with cost-centre data and follow the instructions under How to build an IPE database with ipedb? using ccdb instead of ipedb.
