1 # go-toml v2

     2 

     3 Go library for the [TOML](https://toml.io/en/) format.

     4 

     5 This library supports [TOML v1.0.0](https://toml.io/en/v1.0.0).

     6 

     7 [🐞 Bug Reports](https://github.com/pelletier/go-toml/issues)

     8 

     9 [💬 Anything else](https://github.com/pelletier/go-toml/discussions)

    10 

    11 ## Documentation

    12 

    13 Full API, examples, and implementation notes are available in the Go

    14 documentation.

    15 

    16 [![Go Reference](https://pkg.go.dev/badge/github.com/pelletier/go-toml/v2.svg)](https://pkg.go.dev/github.com/pelletier/go-toml/v2)

    17 

    18 ## Import

    19 

    20 ```go

    21 import "github.com/pelletier/go-toml/v2"

    22 ```

    23 

    24 See [Modules](#Modules).

    25 

    26 ## Features

    27 

    28 ### Stdlib behavior

    29 

    30 As much as possible, this library is designed to behave similarly as the

    31 standard library's `encoding/json`.

    32 

    33 ### Performance

    34 

    35 While go-toml favors usability, it is written with performance in mind. Most

    36 operations should not be shockingly slow. See [benchmarks](#benchmarks).

    37 

    38 ### Strict mode

    39 

    40 `Decoder` can be set to "strict mode", which makes it error when some parts of

    41 the TOML document was not present in the target structure. This is a great way

    42 to check for typos. [See example in the documentation][strict].

    43 

    44 [strict]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#example-Decoder.DisallowUnknownFields

    45 

    46 ### Contextualized errors

    47 

    48 When most decoding errors occur, go-toml returns [`DecodeError`][decode-err]),

    49 which contains a human readable contextualized version of the error. For

    50 example:

    51 

    52 ```

    53 2| key1 = "value1"

    54 3| key2 = "missing2"

    55  | ~~~~ missing field

    56 4| key3 = "missing3"

    57 5| key4 = "value4"

    58 ```

    59 

    60 [decode-err]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#DecodeError

    61 

    62 ### Local date and time support

    63 

    64 TOML supports native [local date/times][ldt]. It allows to represent a given

    65 date, time, or date-time without relation to a timezone or offset. To support

    66 this use-case, go-toml provides [`LocalDate`][tld], [`LocalTime`][tlt], and

    67 [`LocalDateTime`][tldt]. Those types can be transformed to and from `time.Time`,

    68 making them convenient yet unambiguous structures for their respective TOML

    69 representation.

    70 

    71 [ldt]: https://toml.io/en/v1.0.0#local-date-time

    72 [tld]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#LocalDate

    73 [tlt]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#LocalTime

    74 [tldt]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#LocalDateTime

    75 

    76 ## Getting started

    77 

    78 Given the following struct, let's see how to read it and write it as TOML:

    79 

    80 ```go

    81 type MyConfig struct {

    82       Version int

    83       Name    string

    84       Tags    []string

    85 }

    86 ```

    87 

    88 ### Unmarshaling

    89 

    90 [`Unmarshal`][unmarshal] reads a TOML document and fills a Go structure with its

    91 content. For example:

    92 

    93 ```go

    94 doc := `

    95 version = 2

    96 name = "go-toml"

    97 tags = ["go", "toml"]

    98 `

    99 

   100 var cfg MyConfig

   101 err := toml.Unmarshal([]byte(doc), &cfg)

   102 if err != nil {

   103       panic(err)

   104 }

   105 fmt.Println("version:", cfg.Version)

   106 fmt.Println("name:", cfg.Name)

   107 fmt.Println("tags:", cfg.Tags)

   108 

   109 // Output:

   110 // version: 2

   111 // name: go-toml

   112 // tags: [go toml]

   113 ```

   114 

   115 [unmarshal]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#Unmarshal

   116 

   117 ### Marshaling

   118 

   119 [`Marshal`][marshal] is the opposite of Unmarshal: it represents a Go structure

   120 as a TOML document:

   121 

   122 ```go

   123 cfg := MyConfig{

   124       Version: 2,

   125       Name:    "go-toml",

   126       Tags:    []string{"go", "toml"},

   127 }

   128 

   129 b, err := toml.Marshal(cfg)

   130 if err != nil {

   131       panic(err)

   132 }

   133 fmt.Println(string(b))

   134 

   135 // Output:

   136 // Version = 2

   137 // Name = 'go-toml'

   138 // Tags = ['go', 'toml']

   139 ```

   140 

   141 [marshal]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#Marshal

   142 

   143 ## Benchmarks

   144 

   145 Execution time speedup compared to other Go TOML libraries:

   146 

   147 <table>

   148     <thead>

   149         <tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr>

   150     </thead>

   151     <tbody>

   152         <tr><td>Marshal/HugoFrontMatter-2</td><td>1.9x</td><td>1.9x</td></tr>

   153         <tr><td>Marshal/ReferenceFile/map-2</td><td>1.7x</td><td>1.8x</td></tr>

   154         <tr><td>Marshal/ReferenceFile/struct-2</td><td>2.2x</td><td>2.5x</td></tr>

   155         <tr><td>Unmarshal/HugoFrontMatter-2</td><td>2.9x</td><td>2.9x</td></tr>

   156         <tr><td>Unmarshal/ReferenceFile/map-2</td><td>2.6x</td><td>2.9x</td></tr>

   157         <tr><td>Unmarshal/ReferenceFile/struct-2</td><td>4.4x</td><td>5.3x</td></tr>

   158      </tbody>

   159 </table>

   160 <details><summary>See more</summary>

   161 <p>The table above has the results of the most common use-cases. The table below

   162 contains the results of all benchmarks, including unrealistic ones. It is

   163 provided for completeness.</p>

   164 

   165 <table>

   166     <thead>

   167         <tr><th>Benchmark</th><th>go-toml v1</th><th>BurntSushi/toml</th></tr>

   168     </thead>

   169     <tbody>

   170         <tr><td>Marshal/SimpleDocument/map-2</td><td>1.8x</td><td>2.9x</td></tr>

   171         <tr><td>Marshal/SimpleDocument/struct-2</td><td>2.7x</td><td>4.2x</td></tr>

   172         <tr><td>Unmarshal/SimpleDocument/map-2</td><td>4.5x</td><td>3.1x</td></tr>

   173         <tr><td>Unmarshal/SimpleDocument/struct-2</td><td>6.2x</td><td>3.9x</td></tr>

   174         <tr><td>UnmarshalDataset/example-2</td><td>3.1x</td><td>3.5x</td></tr>

   175         <tr><td>UnmarshalDataset/code-2</td><td>2.3x</td><td>3.1x</td></tr>

   176         <tr><td>UnmarshalDataset/twitter-2</td><td>2.5x</td><td>2.6x</td></tr>

   177         <tr><td>UnmarshalDataset/citm_catalog-2</td><td>2.1x</td><td>2.2x</td></tr>

   178         <tr><td>UnmarshalDataset/canada-2</td><td>1.6x</td><td>1.3x</td></tr>

   179         <tr><td>UnmarshalDataset/config-2</td><td>4.3x</td><td>3.2x</td></tr>

   180         <tr><td>[Geo mean]</td><td>2.7x</td><td>2.8x</td></tr>

   181      </tbody>

   182 </table>

   183 <p>This table can be generated with <code>./ci.sh benchmark -a -html</code>.</p>

   184 </details>

   185 

   186 ## Modules

   187 

   188 go-toml uses Go's standard modules system.

   189 

   190 Installation instructions:

   191 

   192 - Go ≥ 1.16: Nothing to do. Use the import in your code. The `go` command deals

   193   with it automatically.

   194 - Go ≥ 1.13: `GO111MODULE=on go get github.com/pelletier/go-toml/v2`.

   195 

   196 In case of trouble: [Go Modules FAQ][mod-faq].

   197 

   198 [mod-faq]: https://github.com/golang/go/wiki/Modules#why-does-installing-a-tool-via-go-get-fail-with-error-cannot-find-main-module

   199 

   200 ## Tools

   201 

   202 Go-toml provides three handy command line tools:

   203 

   204  * `tomljson`: Reads a TOML file and outputs its JSON representation.

   205 

   206     ```

   207     $ go install github.com/pelletier/go-toml/v2/cmd/tomljson@latest

   208     $ tomljson --help

   209     ```

   210 

   211  * `jsontoml`: Reads a JSON file and outputs a TOML representation.

   212 

   213     ```

   214     $ go install github.com/pelletier/go-toml/v2/cmd/jsontoml@latest

   215     $ jsontoml --help

   216     ```

   217 

   218  * `tomll`: Lints and reformats a TOML file.

   219 

   220     ```

   221     $ go install github.com/pelletier/go-toml/v2/cmd/tomll@latest

   222     $ tomll --help

   223     ```

   224 

   225 ### Docker image

   226 

   227 Those tools are also available as a [Docker image][docker]. For example, to use

   228 `tomljson`:

   229 

   230 ```

   231 docker run -i ghcr.io/pelletier/go-toml:v2 tomljson < example.toml

   232 ```

   233 

   234 Multiple versions are availble on [ghcr.io][docker].

   235 

   236 [docker]: https://github.com/pelletier/go-toml/pkgs/container/go-toml

   237 

   238 ## Migrating from v1

   239 

   240 This section describes the differences between v1 and v2, with some pointers on

   241 how to get the original behavior when possible.

   242 

   243 ### Decoding / Unmarshal

   244 

   245 #### Automatic field name guessing

   246 

   247 When unmarshaling to a struct, if a key in the TOML document does not exactly

   248 match the name of a struct field or any of the `toml`-tagged field, v1 tries

   249 multiple variations of the key ([code][v1-keys]).

   250 

   251 V2 instead does a case-insensitive matching, like `encoding/json`.

   252 

   253 This could impact you if you are relying on casing to differentiate two fields,

   254 and one of them is a not using the `toml` struct tag. The recommended solution

   255 is to be specific about tag names for those fields using the `toml` struct tag.

   256 

   257 [v1-keys]: https://github.com/pelletier/go-toml/blob/a2e52561804c6cd9392ebf0048ca64fe4af67a43/marshal.go#L775-L781

   258 

   259 #### Ignore preexisting value in interface

   260 

   261 When decoding into a non-nil `interface{}`, go-toml v1 uses the type of the

   262 element in the interface to decode the object. For example:

   263 

   264 ```go

   265 type inner struct {

   266   B interface{}

   267 }

   268 type doc struct {

   269   A interface{}

   270 }

   271 

   272 d := doc{

   273   A: inner{

   274     B: "Before",

   275   },

   276 }

   277 

   278 data := `

   279 [A]

   280 B = "After"

   281 `

   282 

   283 toml.Unmarshal([]byte(data), &d)

   284 fmt.Printf("toml v1: %#v\n", d)

   285 

   286 // toml v1: main.doc{A:main.inner{B:"After"}}

   287 ```

   288 

   289 In this case, field `A` is of type `interface{}`, containing a `inner` struct.

   290 V1 sees that type and uses it when decoding the object.

   291 

   292 When decoding an object into an `interface{}`, V2 instead disregards whatever

   293 value the `interface{}` may contain and replaces it with a

   294 `map[string]interface{}`. With the same data structure as above, here is what

   295 the result looks like:

   296 

   297 ```go

   298 toml.Unmarshal([]byte(data), &d)

   299 fmt.Printf("toml v2: %#v\n", d)

   300 

   301 // toml v2: main.doc{A:map[string]interface {}{"B":"After"}}

   302 ```

   303 

   304 This is to match `encoding/json`'s behavior. There is no way to make the v2

   305 decoder behave like v1.

   306 

   307 #### Values out of array bounds ignored

   308 

   309 When decoding into an array, v1 returns an error when the number of elements

   310 contained in the doc is superior to the capacity of the array. For example:

   311 

   312 ```go

   313 type doc struct {

   314   A [2]string

   315 }

   316 d := doc{}

   317 err := toml.Unmarshal([]byte(`A = ["one", "two", "many"]`), &d)

   318 fmt.Println(err)

   319 

   320 // (1, 1): unmarshal: TOML array length (3) exceeds destination array length (2)

   321 ```

   322 

   323 In the same situation, v2 ignores the last value:

   324 

   325 ```go

   326 err := toml.Unmarshal([]byte(`A = ["one", "two", "many"]`), &d)

   327 fmt.Println("err:", err, "d:", d)

   328 // err: <nil> d: {[one two]}

   329 ```

   330 

   331 This is to match `encoding/json`'s behavior. There is no way to make the v2

   332 decoder behave like v1.

   333 

   334 #### Support for `toml.Unmarshaler` has been dropped

   335 

   336 This method was not widely used, poorly defined, and added a lot of complexity.

   337 A similar effect can be achieved by implementing the `encoding.TextUnmarshaler`

   338 interface and use strings.

   339 

   340 #### Support for `default` struct tag has been dropped

   341 

   342 This feature adds complexity and a poorly defined API for an effect that can be

   343 accomplished outside of the library.

   344 

   345 It does not seem like other format parsers in Go support that feature (the

   346 project referenced in the original ticket #202 has not been updated since 2017).

   347 Given that go-toml v2 should not touch values not in the document, the same

   348 effect can be achieved by pre-filling the struct with defaults (libraries like

   349 [go-defaults][go-defaults] can help). Also, string representation is not well

   350 defined for all types: it creates issues like #278.

   351 

   352 The recommended replacement is pre-filling the struct before unmarshaling.

   353 

   354 [go-defaults]: https://github.com/mcuadros/go-defaults

   355 

   356 #### `toml.Tree` replacement

   357 

   358 This structure was the initial attempt at providing a document model for

   359 go-toml. It allows manipulating the structure of any document, encoding and

   360 decoding from their TOML representation. While a more robust feature was

   361 initially planned in go-toml v2, this has been ultimately [removed from

   362 scope][nodoc] of this library, with no plan to add it back at the moment. The

   363 closest equivalent at the moment would be to unmarshal into an `interface{}` and

   364 use type assertions and/or reflection to manipulate the arbitrary

   365 structure. However this would fall short of providing all of the TOML features

   366 such as adding comments and be specific about whitespace.

   367 

   368 

   369 #### `toml.Position` are not retrievable anymore

   370 

   371 The API for retrieving the position (line, column) of a specific TOML element do

   372 not exist anymore. This was done to minimize the amount of concepts introduced

   373 by the library (query path), and avoid the performance hit related to storing

   374 positions in the absence of a document model, for a feature that seemed to have

   375 little use. Errors however have gained more detailed position

   376 information. Position retrieval seems better fitted for a document model, which

   377 has been [removed from the scope][nodoc] of go-toml v2 at the moment.

   378 

   379 ### Encoding / Marshal

   380 

   381 #### Default struct fields order

   382 

   383 V1 emits struct fields order alphabetically by default. V2 struct fields are

   384 emitted in order they are defined. For example:

   385 

   386 ```go

   387 type S struct {

   388 	B string

   389 	A string

   390 }

   391 

   392 data := S{

   393 	B: "B",

   394 	A: "A",

   395 }

   396 

   397 b, _ := tomlv1.Marshal(data)

   398 fmt.Println("v1:\n" + string(b))

   399 

   400 b, _ = tomlv2.Marshal(data)

   401 fmt.Println("v2:\n" + string(b))

   402 

   403 // Output:

   404 // v1:

   405 // A = "A"

   406 // B = "B"

   407 

   408 // v2:

   409 // B = 'B'

   410 // A = 'A'

   411 ```

   412 

   413 There is no way to make v2 encoder behave like v1. A workaround could be to

   414 manually sort the fields alphabetically in the struct definition, or generate

   415 struct types using `reflect.StructOf`.

   416 

   417 #### No indentation by default

   418 

   419 V1 automatically indents content of tables by default. V2 does not. However the

   420 same behavior can be obtained using [`Encoder.SetIndentTables`][sit]. For example:

   421 

   422 ```go

   423 data := map[string]interface{}{

   424 	"table": map[string]string{

   425 		"key": "value",

   426 	},

   427 }

   428 

   429 b, _ := tomlv1.Marshal(data)

   430 fmt.Println("v1:\n" + string(b))

   431 

   432 b, _ = tomlv2.Marshal(data)

   433 fmt.Println("v2:\n" + string(b))

   434 

   435 buf := bytes.Buffer{}

   436 enc := tomlv2.NewEncoder(&buf)

   437 enc.SetIndentTables(true)

   438 enc.Encode(data)

   439 fmt.Println("v2 Encoder:\n" + string(buf.Bytes()))

   440 

   441 // Output:

   442 // v1:

   443 //

   444 // [table]

   445 //   key = "value"

   446 //

   447 // v2:

   448 // [table]

   449 // key = 'value'

   450 //

   451 //

   452 // v2 Encoder:

   453 // [table]

   454 //   key = 'value'

   455 ```

   456 

   457 [sit]: https://pkg.go.dev/github.com/pelletier/go-toml/v2#Encoder.SetIndentTables

   458 

   459 #### Keys and strings are single quoted

   460 

   461 V1 always uses double quotes (`"`) around strings and keys that cannot be

   462 represented bare (unquoted). V2 uses single quotes instead by default (`'`),

   463 unless a character cannot be represented, then falls back to double quotes. As a

   464 result of this change, `Encoder.QuoteMapKeys` has been removed, as it is not

   465 useful anymore.

   466 

   467 There is no way to make v2 encoder behave like v1.

   468 

   469 #### `TextMarshaler` emits as a string, not TOML

   470 

   471 Types that implement [`encoding.TextMarshaler`][tm] can emit arbitrary TOML in

   472 v1. The encoder would append the result to the output directly. In v2 the result

   473 is wrapped in a string. As a result, this interface cannot be implemented by the

   474 root object.

   475 

   476 There is no way to make v2 encoder behave like v1.

   477 

   478 [tm]: https://golang.org/pkg/encoding/#TextMarshaler

   479 

   480 #### `Encoder.CompactComments` has been removed

   481 

   482 Emitting compact comments is now the default behavior of go-toml. This option

   483 is not necessary anymore.

   484 

   485 #### Struct tags have been merged

   486 

   487 V1 used to provide multiple struct tags: `comment`, `commented`, `multiline`,

   488 `toml`, and `omitempty`. To behave more like the standard library, v2 has merged

   489 `toml`, `multiline`, and `omitempty`. For example:

   490 

   491 ```go

   492 type doc struct {

   493 	// v1

   494 	F string `toml:"field" multiline:"true" omitempty:"true"`

   495 	// v2

   496 	F string `toml:"field,multiline,omitempty"`

   497 }

   498 ```

   499 

   500 Has a result, the `Encoder.SetTag*` methods have been removed, as there is just

   501 one tag now.

   502 

   503 

   504 #### `commented` tag has been removed

   505 

   506 There is no replacement for the `commented` tag. This feature would be better

   507 suited in a proper document model for go-toml v2, which has been [cut from

   508 scope][nodoc] at the moment.

   509 

   510 #### `Encoder.ArraysWithOneElementPerLine` has been renamed

   511 

   512 The new name is `Encoder.SetArraysMultiline`. The behavior should be the same.

   513 

   514 #### `Encoder.Indentation` has been renamed

   515 

   516 The new name is `Encoder.SetIndentSymbol`. The behavior should be the same.

   517 

   518 

   519 #### Embedded structs behave like stdlib

   520 

   521 V1 defaults to merging embedded struct fields into the embedding struct. This

   522 behavior was unexpected because it does not follow the standard library. To

   523 avoid breaking backward compatibility, the `Encoder.PromoteAnonymous` method was

   524 added to make the encoder behave correctly. Given backward compatibility is not

   525 a problem anymore, v2 does the right thing by default: it follows the behavior

   526 of `encoding/json`. `Encoder.PromoteAnonymous` has been removed.

   527 

   528 [nodoc]: https://github.com/pelletier/go-toml/discussions/506#discussioncomment-1526038

   529 

   530 ### `query`

   531 

   532 go-toml v1 provided the [`go-toml/query`][query] package. It allowed to run

   533 JSONPath-style queries on TOML files. This feature is not available in v2. For a

   534 replacement, check out [dasel][dasel].

   535 

   536 This package has been removed because it was essentially not supported anymore

   537 (last commit May 2020), increased the complexity of the code base, and more

   538 complete solutions exist out there.

   539 

   540 [query]: https://github.com/pelletier/go-toml/tree/f99d6bbca119636aeafcf351ee52b3d202782627/query

   541 [dasel]: https://github.com/TomWright/dasel

   542 

   543 ## Versioning

   544 

   545 Go-toml follows [Semantic Versioning](http://semver.org/). The supported version

   546 of [TOML](https://github.com/toml-lang/toml) is indicated at the beginning of

   547 this document. The last two major versions of Go are supported

   548 (see [Go Release Policy](https://golang.org/doc/devel/release.html#policy)).

   549 

   550 ## License

   551 

   552 The MIT License (MIT). Read [LICENSE](LICENSE).