1 ![viper logo](https://cloud.githubusercontent.com/assets/173412/10886745/998df88a-8151-11e5-9448-4736db51020d.png) |
1 > ## Viper v2 feedback |
2 |
2 > Viper is heading towards v2 and we would love to hear what _**you**_ would like to see in it. Share your thoughts here: https://forms.gle/R6faU74qPRPAzchZ9 |
3 Go configuration with fangs! |
3 > |
4 |
4 > **Thank you!** |
5 [![Actions](https://github.com/spf13/viper/workflows/CI/badge.svg)](https://github.com/spf13/viper) |
5 |
|
6 ![Viper](.github/logo.png?raw=true) |
|
7 |
|
8 [![Mentioned in Awesome Go](https://awesome.re/mentioned-badge-flat.svg)](https://github.com/avelino/awesome-go#configuration) |
|
9 [![run on repl.it](https://repl.it/badge/github/sagikazarmark/Viper-example)](https://repl.it/@sagikazarmark/Viper-example#main.go) |
|
10 |
|
11 [![GitHub Workflow Status](https://img.shields.io/github/workflow/status/spf13/viper/CI?style=flat-square)](https://github.com/spf13/viper/actions?query=workflow%3ACI) |
6 [![Join the chat at https://gitter.im/spf13/viper](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/spf13/viper?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) |
12 [![Join the chat at https://gitter.im/spf13/viper](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/spf13/viper?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) |
7 [![GoDoc](https://godoc.org/github.com/spf13/viper?status.svg)](https://godoc.org/github.com/spf13/viper) |
13 [![Go Report Card](https://goreportcard.com/badge/github.com/spf13/viper?style=flat-square)](https://goreportcard.com/report/github.com/spf13/viper) |
|
14 ![Go Version](https://img.shields.io/badge/go%20version-%3E=1.14-61CFDD.svg?style=flat-square) |
|
15 [![PkgGoDev](https://pkg.go.dev/badge/mod/github.com/spf13/viper)](https://pkg.go.dev/mod/github.com/spf13/viper) |
|
16 |
|
17 **Go configuration with fangs!** |
8 |
18 |
9 Many Go projects are built using Viper including: |
19 Many Go projects are built using Viper including: |
10 |
20 |
11 * [Hugo](http://gohugo.io) |
21 * [Hugo](http://gohugo.io) |
12 * [EMC RexRay](http://rexray.readthedocs.org/en/stable/) |
22 * [EMC RexRay](http://rexray.readthedocs.org/en/stable/) |
105 viper.AddConfigPath("/etc/appname/") // path to look for the config file in |
117 viper.AddConfigPath("/etc/appname/") // path to look for the config file in |
106 viper.AddConfigPath("$HOME/.appname") // call multiple times to add many search paths |
118 viper.AddConfigPath("$HOME/.appname") // call multiple times to add many search paths |
107 viper.AddConfigPath(".") // optionally look for config in the working directory |
119 viper.AddConfigPath(".") // optionally look for config in the working directory |
108 err := viper.ReadInConfig() // Find and read the config file |
120 err := viper.ReadInConfig() // Find and read the config file |
109 if err != nil { // Handle errors reading the config file |
121 if err != nil { // Handle errors reading the config file |
110 panic(fmt.Errorf("Fatal error config file: %s \n", err)) |
122 panic(fmt.Errorf("Fatal error config file: %w \n", err)) |
111 } |
123 } |
112 ``` |
124 ``` |
113 |
125 |
114 You can handle the specific case where no config file is found like this: |
126 You can handle the specific case where no config file is found like this: |
115 |
127 |
240 Viper provides a mechanism to try to ensure that ENV variables are unique. By |
252 Viper provides a mechanism to try to ensure that ENV variables are unique. By |
241 using `SetEnvPrefix`, you can tell Viper to use a prefix while reading from |
253 using `SetEnvPrefix`, you can tell Viper to use a prefix while reading from |
242 the environment variables. Both `BindEnv` and `AutomaticEnv` will use this |
254 the environment variables. Both `BindEnv` and `AutomaticEnv` will use this |
243 prefix. |
255 prefix. |
244 |
256 |
245 `BindEnv` takes one or two parameters. The first parameter is the key name, the |
257 `BindEnv` takes one or more parameters. The first parameter is the key name, the |
246 second is the name of the environment variable. The name of the environment |
258 rest are the name of the environment variables to bind to this key. If more than |
247 variable is case sensitive. If the ENV variable name is not provided, then |
259 one are provided, they will take precedence in the specified order. The name of |
|
260 the environment variable is case sensitive. If the ENV variable name is not provided, then |
248 Viper will automatically assume that the ENV variable matches the following format: prefix + "_" + the key name in ALL CAPS. When you explicitly provide the ENV variable name (the second parameter), |
261 Viper will automatically assume that the ENV variable matches the following format: prefix + "_" + the key name in ALL CAPS. When you explicitly provide the ENV variable name (the second parameter), |
249 it **does not** automatically add the prefix. For example if the second parameter is "id", |
262 it **does not** automatically add the prefix. For example if the second parameter is "id", |
250 Viper will look for the ENV variable "ID". |
263 Viper will look for the ENV variable "ID". |
251 |
264 |
252 One important thing to recognize when working with ENV variables is that the |
265 One important thing to recognize when working with ENV variables is that the |
254 the `BindEnv` is called. |
267 the `BindEnv` is called. |
255 |
268 |
256 `AutomaticEnv` is a powerful helper especially when combined with |
269 `AutomaticEnv` is a powerful helper especially when combined with |
257 `SetEnvPrefix`. When called, Viper will check for an environment variable any |
270 `SetEnvPrefix`. When called, Viper will check for an environment variable any |
258 time a `viper.Get` request is made. It will apply the following rules. It will |
271 time a `viper.Get` request is made. It will apply the following rules. It will |
259 check for a environment variable with a name matching the key uppercased and |
272 check for an environment variable with a name matching the key uppercased and |
260 prefixed with the `EnvPrefix` if set. |
273 prefixed with the `EnvPrefix` if set. |
261 |
274 |
262 `SetEnvKeyReplacer` allows you to use a `strings.Replacer` object to rewrite Env |
275 `SetEnvKeyReplacer` allows you to use a `strings.Replacer` object to rewrite Env |
263 keys to an extent. This is useful if you want to use `-` or something in your |
276 keys to an extent. This is useful if you want to use `-` or something in your |
264 `Get()` calls, but want your environmental variables to use `_` delimiters. An |
277 `Get()` calls, but want your environmental variables to use `_` delimiters. An |
398 Viper will read a config string (as JSON, TOML, YAML, HCL or envfile) retrieved from a path |
411 Viper will read a config string (as JSON, TOML, YAML, HCL or envfile) retrieved from a path |
399 in a Key/Value store such as etcd or Consul. These values take precedence over |
412 in a Key/Value store such as etcd or Consul. These values take precedence over |
400 default values, but are overridden by configuration values retrieved from disk, |
413 default values, but are overridden by configuration values retrieved from disk, |
401 flags, or environment variables. |
414 flags, or environment variables. |
402 |
415 |
403 Viper uses [crypt](https://github.com/xordataexchange/crypt) to retrieve |
416 Viper uses [crypt](https://github.com/bketelsen/crypt) to retrieve |
404 configuration from the K/V store, which means that you can store your |
417 configuration from the K/V store, which means that you can store your |
405 configuration values encrypted and have them automatically decrypted if you have |
418 configuration values encrypted and have them automatically decrypted if you have |
406 the correct gpg keyring. Encryption is optional. |
419 the correct gpg keyring. Encryption is optional. |
407 |
420 |
408 You can use remote configuration in conjunction with local configuration, or |
421 You can use remote configuration in conjunction with local configuration, or |
410 |
423 |
411 `crypt` has a command-line helper that you can use to put configurations in your |
424 `crypt` has a command-line helper that you can use to put configurations in your |
412 K/V store. `crypt` defaults to etcd on http://127.0.0.1:4001. |
425 K/V store. `crypt` defaults to etcd on http://127.0.0.1:4001. |
413 |
426 |
414 ```bash |
427 ```bash |
415 $ go get github.com/xordataexchange/crypt/bin/crypt |
428 $ go get github.com/bketelsen/crypt/bin/crypt |
416 $ crypt set -plaintext /config/hugo.json /Users/hugo/settings/config.json |
429 $ crypt set -plaintext /config/hugo.json /Users/hugo/settings/config.json |
417 ``` |
430 ``` |
418 |
431 |
419 Confirm that your value was set: |
432 Confirm that your value was set: |
420 |
433 |
433 viper.SetConfigType("json") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop", "env", "dotenv" |
446 viper.SetConfigType("json") // because there is no file extension in a stream of bytes, supported extensions are "json", "toml", "yaml", "yml", "properties", "props", "prop", "env", "dotenv" |
434 err := viper.ReadRemoteConfig() |
447 err := viper.ReadRemoteConfig() |
435 ``` |
448 ``` |
436 |
449 |
437 #### Consul |
450 #### Consul |
438 You need to set a key to Consul key/value storage with JSON value containing your desired config. |
451 You need to set a key to Consul key/value storage with JSON value containing your desired config. |
439 For example, create a Consul key/value store key `MY_CONSUL_KEY` with value: |
452 For example, create a Consul key/value store key `MY_CONSUL_KEY` with value: |
440 |
453 |
441 ```json |
454 ```json |
442 { |
455 { |
443 "port": 8080, |
456 "port": 8080, |
451 err := viper.ReadRemoteConfig() |
464 err := viper.ReadRemoteConfig() |
452 |
465 |
453 fmt.Println(viper.Get("port")) // 8080 |
466 fmt.Println(viper.Get("port")) // 8080 |
454 fmt.Println(viper.Get("hostname")) // myhostname.com |
467 fmt.Println(viper.Get("hostname")) // myhostname.com |
455 ``` |
468 ``` |
|
469 |
|
470 #### Firestore |
|
471 |
|
472 ```go |
|
473 viper.AddRemoteProvider("firestore", "google-cloud-project-id", "collection/document") |
|
474 viper.SetConfigType("json") // Config's format: "json", "toml", "yaml", "yml" |
|
475 err := viper.ReadRemoteConfig() |
|
476 ``` |
|
477 |
|
478 Of course, you're allowed to use `SecureRemoteProvider` also |
456 |
479 |
457 ### Remote Key/Value Store Example - Encrypted |
480 ### Remote Key/Value Store Example - Encrypted |
458 |
481 |
459 ```go |
482 ```go |
460 viper.AddSecureRemoteProvider("etcd","http://127.0.0.1:4001","/config/hugo.json","/etc/secrets/mykeyring.gpg") |
483 viper.AddSecureRemoteProvider("etcd","http://127.0.0.1:4001","/config/hugo.json","/etc/secrets/mykeyring.gpg") |
566 |
589 |
567 However, if `datastore.metric` was overridden (by a flag, an environment variable, |
590 However, if `datastore.metric` was overridden (by a flag, an environment variable, |
568 the `Set()` method, …) with an immediate value, then all sub-keys of |
591 the `Set()` method, …) with an immediate value, then all sub-keys of |
569 `datastore.metric` become undefined, they are “shadowed” by the higher-priority |
592 `datastore.metric` become undefined, they are “shadowed” by the higher-priority |
570 configuration level. |
593 configuration level. |
|
594 |
|
595 Viper can access array indices by using numbers in the path. For example: |
|
596 |
|
597 ```json |
|
598 { |
|
599 "host": { |
|
600 "address": "localhost", |
|
601 "ports": [ |
|
602 5799, |
|
603 6029 |
|
604 ] |
|
605 }, |
|
606 "datastore": { |
|
607 "metric": { |
|
608 "host": "127.0.0.1", |
|
609 "port": 3099 |
|
610 }, |
|
611 "warehouse": { |
|
612 "host": "198.0.0.1", |
|
613 "port": 2112 |
|
614 } |
|
615 } |
|
616 } |
|
617 |
|
618 GetInt("host.ports.1") // returns 6029 |
|
619 |
|
620 ``` |
571 |
621 |
572 Lastly, if there exists a key that matches the delimited key path, its value |
622 Lastly, if there exists a key that matches the delimited key path, its value |
573 will be returned instead. E.g. |
623 will be returned instead. E.g. |
574 |
624 |
575 ```json |
625 ```json |
592 } |
642 } |
593 |
643 |
594 GetString("datastore.metric.host") // returns "0.0.0.0" |
644 GetString("datastore.metric.host") // returns "0.0.0.0" |
595 ``` |
645 ``` |
596 |
646 |
597 ### Extract sub-tree |
647 ### Extracting a sub-tree |
598 |
648 |
599 Extract sub-tree from Viper. |
649 When developing reusable modules, it's often useful to extract a subset of the configuration |
600 |
650 and pass it to a module. This way the module can be instantiated more than once, with different configurations. |
601 For example, `viper` represents: |
651 |
602 |
652 For example, an application might use multiple different cache stores for different purposes: |
603 ```json |
653 |
604 app: |
654 ```yaml |
|
655 cache: |
605 cache1: |
656 cache1: |
606 max-items: 100 |
657 max-items: 100 |
607 item-size: 64 |
658 item-size: 64 |
608 cache2: |
659 cache2: |
609 max-items: 200 |
660 max-items: 200 |
610 item-size: 80 |
661 item-size: 80 |
611 ``` |
662 ``` |
612 |
663 |
613 After executing: |
664 We could pass the cache name to a module (eg. `NewCache("cache1")`), |
614 |
665 but it would require weird concatenation for accessing config keys and would be less separated from the global config. |
615 ```go |
666 |
616 subv := viper.Sub("app.cache1") |
667 So instead of doing that let's pass a Viper instance to the constructor that represents a subset of the configuration: |
617 ``` |
668 |
618 |
669 ```go |
619 `subv` represents: |
670 cache1Config := viper.Sub("cache.cache1") |
620 |
671 if cache1Config == nil { // Sub returns nil if the key cannot be found |
621 ```json |
672 panic("cache configuration not found") |
622 max-items: 100 |
673 } |
623 item-size: 64 |
674 |
624 ``` |
675 cache1 := NewCache(cache1Config) |
625 |
676 ``` |
626 Suppose we have: |
677 |
627 |
678 **Note:** Always check the return value of `Sub`. It returns `nil` if a key cannot be found. |
628 ```go |
679 |
629 func NewCache(cfg *Viper) *Cache {...} |
680 Internally, the `NewCache` function can address `max-items` and `item-size` keys directly: |
630 ``` |
681 |
631 |
682 ```go |
632 which creates a cache based on config information formatted as `subv`. |
683 func NewCache(v *Viper) *Cache { |
633 Now it’s easy to create these 2 caches separately as: |
684 return &Cache{ |
634 |
685 MaxItems: v.GetInt("max-items"), |
635 ```go |
686 ItemSize: v.GetInt("item-size"), |
636 cfg1 := viper.Sub("app.cache1") |
687 } |
637 cache1 := NewCache(cfg1) |
688 } |
638 |
689 ``` |
639 cfg2 := viper.Sub("app.cache2") |
690 |
640 cache2 := NewCache(cfg2) |
691 The resulting code is easy to test, since it's decoupled from the main config structure, |
641 ``` |
692 and easier to reuse (for the same reason). |
|
693 |
642 |
694 |
643 ### Unmarshaling |
695 ### Unmarshaling |
644 |
696 |
645 You also have the option of Unmarshaling all or a specific value to a struct, map, |
697 You also have the option of Unmarshaling all or a specific value to a struct, map, |
646 etc. |
698 etc. |
726 |
778 |
727 Viper uses [github.com/mitchellh/mapstructure](https://github.com/mitchellh/mapstructure) under the hood for unmarshaling values which uses `mapstructure` tags by default. |
779 Viper uses [github.com/mitchellh/mapstructure](https://github.com/mitchellh/mapstructure) under the hood for unmarshaling values which uses `mapstructure` tags by default. |
728 |
780 |
729 ### Marshalling to string |
781 ### Marshalling to string |
730 |
782 |
731 You may need to marshal all the settings held in viper into a string rather than write them to a file. |
783 You may need to marshal all the settings held in viper into a string rather than write them to a file. |
732 You can use your favorite format's marshaller with the config returned by `AllSettings()`. |
784 You can use your favorite format's marshaller with the config returned by `AllSettings()`. |
733 |
785 |
734 ```go |
786 ```go |
735 import ( |
787 import ( |
736 yaml "gopkg.in/yaml.v2" |
788 yaml "gopkg.in/yaml.v2" |
737 // ... |
789 // ... |
738 ) |
790 ) |
739 |
791 |
740 func yamlStringSettings() string { |
792 func yamlStringSettings() string { |
741 c := viper.AllSettings() |
793 c := viper.AllSettings() |
742 bs, err := yaml.Marshal(c) |
794 bs, err := yaml.Marshal(c) |
743 if err != nil { |
795 if err != nil { |
777 ``` |
829 ``` |
778 |
830 |
779 When working with multiple vipers, it is up to the user to keep track of the |
831 When working with multiple vipers, it is up to the user to keep track of the |
780 different vipers. |
832 different vipers. |
781 |
833 |
|
834 |
782 ## Q & A |
835 ## Q & A |
783 |
836 |
784 Q: Why is it called “Viper”? |
837 ### Why is it called “Viper”? |
785 |
838 |
786 A: Viper is designed to be a [companion](http://en.wikipedia.org/wiki/Viper_(G.I._Joe)) |
839 A: Viper is designed to be a [companion](http://en.wikipedia.org/wiki/Viper_(G.I._Joe)) |
787 to [Cobra](https://github.com/spf13/cobra). While both can operate completely |
840 to [Cobra](https://github.com/spf13/cobra). While both can operate completely |
788 independently, together they make a powerful pair to handle much of your |
841 independently, together they make a powerful pair to handle much of your |
789 application foundation needs. |
842 application foundation needs. |
790 |
843 |
791 Q: Why is it called “Cobra”? |
844 ### Why is it called “Cobra”? |
792 |
845 |
793 A: Is there a better name for a [commander](http://en.wikipedia.org/wiki/Cobra_Commander)? |
846 Is there a better name for a [commander](http://en.wikipedia.org/wiki/Cobra_Commander)? |
|
847 |
|
848 ### Does Viper support case sensitive keys? |
|
849 |
|
850 **tl;dr:** No. |
|
851 |
|
852 Viper merges configuration from various sources, many of which are either case insensitive or uses different casing than the rest of the sources (eg. env vars). |
|
853 In order to provide the best experience when using multiple sources, the decision has been made to make all keys case insensitive. |
|
854 |
|
855 There has been several attempts to implement case sensitivity, but unfortunately it's not that trivial. We might take a stab at implementing it in [Viper v2](https://github.com/spf13/viper/issues/772), but despite the initial noise, it does not seem to be requested that much. |
|
856 |
|
857 You can vote for case sensitivity by filling out this feedback form: https://forms.gle/R6faU74qPRPAzchZ9 |
|
858 |
|
859 ### Is it safe to concurrently read and write to a viper? |
|
860 |
|
861 No, you will need to synchronize access to the viper yourself (for example by using the `sync` package). Concurrent reads and writes can cause a panic. |
|
862 |
|
863 ## Troubleshooting |
|
864 |
|
865 See [TROUBLESHOOTING.md](TROUBLESHOOTING.md). |