*: bump to v2.3.0-alpha.0

contrib: add example systemd unit file

.travis.yml

@@ -4,8 +4,10 @@ go:
   - 1.4
   - 1.5
 
-install:
- - go get github.com/barakmich/go-nyet
+addons:
+  apt:
+    packages:
+    - libpcap-dev
 
 script:
- - INTEGRATION=y ./test
+ - ./test

CONTRIBUTING.md

@@ -35,7 +35,7 @@ Thanks for your contributions!
 
 ### Code style
 
-The coding style suggested by the Golang community is used in etcd. See the [style doc](https://code.google.com/p/go-wiki/wiki/CodeReviewComments) for details.
+The coding style suggested by the Golang community is used in etcd. See the [style doc](https://github.com/golang/go/wiki/CodeReviewComments) for details.
 
 Please follow this style to make etcd easy to review, maintain and develop.
 

Documentation/04_to_2_snapshot_migration.md

@@ -1,4 +1,4 @@
-## Snapshot Migration
+# Snapshot Migration
 
 You can migrate a snapshot of your data from a v0.4.9+ cluster into a new etcd 2.2 cluster using a snapshot migration. After snapshot migration, the etcd indexes of your data will change. Many etcd applications rely on these indexes to behave correctly. This operation should only be done while all etcd applications are stopped.
 
@@ -15,7 +15,7 @@ etcdctl --endpoint new_cluster.example.com import --snap backup.snap
 ```
 
 If you have a large amount of data, you can specify more concurrent works to copy data in parallel by using `-c` flag.
-If you have hidden keys to copy, you can use `--hidden` flag to specify.
+If you have hidden keys to copy, you can use `--hidden` flag to specify. For example fleet uses `/_coreos.com/fleet` so to import those keys use `--hidden /_coreos.com`.
 
 And the data will quickly copy into the new cluster:
 

Documentation/admin_guide.md

@@ -1,8 +1,8 @@
-## Administration
+# Administration
 
-### Data Directory
+## Data Directory
 
-#### Lifecycle
+### Lifecycle
 
 When first started, etcd stores its configuration into a data directory specified by the data-dir configuration parameter.
 Configuration is stored in the write ahead log and includes: the local member ID, cluster ID, and initial cluster configuration.
@@ -20,7 +20,7 @@ Once removed the member can be re-added with an empty data directory.
 
 [remove-a-member]: runtime-configuration.md#remove-a-member
 
-#### Contents
+### Contents
 
 The data directory has two sub-directories in it:
 
@@ -32,18 +32,18 @@ If `--wal-dir` flag is set, etcd will write the write ahead log files to the spe
 [wal-pkg]: http://godoc.org/github.com/coreos/etcd/wal
 [snap-pkg]: http://godoc.org/github.com/coreos/etcd/snap
 
-### Cluster Management
+## Cluster Management
 
-#### Lifecycle
+### Lifecycle
 
 If you are spinning up multiple clusters for testing it is recommended that you specify a unique initial-cluster-token for the different clusters.
 This can protect you from cluster corruption in case of mis-configuration because two members started with different cluster tokens will refuse members from each other.
 
-#### Monitoring
+### Monitoring
 
 It is important to monitor your production etcd cluster for healthy information and runtime metrics.
 
-##### Health Monitoring
+#### Health Monitoring
 
 At lowest level, etcd exposes health information via HTTP at `/health` in JSON format. If it returns `{"health": "true"}`, then the cluster is healthy. Please note the `/health` endpoint is still an experimental one as in etcd 2.2.
 
@@ -63,16 +63,16 @@ member fd422379fda50e48 is healthy: got healthy result from http://127.0.0.1:323
 cluster is healthy
 ```
 
-##### Runtime Metrics
+#### Runtime Metrics
 
 etcd uses [Prometheus](http://prometheus.io/) for metrics reporting in the server. You can read more through the runtime metrics [doc](metrics.md).
 
-#### Debugging
+### Debugging
 
 Debugging a distributed system can be difficult. etcd provides several ways to make debug
 easier.
 
-##### Enabling Debug Logging
+#### Enabling Debug Logging
 
 When you want to debug etcd without stopping it, you can enable debug logging at runtime.
 etcd exposes logging configuration at `/config/local/log`.
@@ -85,7 +85,7 @@ $ curl http://127.0.0.1:2379/config/local/log -XPUT -d '{"Level":"INFO"}'
 $ # debug logging disabled
 ```
 
-##### Debugging Variables
+#### Debugging Variables
 
 Debug variables are exposed for real-time debugging purposes. Developers who are familiar with etcd can utilize these variables to debug unexpected behavior. etcd exposes debug variables via HTTP at `/debug/vars` in JSON format. The debug variables contains
 `cmdline`, `file_descriptor_limit`, `memstats` and `raft.status`.
@@ -107,7 +107,7 @@ Debug variables are exposed for real-time debugging purposes. Developers who are
 }
 ```
 
-#### Optimal Cluster Size
+### Optimal Cluster Size
 
 The recommended etcd cluster size is 3, 5 or 7, which is decided by the fault tolerance requirement. A 7-member cluster can provide enough fault tolerance in most cases. While larger cluster provides better fault tolerance the write performance reduces since data needs to be replicated to more machines.
 
@@ -152,7 +152,7 @@ This example will walk you through the process of migrating the infra1 member to
 |infra2|10.0.1.12:2380|
 
 ```sh
-$ export ETCDCTL_PEERS=http://10.0.1.10:2379,http://10.0.1.11:2379,http://10.0.1.12:2379
+$ export ETCDCTL_ENDPOINT=http://10.0.1.10:2379,http://10.0.1.11:2379,http://10.0.1.12:2379
 ```
 
 ```sh
@@ -293,6 +293,6 @@ If timeout happens several times continuously, administrators should check statu
 
 #### Maximum OS threads
 
-By default, etcd uses the default configuration of the Go 1.4 runtime, which means that at most one operating system thread will be used to execute code simultaneously. (Note that this default behavior [may change in Go 1.5](https://docs.google.com/document/d/1At2Ls5_fhJQ59kDK2DFVhFu3g5mATSXqqV5QrxinasI/edit)).
+By default, etcd uses the default configuration of the Go 1.4 runtime, which means that at most one operating system thread will be used to execute code simultaneously. (Note that this default behavior [has changed in Go 1.5](https://golang.org/doc/go1.5#runtime)).
 
 When using etcd in heavy-load scenarios on machines with multiple cores it will usually be desirable to increase the number of threads that etcd can utilize. To do this, simply set the environment variable `GOMAXPROCS` to the desired number when starting etcd. For more information on this variable, see the Go [runtime](https://golang.org/pkg/runtime) documentation.

Documentation/api.md

@@ -359,7 +359,7 @@ curl 'http://127.0.0.1:2379/v2/keys/foo?wait=true&waitIndex=2008'
 #### Connection being closed prematurely
 
 The server may close a long polling connection before emitting any events.
-This can happend due to a timeout or the server being shutdown.
+This can happen due to a timeout or the server being shutdown.
 Since the HTTP header is sent immediately upon accepting the connection, the response will be seen as empty: `200 OK` and empty body.
 The clients should be prepared to deal with this scenario and retry the watch.
 
@@ -506,7 +506,7 @@ The current comparable conditions are:
 
 2. `prevIndex` - checks the previous modifiedIndex of the key.
 
-3. `prevExist` - checks existence of the key: if `prevExist` is true, it is an `update` request; if prevExist is `false`, it is a `create` request.
+3. `prevExist` - checks existence of the key: if `prevExist` is true, it is an `update` request; if `prevExist` is `false`, it is a `create` request.
 
 Here is a simple example.
 Let's create a key-value pair first: `foo=one`.

Documentation/auth_api.md

@@ -19,7 +19,7 @@ Each role has exact one associated Permission List. An permission list exists fo
 
 The special static ROOT (named `root`) role has a full permissions on all key-value resources, the permission to manage user resources and settings resources. Only the ROOT role has the permission to manage user resources and modify settings resources. The ROOT role is built-in and does not need to be created.
 
-There is also a special GUEST role, named 'guest'. These are the permissions given to unauthenticated requests to etcd. This role will be created automatically, and by default allows access to the full keyspace due to backward compatability. (etcd did not previously authenticate any actions.). This role can be modified by a ROOT role holder at any time, to reduce the capabilities of unauthenticated users.
+There is also a special GUEST role, named 'guest'. These are the permissions given to unauthenticated requests to etcd. This role will be created automatically, and by default allows access to the full keyspace due to backward compatibility. (etcd did not previously authenticate any actions.). This role can be modified by a ROOT role holder at any time, to reduce the capabilities of unauthenticated users.
 
 #### Permissions
 
@@ -124,7 +124,7 @@ The User JSON object is formed as follows:
 
 Password is only passed when necessary.
 
-**Get a list of users**
+**Get a List of Users**
 
 GET/HEAD  /v2/auth/users
 
@@ -137,7 +137,36 @@ GET/HEAD  /v2/auth/users
         Content-type: application/json
     200 Body:
         {
-          "users": ["alice", "bob", "eve"]
+          "users": [
+            {
+              "user": "alice",
+              "roles": [
+                {
+                  "role": "root",
+                  "permissions": {
+                    "kv": {
+                      "read": ["*"],
+                      "write": ["*"]
+                    }
+                  }
+                }
+              ]
+            },
+            {
+              "user": "bob",
+              "roles": [
+                {
+                  "role": "guest",
+                  "permissions": {
+                    "kv": {
+                      "read": ["*"],
+                      "write": ["*"]
+                    }
+                  }
+                }
+              ]
+            }
+          ]
         }
 
 **Get User Details**
@@ -155,7 +184,26 @@ GET/HEAD  /v2/auth/users/alice
     200 Body:
         {
           "user" : "alice",
-          "roles" : ["fleet", "etcd"]
+          "roles" : [
+            {
+              "role": "fleet",
+              "permissions" : {
+                "kv" : {
+                  "read": [ "/fleet/" ],
+                  "write": [ "/fleet/" ]
+                }
+              }
+            },
+            {
+              "role": "etcd",
+              "permissions" : {
+                "kv" : {
+                  "read": [ "*" ],
+                  "write": [ "*" ]
+                }
+              }
+            }
+          ]
         }
 
 **Create Or Update A User**
@@ -213,22 +261,6 @@ A full role structure may look like this. A Permission List structure is used fo
 }
 ```
 
-**Get a list of Roles**
-
-GET/HEAD  /v2/auth/roles
-
-    Sent Headers:
-        Authorization: Basic <BasicAuthString>
-    Possible Status Codes:
-        200 OK
-        401 Unauthorized
-    200 Headers:
-        Content-type: application/json
-    200 Body:
-        {
-          "roles": ["fleet", "etcd", "quay"]
-        }
-
 **Get Role Details**
 
 GET/HEAD  /v2/auth/roles/fleet
@@ -252,6 +284,50 @@ GET/HEAD  /v2/auth/roles/fleet
           }
         }
 
+**Get a list of Roles**
+
+GET/HEAD  /v2/auth/roles
+
+    Sent Headers:
+        Authorization: Basic <BasicAuthString>
+    Possible Status Codes:
+        200 OK
+        401 Unauthorized
+    200 Headers:
+        Content-type: application/json
+    200 Body:
+        {
+          "roles": [
+            {
+              "role": "fleet",
+              "permissions": {
+                "kv": {
+                  "read": ["/fleet/"],
+                  "write": ["/fleet/"]
+                }
+              }
+            },
+            {
+              "role": "etcd",
+              "permissions": {
+                "kv": {
+                  "read": ["*"],
+                  "write": ["*"]
+                }
+              }
+            },
+            {
+              "role": "quay",
+              "permissions": {
+                "kv": {
+                  "read": ["*"],
+                  "write": ["*"]
+                }
+              }
+            }
+          ]
+        }
+
 **Create Or Update A Role**
 
 PUT  /v2/auth/roles/rkt

Documentation/authentication.md

@@ -134,7 +134,7 @@ $ etcdctl role remove myrolename
 
 ## Enabling authentication
 
-The minimal steps to enabling auth follow. The administrator can set up users and roles before or after enabling authentication, as a matter of preference. 
+The minimal steps to enabling auth are as follows. The administrator can set up users and roles before or after enabling authentication, as a matter of preference. 
 
 Make sure the root user is created:
 

Documentation/benchmarks/etcd-storage-memory-benchmark.md

@@ -0,0 +1,98 @@
+# Storage Memory Usage Benchmark
+
+<!---todo: link storage to storage design doc-->
+Two components of etcd storage consume physical memory. The etcd process allocates an *in-memory index* to speed key lookup. The process's *page cache*, managed by the operating system, stores recently-accessed data from disk for quick re-use.
+
+The in-memory index holds all the keys in a [B-tree][btree] data structure, along with pointers to the on-disk data (the values). Each key in the B-tree may contain multiple pointers, pointing to different versions of its values. The theoretical memory consumption of the in-memory index can hence be approximated with the formula:
+
+`N * (c1 + avg_key_size) + N * (avg_versions_of_key) * (c2 + size_of_pointer)`
+
+where `c1` is the key metadata overhead and `c2` is the version metadata overhead.
+
+The graph shows the detailed structure of the in-memory index B-tree.
+
+```
+
+
+                                In mem index
+
+                               +------------+
+                               | key || ... |
+  +--------------+             |     ||     |
+  |              |             +------------+
+  |              |             | v1  || ... |
+  |   disk    <----------------|     ||     | Tree Node
+  |              |             +------------+
+  |              |             | v2  || ... |
+  |           <----------------+     ||     |
+  |              |             +------------+
+  +--------------+       +-----+    |   |   |
+                         |     |    |   |   |
+                         |     +------------+
+                         |
+                         |
+                         ^
+                      ------+
+                      | ... |
+                      |     |
+                      +-----+
+                      | ... | Tree Node
+                      |     |
+                      +-----+
+                      | ... |
+                      |     |
+                      ------+
+```
+
+[Page cache memory][pagecache] is managed by the operating system and is not covered in detail in this document.
+
+## Testing Environment
+
+etcd version
+- git head https://github.com/coreos/etcd/commit/776e9fb7be7eee5e6b58ab977c8887b4fe4d48db
+
+GCE n1-standard-2 machine type
+
+- 7.5 GB memory
+- 2x CPUs
+
+## In-memory index memory usage
+
+In this test, we only benchmark the memory usage of the in-memory index. The goal is to find `c1` and `c2` mentioned above and to understand the hard limit of memory consumption of the storage.
+
+We calculate the memory usage consumption via the Go runtime.ReadMemStats. We calculate the total allocated bytes difference before creating the index and after creating the index. It cannot perfectly reflect the memory usage of the in-memory index itself but can show the rough consumption pattern. 
+
+| N    | versions | key size | memory usage |
+|------|----------|----------|--------------|
+| 100K | 1        | 64bytes  | 22MB         |
+| 100K | 5        | 64bytes  | 39MB         |
+| 1M   | 1        | 64bytes  | 218MB        |
+| 1M   | 5        | 64bytes  | 432MB        |
+| 100K | 1        | 256bytes | 41MB         |
+| 100K | 5        | 256bytes | 65MB         |
+| 1M   | 1        | 256bytes | 409MB        |
+| 1M   | 5        | 256bytes | 506MB        |
+
+
+Based on the result, we can calculate `c1=120bytes`, `c2=30bytes`. We only need two sets of data to calculate `c1` and `c2`, since they are the only unknown variable in the formula. The `c1=120bytes` and `c2=30bytes` are the average value of the 4 sets of `c1` and `c2` we calculated. The key metadata overhead is still relatively nontrivial (50%) for small key-value pairs. However, this is a significant improvement over the old store, which had at least 1000% overhead.
+
+## Overall memory usage
+
+The overall memory usage captures how much RSS etcd consumes with the storage. The value size should have very little impact on the overall memory usage of etcd, since we keep values on disk and only retain hot values in memory, managed by the OS page cache.
+
+| N    | versions | key size | value size | memory usage |
+|------|----------|----------|------------|--------------|
+| 100K | 1        | 64bytes  | 256bytes   | 40MB         |
+| 100K | 5        | 64bytes  | 256bytes   | 89MB         |
+| 1M   | 1        | 64bytes  | 256bytes   | 470MB        |
+| 1M   | 5        | 64bytes  | 256bytes   | 880MB        |
+| 100K | 1        | 64bytes  | 1KB        | 102MB        |
+| 100K | 5        | 64bytes  | 1KB        | 164MB        |
+| 1M   | 1        | 64bytes  | 1KB        | 587MB        |
+| 1M   | 5        | 64bytes  | 1KB        | 836MB        |
+
+Based on the result, we know the value size does not significantly impact the memory consumption. There is some minor increase due to more data held in the OS page cache.
+
+[btree]: https://en.wikipedia.org/wiki/B-tree
+[pagecache]: https://en.wikipedia.org/wiki/Page_cache
+

Documentation/branch_management.md

@@ -1,6 +1,6 @@
-## Branch Management
+# Branch Management
 
-### Guide
+## Guide
 
 - New development occurs on the [master branch](https://github.com/coreos/etcd/tree/master)
 - Master branch should always have a green build!

Documentation/clustering.md

@@ -124,7 +124,7 @@ There two methods that can be used for discovery:
 
 ### etcd Discovery
 
-To better understand the design about discovery service protocol, we suggest you read [this](./discovery_protocol.md).
+To better understand the design about discovery service protocol, we suggest you read [this](discovery_protocol.md).
 
 #### Lifetime of a Discovery URL
 
@@ -148,7 +148,7 @@ If you bootstrap an etcd cluster using discovery service with more than the expe
 
 The URL you will use in this case will be `https://myetcd.local/v2/keys/discovery/6c007a14875d53d9bf0ef5a6fc0257c817f0fb83` and the etcd members will use the `https://myetcd.local/v2/keys/discovery/6c007a14875d53d9bf0ef5a6fc0257c817f0fb83` directory for registration as they start.
 
-Each member must have a different name flag specified. Or discovery will fail due to duplicated name.
+**Each member must have a different name flag specified. `Hostname` or `machine-id` can be a good choice. Or discovery will fail due to duplicated name.** 
 
 Now we start etcd with those relevant flags for each member:
 
@@ -200,7 +200,7 @@ ETCD_DISCOVERY=https://discovery.etcd.io/3e86b59982e49066c5d813af1c2e2579cbf573d
 -discovery https://discovery.etcd.io/3e86b59982e49066c5d813af1c2e2579cbf573de
 ```
 
-Each member must have a different name flag specified. Or discovery will fail due to duplicated name.
+**Each member must have a different name flag specified. `Hostname` or `machine-id` can be a good choice. Or discovery will fail due to duplicated name.** 
 
 Now we start etcd with those relevant flags for each member:
 
@@ -384,7 +384,7 @@ $ etcd --proxy on -discovery-srv example.com
 
 #### Error Cases
 
-You might see the an error like `cannot find local etcd $name from SRV records.`. That means the etcd member fails to find itself from the cluster defined in SRV records. The resolved address in `-initial-advertise-peer-urls` *must match* one of the resolved addresses in the SRV targets.
+You might see an error like `cannot find local etcd $name from SRV records.`. That means the etcd member fails to find itself from the cluster defined in SRV records. The resolved address in `-initial-advertise-peer-urls` *must match* one of the resolved addresses in the SRV targets.
 
 # 0.4 to 2.0+ Migration Guide
 

Documentation/configuration.md

@@ -1,4 +1,4 @@
-## Configuration Flags
+# Configuration Flags
 
 etcd is configurable through command-line flags and environment variables. Options set on the command line take precedence over those from the environment.
 
@@ -8,251 +8,256 @@ To start etcd automatically using custom settings at startup in Linux, using a [
 
 [systemd-intro]: http://freedesktop.org/wiki/Software/systemd/
 
-### Member Flags
+## Member Flags
 
-##### -name
+### -name
 + Human-readable name for this member.
 + default: "default"
 + env variable: ETCD_NAME
-+ This value is referenced as this node's own entries listed in the `-initial-cluster` flag (Ex: `default=http://localhost:2380` or `default=http://localhost:2380,default=http://localhost:7001`). This needs to match the key used in the flag if you're using [static boostrapping](clustering.md#static).
++ This value is referenced as this node's own entries listed in the `-initial-cluster` flag (Ex: `default=http://localhost:2380` or `default=http://localhost:2380,default=http://localhost:7001`). This needs to match the key used in the flag if you're using [static bootstrapping](clustering.md#static). When using discovery, each member must have a unique name. `Hostname` or `machine-id` can be a good choice.
 
-##### -data-dir
+### -data-dir
 + Path to the data directory.
 + default: "${name}.etcd"
 + env variable: ETCD_DATA_DIR
 
-##### -wal-dir
+### -wal-dir
 + Path to the dedicated wal directory. If this flag is set, etcd will write the WAL files to the walDir rather than the dataDir. This allows a dedicated disk to be used, and helps avoid io competition between logging and other IO operations.
 + default: ""
 + env variable: ETCD_WAL_DIR
 
-##### -snapshot-count
+### -snapshot-count
 + Number of committed transactions to trigger a snapshot to disk.
 + default: "10000"
 + env variable: ETCD_SNAPSHOT_COUNT
 
-##### -heartbeat-interval
+### -heartbeat-interval
 + Time (in milliseconds) of a heartbeat interval.
 + default: "100"
 + env variable: ETCD_HEARTBEAT_INTERVAL
 
-##### -election-timeout
+### -election-timeout
 + Time (in milliseconds) for an election to timeout. See [Documentation/tuning.md](tuning.md#time-parameters) for details.
 + default: "1000"
 + env variable: ETCD_ELECTION_TIMEOUT
 
-##### -listen-peer-urls
+### -listen-peer-urls
 + List of URLs to listen on for peer traffic. This flag tells the etcd to accept incoming requests from its peers on the specified scheme://IP:port combinations. Scheme can be either http or https.If 0.0.0.0 is specified as the IP, etcd listens to the given port on all interfaces. If an IP address is given as well as a port, etcd will listen on the given port and interface. Multiple URLs may be used to specify a number of addresses and ports to listen on. The etcd will respond to requests from any of the listed addresses and ports.
 + default: "http://localhost:2380,http://localhost:7001"
 + env variable: ETCD_LISTEN_PEER_URLS
 + example: "http://10.0.0.1:2380"
 + invalid example: "http://example.com:2380" (domain name is invalid for binding)
 
-##### -listen-client-urls
+### -listen-client-urls
 + List of URLs to listen on for client traffic. This flag tells the etcd to accept incoming requests from the clients on the specified scheme://IP:port combinations. Scheme can be either http or https. If 0.0.0.0 is specified as the IP, etcd listens to the given port on all interfaces. If an IP address is given as well as a port, etcd will listen on the given port and interface. Multiple URLs may be used to specify a number of addresses and ports to listen on. The etcd will respond to requests from any of the listed addresses and ports.
 + default: "http://localhost:2379,http://localhost:4001"
 + env variable: ETCD_LISTEN_CLIENT_URLS
 + example: "http://10.0.0.1:2379"
 + invalid example: "http://example.com:2379" (domain name is invalid for binding)
 
-##### -max-snapshots
+### -max-snapshots
 + Maximum number of snapshot files to retain (0 is unlimited)
 + default: 5
 + env variable: ETCD_MAX_SNAPSHOTS
 + The default for users on Windows is unlimited, and manual purging down to 5 (or your preference for safety) is recommended.
 
-##### -max-wals
+### -max-wals
 + Maximum number of wal files to retain (0 is unlimited)
 + default: 5
 + env variable: ETCD_MAX_WALS
 + The default for users on Windows is unlimited, and manual purging down to 5 (or your preference for safety) is recommended.
 
-##### -cors
+### -cors
 + Comma-separated white list of origins for CORS (cross-origin resource sharing).
 + default: none
 + env variable: ETCD_CORS
 
-### Clustering Flags
+## Clustering Flags
 
 `-initial` prefix flags are used in bootstrapping ([static bootstrap][build-cluster], [discovery-service bootstrap][discovery] or [runtime reconfiguration][reconfig]) a new member, and ignored when restarting an existing member.
 
 `-discovery` prefix flags need to be set when using [discovery service][discovery].
 
-##### -initial-advertise-peer-urls
+### -initial-advertise-peer-urls
 
 + List of this member's peer URLs to advertise to the rest of the cluster. These addresses are used for communicating etcd data around the cluster. At least one must be routable to all cluster members. These URLs can contain domain names.
 + default: "http://localhost:2380,http://localhost:7001"
 + env variable: ETCD_INITIAL_ADVERTISE_PEER_URLS
 + example: "http://example.com:2380, http://10.0.0.1:2380"
 
-##### -initial-cluster
+### -initial-cluster
 + Initial cluster configuration for bootstrapping.
 + default: "default=http://localhost:2380,default=http://localhost:7001"
 + env variable: ETCD_INITIAL_CLUSTER
 + The key is the value of the `-name` flag for each node provided. The default uses `default` for the key because this is the default for the `-name` flag.
 
-##### -initial-cluster-state
+### -initial-cluster-state
 + Initial cluster state ("new" or "existing"). Set to `new` for all members present during initial static or DNS bootstrapping. If this option is set to `existing`, etcd will attempt to join the existing cluster. If the wrong value is set, etcd will attempt to start but fail safely.
 + default: "new"
 + env variable: ETCD_INITIAL_CLUSTER_STATE
 
 [static bootstrap]: clustering.md#static
 
-##### -initial-cluster-token
+### -initial-cluster-token
 + Initial cluster token for the etcd cluster during bootstrap.
 + default: "etcd-cluster"
 + env variable: ETCD_INITIAL_CLUSTER_TOKEN
 
-##### -advertise-client-urls
+### -advertise-client-urls
 + List of this member's client URLs to advertise to the rest of the cluster. These URLs can contain domain names.
 + default: "http://localhost:2379,http://localhost:4001"
 + env variable: ETCD_ADVERTISE_CLIENT_URLS
 + example: "http://example.com:2379, http://10.0.0.1:2379"
 + Be careful if you are advertising URLs such as http://localhost:2379 from a cluster member and are using the proxy feature of etcd. This will cause loops, because the proxy will be forwarding requests to itself until its resources (memory, file descriptors) are eventually depleted.
 
-##### -discovery
+### -discovery
 + Discovery URL used to bootstrap the cluster.
 + default: none
 + env variable: ETCD_DISCOVERY
 
-##### -discovery-srv
+### -discovery-srv
 + DNS srv domain used to bootstrap the cluster.
 + default: none
 + env variable: ETCD_DISCOVERY_SRV
 
-##### -discovery-fallback
+### -discovery-fallback
 + Expected behavior ("exit" or "proxy") when discovery services fails.
 + default: "proxy"
 + env variable: ETCD_DISCOVERY_FALLBACK
 
-##### -discovery-proxy
+### -discovery-proxy
 + HTTP proxy to use for traffic to discovery service.
 + default: none
 + env variable: ETCD_DISCOVERY_PROXY
 
-### Proxy Flags
+### -strict-reconfig-check
++ Reject reconfiguration requests that would cause quorum loss.
++ default: false
++ env variable: ETCD_STRICT_RECONFIG_CHECK
+
+## Proxy Flags
 
 `-proxy` prefix flags configures etcd to run in [proxy mode][proxy].
 
-##### -proxy
+### -proxy
 + Proxy mode setting ("off", "readonly" or "on").
 + default: "off"
 + env variable: ETCD_PROXY
 
-##### -proxy-failure-wait
+### -proxy-failure-wait
 + Time (in milliseconds) an endpoint will be held in a failed state before being reconsidered for proxied requests.
 + default: 5000
 + env variable: ETCD_PROXY_FAILURE_WAIT
 
-##### -proxy-refresh-interval
+### -proxy-refresh-interval
 + Time (in milliseconds) of the endpoints refresh interval.
 + default: 30000
 + env variable: ETCD_PROXY_REFRESH_INTERVAL
 
-##### -proxy-dial-timeout
+### -proxy-dial-timeout
 + Time (in milliseconds) for a dial to timeout or 0 to disable the timeout
 + default: 1000
 + env variable: ETCD_PROXY_DIAL_TIMEOUT
 
-##### -proxy-write-timeout
+### -proxy-write-timeout
 + Time (in milliseconds) for a write to timeout or 0 to disable the timeout.
 + default: 5000
 + env variable: ETCD_PROXY_WRITE_TIMEOUT
 
-##### -proxy-read-timeout
+### -proxy-read-timeout
 + Time (in milliseconds) for a read to timeout or 0 to disable the timeout.
 + Don't change this value if you use watches because they are using long polling requests.
 + default: 0
 + env variable: ETCD_PROXY_READ_TIMEOUT
 
-### Security Flags
+## Security Flags
 
 The security flags help to [build a secure etcd cluster][security].
 
-##### -ca-file [DEPRECATED]
+### -ca-file [DEPRECATED]
 + Path to the client server TLS CA file. `-ca-file ca.crt` could be replaced by `-trusted-ca-file ca.crt -client-cert-auth` and etcd will perform the same.
 + default: none
 + env variable: ETCD_CA_FILE
 
-##### -cert-file
+### -cert-file
 + Path to the client server TLS cert file.
 + default: none
 + env variable: ETCD_CERT_FILE
 
-##### -key-file
+### -key-file
 + Path to the client server TLS key file.
 + default: none
 + env variable: ETCD_KEY_FILE
 
-##### -client-cert-auth
+### -client-cert-auth
 + Enable client cert authentication.
 + default: false
 + env variable: ETCD_CLIENT_CERT_AUTH
 
-##### -trusted-ca-file
+### -trusted-ca-file
 + Path to the client server TLS trusted CA key file.
 + default: none
 + env variable: ETCD_TRUSTED_CA_FILE
 
-##### -peer-ca-file [DEPRECATED]
+### -peer-ca-file [DEPRECATED]
 + Path to the peer server TLS CA file. `-peer-ca-file ca.crt` could be replaced by `-peer-trusted-ca-file ca.crt -peer-client-cert-auth` and etcd will perform the same.
 + default: none
 + env variable: ETCD_PEER_CA_FILE
 
-##### -peer-cert-file
+### -peer-cert-file
 + Path to the peer server TLS cert file.
 + default: none
 + env variable: ETCD_PEER_CERT_FILE
 
-##### -peer-key-file
+### -peer-key-file
 + Path to the peer server TLS key file.
 + default: none
 + env variable: ETCD_PEER_KEY_FILE
 
-##### -peer-client-cert-auth
+### -peer-client-cert-auth
 + Enable peer client cert authentication.
 + default: false
 + env variable: ETCD_PEER_CLIENT_CERT_AUTH
 
-##### -peer-trusted-ca-file
+### -peer-trusted-ca-file
 + Path to the peer server TLS trusted CA file.
 + default: none
 + env variable: ETCD_PEER_TRUSTED_CA_FILE
 
-### Logging Flags
+## Logging Flags
 
-##### -debug
+### -debug
 + Drop the default log level to DEBUG for all subpackages.
 + default: false (INFO for all packages)
 + env variable: ETCD_DEBUG
 
-##### -log-package-levels
+### -log-package-levels
 + Set individual etcd subpackages to specific log levels. An example being `etcdserver=WARNING,security=DEBUG` 
 + default: none (INFO for all packages)
 + env variable: ETCD_LOG_PACKAGE_LEVELS
 
 
-### Unsafe Flags
+## Unsafe Flags
 
 Please be CAUTIOUS when using unsafe flags because it will break the guarantees given by the consensus protocol.
 For example, it may panic if other members in the cluster are still alive.
 Follow the instructions when using these flags.
 
-##### -force-new-cluster
-+ Force to create a new one-member cluster. It commits configuration changes in force to remove all existing members in the cluster and add itself. It needs to be set to [restore a backup][restore].
+### -force-new-cluster
++ Force to create a new one-member cluster. It commits configuration changes forcing to remove all existing members in the cluster and add itself. It needs to be set to [restore a backup][restore].
 + default: false
 + env variable: ETCD_FORCE_NEW_CLUSTER
 
-### Experimental Flags
+## Experimental Flags
 
-##### -experimental-v3demo
+### -experimental-v3demo
 + Enable experimental [v3 demo API](rfc/v3api.proto).
 + default: false
 + env variable: ETCD_EXPERIMENTAL_V3DEMO
 
-### Miscellaneous Flags
+## Miscellaneous Flags
 
-##### -version
+### -version
 + Print the version and exit.
 + default: false
 

Documentation/discovery_protocol.md

@@ -4,7 +4,7 @@ Discovery service protocol helps new etcd member to discover all other members i
 
 Discovery service protocol is _only_ used in cluster bootstrap phase, and cannot be used for runtime reconfiguration or cluster monitoring.
 
-The protocol uses a new discovery token to bootstrap one _unique_ etcd cluster. Remember that one discovery token can represent only one etcd cluster. As long as discovery protocol on this token starts, even if fails halfway, it must not be used to bootstrap another etcd cluster.
+The protocol uses a new discovery token to bootstrap one _unique_ etcd cluster. Remember that one discovery token can represent only one etcd cluster. As long as discovery protocol on this token starts, even if it fails halfway, it must not be used to bootstrap another etcd cluster.
 
 The rest of this article will walk through the discovery process with examples that correspond to a self-hosted discovery cluster. The public discovery service, discovery.etcd.io, functions the same way, but with a layer of polish to abstract away ugly URLs, generate UUIDs automatically, and provide some protections against excessive requests. At its core, the public discovery service still uses an etcd cluster as the data store as described in this document.
 

Documentation/errorcode.md

@@ -1,4 +1,4 @@
-Error Code
+# Error Code
 ======
 
 This document describes the error code used in key space '/v2/keys'. Feel free to import 'github.com/coreos/etcd/error' to use.

Documentation/faq.md

@@ -37,7 +37,7 @@ timeout.
 
 A proxy is a redirection server to the etcd cluster. The proxy handles the
 redirection of a client to the current configuration of the etcd cluster. A
-typical usecase is to start a proxy on a machine, and on first boot up of the
+typical use case is to start a proxy on a machine, and on first boot up of the
 proxy specify both the `--proxy` flag and the `--initial-cluster` flag. 
 
 From there, any etcdctl client that starts up automatically speaks to the local
@@ -57,24 +57,26 @@ and their integration with the reconfiguration API.
 Thus, a member that is down, even infinitely, will never be automatically
 removed from the etcd cluster member list. 
 
-This makes sense because its usually an application level / administrative
+This makes sense because it's usually an application level / administrative
 action to determine whether a reconfiguration should happen based on health. 
 
-For more information, refer to [Documentation/runtime-reconfiguration.md].
+For more information, refer to
+[Documentation/runtime-reconf-design.md](https://github.com/coreos/etcd/blob/master/Documentation/runtime-reconf-design.md).
 
-## 6) how does --peers work with etcdctl? 
+## 6) how does --endpoint work with etcdctl? 
 
-The `--peers` flag can specify any number of etcd cluster members in a comma
+The `--endpoint` flag can specify any number of etcd cluster members in a comma
 separated list. This list might be a subset, equal to, or more than the actual
 etcd cluster member list itself. 
 
-If only one peer is specified via the `--peers` flag, the etcdctl discovers the
+If only one peer is specified via the `--endpoint` flag, the etcdctl discovers the
 rest of the cluster via the member list of that one peer, and then it randomly
 chooses a member to use.  Again, the client can use the `quorum=true` flag on
 reads, which will always fail when using a member in the minority. 
 
-If peers from multiple clusters are specified via the `--peers` flag, etcdctl
+If peers from multiple clusters are specified via the `--endpoint` flag, etcdctl
 will randomly choose a peer, and the request will simply get routed to one of
 the clusters. This is probably not what you want. 
 
-
+Note: --peers flag is now deprecated and --endpoint should be used instead, 
+as it might confuse users to give etcdctl a peerURL.

Documentation/glossary.md

@@ -1,35 +1,35 @@
-## Glossary
+# Glossary
 
 This document defines the various terms used in etcd documentation, command line and source code.
 
-### Node
+## Node
 
 Node is an instance of raft state machine.
 
 It has a unique identification, and records other nodes' progress internally when it is the leader.
 
-### Member
+## Member
 
 Member is an instance of etcd. It hosts a node, and provides service to clients.
 
-### Cluster
+## Cluster
 
 Cluster consists of several members.
 
 The node in each member follows raft consensus protocol to replicate logs. Cluster receives proposals from members, commits them and apply to local store.
 
-### Peer
+## Peer
 
 Peer is another member of the same cluster.
 
-### Proposal
+## Proposal
 
 A proposal is a request (for example a write request, a configuration change request) that needs to go through raft protocol.
 
-### Client
+## Client
 
 Client is a caller of the cluster's HTTP API.
 
-### Machine (deprecated)
+## Machine (deprecated)
 
 The alternative of Member in etcd before 2.0

Documentation/implementation-faq.md

@@ -46,7 +46,7 @@ ExecStart=/usr/bin/etcd
 
 There are several error cases:
 
-0) Init has already ran and the data directory is already configured
+0) Init has already run and the data directory is already configured
 1) Discovery fails because of network timeout, etc
 2) Discovery fails because the cluster is already full and etcd needs to fall back to proxy
 3) Static cluster configuration fails because of conflict, misconfiguration or timeout

Documentation/libraries-and-tools.md

@@ -1,19 +1,24 @@
-## Libraries and Tools
+# Libraries and Tools
 
 **Tools**
 
-- [etcdctl](https://github.com/coreos/etcdctl) - A command line client for etcd
+- [etcdctl](https://github.com/coreos/etcd/tree/master/etcdctl) - A command line client for etcd
 - [etcd-backup](https://github.com/fanhattan/etcd-backup) - A powerful command line utility for dumping/restoring etcd - Supports v2
 - [etcd-dump](https://npmjs.org/package/etcd-dump) - Command line utility for dumping/restoring etcd.
 - [etcd-fs](https://github.com/xetorthio/etcd-fs) - FUSE filesystem for etcd
+- [etcddir](https://github.com/rekby/etcddir) - Realtime sync etcd and local directory. Work with windows and linux.
 - [etcd-browser](https://github.com/henszey/etcd-browser) - A web-based key/value editor for etcd using AngularJS
 - [etcd-lock](https://github.com/datawisesystems/etcd-lock) - Master election & distributed r/w lock implementation using etcd - Supports v2
 - [etcd-console](https://github.com/matishsiao/etcd-console) - A web-base key/value editor for etcd using PHP
 - [etcd-viewer](https://github.com/nikfoundas/etcd-viewer) - An etcd key-value store editor/viewer written in Java
+- [etcd-export](https://github.com/mickep76/etcd-export) - Export/Import etcd directory as JSON/YAML/TOML and Validate directory using JSON schema
+- [etcd-rest](https://github.com/mickep76/etcd-rest) - Create generic REST API in Go using etcd as a backend with validation using JSON schema
+- [etcdsh](https://github.com/kamilhark/etcdsh) - A command line client with support of command history and tab completion. Supports v2
 
 **Go libraries**
 
-- [go-etcd](https://github.com/coreos/go-etcd) - Supports v2
+- [etcd/client](https://github.com/coreos/etcd/blob/master/client) - the officially maintained Go client
+- [go-etcd](https://github.com/coreos/go-etcd) - the deprecated official client. May be useful for older (<2.0.0) versions of etcd.
 
 **Java libraries**
 
@@ -49,6 +54,7 @@
 
 **C++ libraries**
 - [edwardcapriolo/etcdcpp](https://github.com/edwardcapriolo/etcdcpp) - Supports v2
+- [suryanathan/etcdcpp](https://github.com/suryanathan/etcdcpp) - Supports v2 (with waits)
 
 **Clojure libraries**
 
@@ -111,7 +117,7 @@ A detailed recap of client functionalities can be found in the [clients compatib
 - [configdb](https://git.autistici.org/ai/configdb/tree/master) - A REST relational abstraction on top of arbitrary database backends, aimed at storing configs and inventories.
 - [scrz](https://github.com/scrz/scrz) - Container manager, stores configuration in etcd.
 - [fleet](https://github.com/coreos/fleet) - Distributed init system
-- [GoogleCloudPlatform/kubernetes](https://github.com/GoogleCloudPlatform/kubernetes) - Container cluster manager.
+- [kubernetes/kubernetes](https://github.com/kubernetes/kubernetes) - Container cluster manager introduced by Google.
 - [mailgun/vulcand](https://github.com/mailgun/vulcand) - HTTP proxy that uses etcd as a configuration backend.
 - [duedil-ltd/discodns](https://github.com/duedil-ltd/discodns) - Simple DNS nameserver using etcd as a database for names and records.
 - [skynetservices/skydns](https://github.com/skynetservices/skydns) - RFC compliant DNS server

Documentation/metrics.md

@@ -1,19 +1,19 @@
-## Metrics
+# Metrics
 
-**NOTE: The metrics feature is considered as an experimental. We might add/change/remove metrics without warning in the future releases.**
+**NOTE: The metrics feature is considered experimental. We may add/change/remove metrics without warning in future releases.**
 
 etcd uses [Prometheus](http://prometheus.io/) for metrics reporting in the server. The metrics can be used for real-time monitoring and debugging.
 
 The simplest way to see the available metrics is to cURL the metrics endpoint `/metrics` of etcd. The format is described [here](http://prometheus.io/docs/instrumenting/exposition_formats/).
 
 
-You can also follow the doc [here](http://prometheus.io/docs/introduction/getting_started/) to start a Promethus server and monitor etcd metrics.
+You can also follow the doc [here](http://prometheus.io/docs/introduction/getting_started/) to start a Prometheus server and monitor etcd metrics.
 
-The naming of metrics follows the suggested [best practice of Promethus](http://prometheus.io/docs/practices/naming/). A metric name has an `etcd` prefix as its namespace and a subsystem prefix (for example `wal` and `etcdserver`).
+The naming of metrics follows the suggested [best practice of Prometheus](http://prometheus.io/docs/practices/naming/). A metric name has an `etcd` prefix as its namespace and a subsystem prefix (for example `wal` and `etcdserver`).
 
 etcd now exposes the following metrics:
 
-### etcdserver
+## etcdserver
 
 | Name                                    | Description                                      | Type    |
 |-----------------------------------------|--------------------------------------------------|---------|
@@ -30,46 +30,7 @@ Pending proposal (`pending_proposal_total`) gives you an idea about how many pro
 
 Failed proposals (`proposal_failed_total`) are normally related to two issues: temporary failures related to a leader election or longer duration downtime caused by a loss of quorum in the cluster.
 
-
-### store
-
-These metrics describe the accesses into the data store of etcd members that exist in the cluster. They 
-are useful to count what kind of actions are taken by users. It is also useful to see and whether all etcd members 
-"see" the same set of data mutations, and whether reads and watches (which are local) are equally distributed.
-
-All these metrics are prefixed with `etcd_store_`. 
-
-| Name                      | Description                                                                          | Type                   |
-|---------------------------|------------------------------------------------------------------------------------------|--------------------|
-| reads_total               | Total number of reads from store, should differ among etcd members (local reads).    | Counter(action)        |
-| writes_total              | Total number of writes to store, should be same among all etcd members.              | Counter(action)        |
-| reads_failed_total        | Number of failed reads from store (e.g. key missing) on local reads.                 | Counter(action)        |
-| writes_failed_total     | Number of failed writes to store (e.g. failed compare and swap).                       | Counter(action)        |
-| expires_total             | Total number of expired keys (due to TTL).                                           | Counter                |
-| watch_requests_totals     | Total number of incoming watch requests to this etcd member (local watches).         | Counter                | 
-| watchers                  | Current count of active watchers on this etcd member.                                | Gauge                  |
-
-Both `reads_total` and `writes_total` count both successful and failed requests. `reads_failed_total` and 
-`writes_failed_total` count failed requests. A lot of failed writes indicate possible contentions on keys (e.g. when 
-doing  `compareAndSet`), and read failures indicate that some clients try to access keys that don't exist.
-
-Example Prometheus queries that may be useful from these metrics (across all etcd members):
-
- *  `sum(rate(etcd_store_reads_total{job="etcd"}[1m])) by (action)`
-    `max(rate(etcd_store_writes_total{job="etcd"}[1m])) by (action)`
-    
-    Rate of reads and writes by action, across all servers across a time window of `1m`. The reason why `max` is used
-     for writes as opposed to `sum` for reads is because all of etcd nodes in the cluster apply all writes to their stores.
-    Shows the rate of successfull readonly/write queries across all servers, across a time window of `1m`.
- * `sum(rate(etcd_store_watch_requests_total{job="etcd"}[1m]))`
-    
-    Shows rate of new watch requests per second. Likely driven by how often watched keys change. 
- * `sum(etcd_store_watchers{job="etcd"})`
-    
-    Number of active watchers across all etcd servers.        
-
-
-### wal
+## wal
 
 | Name                               | Description                                      | Type    |
 |------------------------------------|--------------------------------------------------|---------|
@@ -78,7 +39,39 @@ Example Prometheus queries that may be useful from these metrics (across all etc
 
 Abnormally high fsync duration (`fsync_durations_microseconds`) indicates disk issues and might cause the cluster to be unstable.
 
-### snapshot
+
+## http requests
+
+These metrics describe the serving of requests (non-watch events) served by etcd members in non-proxy mode: total 
+incoming requests, request failures and processing latency (inc. raft rounds for storage). They are useful for tracking
+ user-generated traffic hitting the etcd cluster . 
+
+All these metrics are prefixed with `etcd_http_`
+
+| Name                           | Description                                                                         | Type                   |
+|--------------------------------|-----------------------------------------------------------------------------------------|--------------------|
+| received_total                 | Total number of events after parsing and auth.                                      | Counter(method)        |
+| failed_total                   | Total number of failed events.                                                      | Counter(method,error)  |
+| successful_duration_second     |  Bucketed handling times of the requests, including raft rounds for writes.          | Histogram(method)      |
+
+
+Example Prometheus queries that may be useful from these metrics (across all etcd members):
+ 
+ * `sum(rate(etcd_http_failed_total{job="etcd"}[1m]) by (method) / sum(rate(etcd_http_events_received_total{job="etcd"})[1m]) by (method)` 
+    
+    Shows the fraction of events that failed by HTTP method across all members, across a time window of `1m`.
+ 
+ * `sum(rate(etcd_http_received_total{job="etcd",method="GET})[1m]) by (method)`
+   `sum(rate(etcd_http_received_total{job="etcd",method~="GET})[1m]) by (method)`
+    
+    Shows the rate of successful readonly/write queries across all servers, across a time window of `1m`.
+    
+ * `histogram_quantile(0.9, sum(increase(etcd_http_successful_processing_seconds{job="etcd",method="GET"}[5m]) ) by (le))`
+   `histogram_quantile(0.9, sum(increase(etcd_http_successful_processing_seconds{job="etcd",method!="GET"}[5m]) ) by (le))`
+    
+    Show the 0.90-tile latency (in seconds) of read/write (respectively) event handling across all members, with a window of `5m`.      
+
+## snapshot
 
 | Name                                       | Description                                                | Type    |
 |--------------------------------------------|------------------------------------------------------------|---------|
@@ -87,7 +80,7 @@ Abnormally high fsync duration (`fsync_durations_microseconds`) indicates disk i
 Abnormally high snapshot duration (`snapshot_save_total_durations_microseconds`) indicates disk issues and might cause the cluster to be unstable.
 
 
-### rafthttp
+## rafthttp
 
 | Name                              | Description                                | Type    | Labels                         |
 |-----------------------------------|--------------------------------------------|---------|--------------------------------|
@@ -106,7 +99,7 @@ Label `msgType` is the type of raft message. `MsgApp` is log replication message
 Label `remoteID` is the member ID of the message destination.
 
 
-### proxy
+## proxy
 
 etcd members operating in proxy mode do not do store operations. They forward all requests
  to cluster instances.
@@ -134,4 +127,4 @@ Example Prometheus queries that may be useful from these metrics (across all etc
  * `sum(rate(etcd_proxy_dropped_total{job="etcd"}[1m])) by (proxying_error)`
     
     Number of failed request on the proxy. This should be 0, spikes here indicate connectivity issues to etcd cluster.
-            
+ 

Documentation/other_apis.md

@@ -1,4 +1,4 @@
-## Members API
+# Members API
 
 * [List members](#list-members)
 * [Add a member](#add-a-member)
@@ -103,7 +103,7 @@ Change the peer urls of a given member. The member ID must be a hex-encoded uint
 
 If the POST body is malformed an HTTP 400 will be returned. If the member does not exist in the cluster an HTTP 404 will be returned. If any of the given peerURLs exists in the cluster an HTTP 409 will be returned. If the cluster fails to process the request within timeout an HTTP 500 will be returned, though the request may be processed later.
 
-#### Request
+### Request
 
 ```
 PUT /v2/members/<id> HTTP/1.1
@@ -111,7 +111,7 @@ PUT /v2/members/<id> HTTP/1.1
 {"peerURLs": ["http://10.0.0.10:2380"]}
 ```
 
-#### Example
+### Example
 
 ```sh
 curl http://10.0.0.10:2379/v2/members/272e204152 -XPUT \

Documentation/production-ready.md

@@ -1,4 +1,6 @@
+# etcd in Production
+
 etcd is being used successfully by many companies in production. It is,
-however, under active development and systems like etcd are difficult to get
-correct. If you are comfortable with bleeding-edge software please use etcd and
+however, under active development, and systems like etcd are difficult to get
+correct. If you are comfortable with bleeding-edge software, please use etcd and
 provide us with the feedback and testing young software needs.

Documentation/proxy.md

@@ -1,6 +1,6 @@
-## Proxy
+# Proxy
 
-etcd can now run as a transparent proxy. Running etcd as a proxy allows for easily discovery of etcd within your infrastructure, since it can run on each machine as a local service. In this mode, etcd acts as a reverse proxy and forwards client requests to an active etcd cluster. The etcd proxy does not participate in the consensus replication of the etcd cluster, thus it neither increases the resilience nor decreases the write performance of the etcd cluster.
+etcd can now run as a transparent proxy. Running etcd as a proxy allows for easy discovery of etcd within your infrastructure, since it can run on each machine as a local service. In this mode, etcd acts as a reverse proxy and forwards client requests to an active etcd cluster. The etcd proxy does not participate in the consensus replication of the etcd cluster, thus it neither increases the resilience nor decreases the write performance of the etcd cluster.
 
 etcd currently supports two proxy modes: `readwrite` and `readonly`. The default mode is `readwrite`, which forwards both read and write requests to the etcd cluster. A `readonly` etcd proxy only forwards read requests to the etcd cluster, and returns `HTTP 501` to all write requests. 
 
@@ -8,30 +8,114 @@ The proxy will shuffle the list of cluster members periodically to avoid sending
 
 The member list used by proxy consists of all client URLs advertised within the cluster, as specified in each members' `-advertise-client-urls` flag. If this flag is set incorrectly, requests sent to the proxy are forwarded to wrong addresses and then fail. Including URLs in the `-advertise-client-urls` flag that point to the proxy itself, e.g. http://localhost:2379, is even more problematic as it will cause loops, because the proxy keeps trying to forward requests to itself until its resources (memory, file descriptors) are eventually depleted. The fix for this problem is to restart etcd member with correct `-advertise-client-urls` flag. After client URLs list in proxy is recalculated, which happens every 30 seconds, requests will be forwarded correctly.
 
-### Using an etcd proxy
+## Using an etcd proxy
 To start etcd in proxy mode, you need to provide three flags: `proxy`, `listen-client-urls`, and `initial-cluster` (or `discovery`). 
 
 To start a readwrite proxy, set `-proxy on`; To start a readonly proxy, set `-proxy readonly`.
 
 The proxy will be listening on `listen-client-urls` and forward requests to the etcd cluster discovered from in `initial-cluster` or `discovery` url. 
 
-#### Start an etcd proxy with a static configuration
+### Start an etcd proxy with a static configuration
 To start a proxy that will connect to a statically defined etcd cluster, specify the `initial-cluster` flag:
 
 ```
-etcd -proxy on -listen-client-urls http://127.0.0.1:8080 -initial-cluster infra0=http://10.0.1.10:2380,infra1=http://10.0.1.11:2380,infra2=http://10.0.1.12:2380
+etcd -proxy on \
+-listen-client-urls http://127.0.0.1:8080 \
+-initial-cluster infra0=http://10.0.1.10:2380,infra1=http://10.0.1.11:2380,infra2=http://10.0.1.12:2380
 ```
 
-#### Start an etcd proxy with the discovery service
+### Start an etcd proxy with the discovery service
 If you bootstrap an etcd cluster using the [discovery service][discovery-service], you can also start the proxy with the same `discovery`. 
 
 To start a proxy using the discovery service, specify the `discovery` flag. The proxy will wait until the etcd cluster defined at the `discovery` url finishes bootstrapping, and then start to forward the requests. 
 
 ```
-etcd -proxy on -listen-client-urls http://127.0.0.1:8080 -discovery https://discovery.etcd.io/3e86b59982e49066c5d813af1c2e2579cbf573de
+etcd -proxy on \
+-listen-client-urls http://127.0.0.1:8080 \
+-discovery https://discovery.etcd.io/3e86b59982e49066c5d813af1c2e2579cbf573de
 ```
 
-#### Fallback to proxy mode with discovery service
+## Fallback to proxy mode with discovery service
+
 If you bootstrap a etcd cluster using [discovery service][discovery-service] with more than the expected number of etcd members, the extra etcd processes will fall back to being `readwrite` proxies by default. They will forward the requests to the cluster as described above. For example, if you create a discovery url with `size=5`, and start ten etcd processes using that same discovery url, the result will be a cluster with five etcd members and five proxies. Note that this behaviour can be disabled with the `proxy-fallback` flag.
 
+## Promote a proxy to a member of etcd cluster
+
+A Proxy is in the part of etcd cluster that does not participate in consensus. A proxy will not promote itself to an etcd member that participates in consensus automtically in any case.
+
+If you want to promote a proxy to an etcd member, there are four steps you need to follow:
+
+- use etcdctl to add the proxy node as an etcd member into the existing cluster
+- stop the etcd proxy process or service
+- remove the existing proxy data directory
+- restart the etcd process with new member configuration
+
+## Example
+
+We assume you have a one member etcd cluster with one proxy. The cluster information is listed below:
+
+|Name|Address|
+|------|---------|
+|infra0|10.0.1.10|
+|proxy0|10.0.1.11|
+
+This example walks you through a case that you promote one proxy to an etcd member. The cluster will become a two member cluster after finishing the four steps.
+
+### Add a new member into the existing cluster
+
+First, use etcdctl to add the member to the cluster, which will output the environment variables need to correctly configure the new member:
+
+``` bash
+$ etcdctl -endpoint http://10.0.1.10:2379 member add infra1 http://10.0.1.11:2380
+added member 9bf1b35fc7761a23 to cluster
+
+ETCD_NAME="infra1"
+ETCD_INITIAL_CLUSTER="infra0=http://10.0.1.10:2380,infra1=http://10.0.1.11:2380"
+ETCD_INITIAL_CLUSTER_STATE=existing
+```
+
+### Stop the proxy process
+
+Stop the existing proxy so we can wipe it's state on disk and reload it with the new configuration:
+
+``` bash
+px aux | grep etcd
+kill %etcd_proxy_pid%
+```
+
+or (if you are running etcd proxy as etcd service under systemd)
+
+``` bash
+sudo systemctl stop etcd
+```
+
+### Remove the existing proxy data dir
+
+``` bash
+rm -rf %data_dir%/proxy
+```
+
+### Start etcd as a new member
+
+Finally, start the reconfigured member and make sure it joins the cluster correctly:
+
+``` bash
+$ export ETCD_NAME="infra1"
+$ export ETCD_INITIAL_CLUSTER="infra0=http://10.0.1.10:2380,infra1=http://10.0.1.11:2380"
+$ export ETCD_INITIAL_CLUSTER_STATE=existing
+$ etcd -listen-client-urls http://10.0.1.11:2379 \
+-advertise-client-urls http://10.0.1.11:2379 \
+-listen-peer-urls http://10.0.1.11:2380 \
+-initial-advertise-peer-urls http://10.0.1.11:2380 \
+-data-dir %data_dir%
+```
+
+If you are running etcd under systemd, you should modify the service file with correct configuration and restart the service:
+
+``` bash
+sudo systemd restart etcd
+```
+
+If you see an error, you can read the [add member troubleshooting doc](runtime-configuration.md#error-cases).
+
 [discovery-service]: clustering.md#discovery

Documentation/reporting_bugs.md

@@ -1,4 +1,4 @@
-## Reporting Bugs
+# Reporting Bugs
 
 If you find bugs or documentation mistakes in etcd project, please let us know by [opening an issue](https://github.com/coreos/etcd/issues/new). We treat bugs and mistakes very seriously and believe no issue is too small. Before creating a bug report, please check there that one does not already exist.
 
@@ -20,7 +20,7 @@ We might ask you for further information to locate a bug. A duplicated bug repor
 
 ## Frequently Asked Questions
 
-### How to get stack trace
+### How to get a stack trace
 
 ``` bash
 $ kill -QUIT $PID

Documentation/rfc/v3api.md

@@ -1,4 +1,4 @@
-## Design
+# Design
 
 1. Flatten binary key-value space
     
@@ -32,9 +32,9 @@
 
 [protobuf](./v3api.proto)
 
-### Examples
+## Examples
 
-#### Put a key (foo=bar)
+### Put a key (foo=bar)
 ```
 // A put is always successful
 Put( PutRequest { key = foo, value = bar } )
@@ -47,7 +47,7 @@ PutResponse {
 }
 ```
 
-#### Get a key (assume we have foo=bar)
+### Get a key (assume we have foo=bar)
 ```
 Get ( RangeRequest { key = foo } )
 
@@ -68,7 +68,7 @@ RangeResponse {
 }
 ```
 
-#### Range over a key space (assume we have foo0=bar0… foo100=bar100)
+### Range over a key space (assume we have foo0=bar0… foo100=bar100)
 ```
 Range ( RangeRequest { key = foo, end_key = foo80, limit = 30  } )
 
@@ -97,7 +97,7 @@ RangeResponse {
 }
 ```
 
-#### Finish a txn (assume we have foo0=bar0, foo1=bar1)
+### Finish a txn (assume we have foo0=bar0, foo1=bar1)
 ```
 Txn(TxnRequest {
     // mod_revision of foo0 is equal to 1, mod_revision of foo1 is greater than 1
@@ -129,7 +129,7 @@ TxnResponse {
 }
 ```
 
-#### Watch on a key/range
+### Watch on a key/range
 
 ```
 Watch( WatchRequest{

Documentation/rfc/v3api.proto

@@ -1,7 +1,6 @@
 syntax = "proto3";
 
-// Interface exported by the server.
-service etcd {
+service KV {
   // Range gets the keys in the range from the store.
   rpc Range(RangeRequest) returns (RangeResponse) {}
 
@@ -20,11 +19,6 @@ service etcd {
   // and generates events with the same revision in the event history.
   rpc Txn(TxnRequest) returns (TxnResponse) {}
 
-  // Watch watches the events happening or happened in etcd. Both input and output
-  // are stream. One watch rpc can watch for multiple ranges and get a stream of
-  // events. The whole events history can be watched unless compacted.
-  rpc WatchRange(stream WatchRangeRequest) returns (stream WatchRangeResponse) {}
-
   // Compact compacts the event history in etcd. User should compact the
   // event history periodically, or it will grow infinitely.
   rpc Compact(CompactionRequest) returns (CompactionResponse) {}
@@ -50,6 +44,14 @@ service etcd {
   rpc LeaseKeepAlive(stream LeaseKeepAliveRequest) returns (stream LeaseKeepAliveResponse) {}
 }
 
+service watch {
+  // Watch watches the events happening or happened. Both input and output
+  // are stream. One watch rpc can watch for multiple keys or prefixs and
+  // get a stream of events. The whole events history can be watched unless
+  // compacted.
+  rpc Watch(stream WatchRequest) returns (stream WatchResponse) {}
+}
+
 message ResponseHeader {
   // an error type message?
   string error = 1;
@@ -190,21 +192,22 @@ message KeyValue {
   bytes value = 5;
 }
 
-message WatchRangeRequest {
-  // if the range_end is not given, the request returns the key.
+message WatchRequest {
+  // the key to be watched
   bytes key = 1;
-  // if the range_end is given, it gets the keys in range [key, range_end).
-  bytes range_end = 2;
+  // the prefix to be watched.
+  bytes prefix = 2;
   // start_revision is an optional revision (including) to watch from. No start_revision is "now".
   int64 start_revision = 3;
-  // end_revision is an optional revision (excluding) to end watch. No end_revision is "forever".
-  int64 end_revision = 4;
-  bool progress_notification = 5;
+  // TODO: support Range watch?
+  // TODO: support notification every time interval or revision increase?
+  // TODO: support cancel watch if the server cannot reach with majority?
 }
 
-message WatchRangeResponse {
+message WatchResponse {
   ResponseHeader header = 1;
-  repeated Event events = 2;
+  // TODO: support batched events response?
+  storagepb.Event event = 2;
 }
 
 message Event {

Documentation/runtime-configuration.md

@@ -1,8 +1,8 @@
-## Runtime Reconfiguration
+# Runtime Reconfiguration
 
 etcd comes with support for incremental runtime reconfiguration, which allows users to update the membership of the cluster at run time.
 
-Reconfiguration requests can only be processed when the the majority of the cluster members are functioning. It is **highly recommended** to always have a cluster size greater than two in production. It is unsafe to remove a member from a two member cluster. The majority of a two member cluster is also two. If there is a failure during the removal process, the cluster might not able to make progress and need to [restart from majority failure][majority failure].
+Reconfiguration requests can only be processed when the majority of the cluster members are functioning. It is **highly recommended** to always have a cluster size greater than two in production. It is unsafe to remove a member from a two member cluster. The majority of a two member cluster is also two. If there is a failure during the removal process, the cluster might not able to make progress and need to [restart from majority failure][majority failure].
 
 To better understand the design behind runtime reconfiguration, we suggest you read [this](runtime-reconf-design.md).
 
@@ -131,7 +131,7 @@ The new member will run as a part of the cluster and immediately begin catching
 If you are adding multiple members the best practice is to configure a single member at a time and verify it starts correctly before adding more new members.
 If you add a new member to a 1-node cluster, the cluster cannot make progress before the new member starts because it needs two members as majority to agree on the consensus. You will only see this behavior between the time `etcdctl member add` informs the cluster about the new member and the new member successfully establishing a connection to the existing one.
 
-#### Error Cases
+#### Error Cases When Adding Members
 
 In the following case we have not included our new host in the list of enumerated nodes.
 If this is a new cluster, the node must be added to the list of initial cluster members.
@@ -154,10 +154,18 @@ etcdserver: assign ids error: unmatched member while checking PeerURLs
 exit 1
 ```
 
-When we start etcd using the data directory of a removed member, etcd will exit automatically if it connects to any alive member in the cluster:
+When we start etcd using the data directory of a removed member, etcd will exit automatically if it connects to any active member in the cluster:
 
 ```sh
 $ etcd
 etcd: this member has been permanently removed from the cluster. Exiting.
 exit 1
 ```
+
+### Strict Reconfiguration Check Mode (`-strict-reconfig-check`)
+
+As described in the above, the best practice of adding new members is to configure a single member at a time and verify it starts correctly before adding more new members. This step by step approach is very important because if newly added members is not configured correctly (for example the peer URLs are incorrect), the cluster can lose quorum. The quorum loss happens since the newly added member are counted in the quorum even if that member is not reachable from other existing members. Also quorum loss might happen if there is a connectivity issue or there are operational issues.
+
+For avoiding this problem, etcd provides an option `-strict-reconfig-check`. If this option is passed to etcd, etcd rejects reconfiguration requests if the number of started members will be less than a quorum of the reconfigured cluster.
+
+It is recommended to enable this option. However, it is disabled by default because of keeping compatibility.

Documentation/runtime-reconf-design.md

@@ -1,10 +1,10 @@
-### Design of Runtime Reconfiguration
+# Design of Runtime Reconfiguration
 
 Runtime reconfiguration is one of the hardest and most error prone features in a distributed system, especially in a consensus based system like etcd.
 
 Read on to learn about the design of etcd's runtime reconfiguration commands and how we tackled these problems.
 
-### Two Phase Config Changes Keep you Safe
+## Two Phase Config Changes Keep you Safe
 
 In etcd, every runtime reconfiguration has to go through [two phases](Documentation/runtime-configuration.md#add-a-new-member) for safety reasons. For example, to add a member you need to first inform cluster of new configuration and then start the new member.
 
@@ -22,7 +22,7 @@ Without the explicit workflow around cluster membership etcd would be vulnerable
 
 We think runtime reconfiguration should be a low frequent operation. We made the decision to keep it explicit and user-driven to ensure configuration safety and keep your cluster always running smoothly under your control.
 
-### Permanent Loss of Quorum Requires New Cluster
+## Permanent Loss of Quorum Requires New Cluster
 
 If a cluster permanently loses a majority of its members, a new cluster will need to be started from an old data directory to recover the previous state.
 
@@ -30,7 +30,7 @@ It is entirely possible to force removing the failed members from the existing c
 
 If you have a correct deployment, the possibility of permanent majority lose is very low. But it is a severe enough problem that worth special care. We strongly suggest you to read the [disaster recovery documentation](admin_guide.md#disaster-recovery) and prepare for permanent majority lose before you put etcd into production.
 
-### Do Not Use Public Discovery Service For Runtime Reconfiguration
+## Do Not Use Public Discovery Service For Runtime Reconfiguration
 
 The public discovery service should only be used for bootstrapping a cluster. To join member into an existing cluster, you should use runtime reconfiguration API. 
 
@@ -38,10 +38,10 @@ Discovery service is designed for bootstrapping an etcd cluster in the cloud env
 
 It seems that using public discovery service is a convenient way to do runtime reconfiguration, after all discovery service already has all the cluster configuration information. However relying on public discovery service brings troubles: 
 
-1. it introduces a external dependencies for the entire life-cycle of your cluster, not just bootstrap time. If there is a network issue between your cluster and public discover service, your cluster will suffer from it.
+1. it introduces external dependencies for the entire life-cycle of your cluster, not just bootstrap time. If there is a network issue between your cluster and public discovery service, your cluster will suffer from it.
  
 2. public discovery service must reflect correct runtime configuration of your cluster during it life-cycle. It has to provide security mechanism to avoid bad actions, and it is hard. 
 
 3. public discovery service has to keep tens of thousands of cluster configurations. Our public discovery service backend is not ready for that workload.
 
-If you want to have a discovery service that supports runtime reconfiguration, the best choice is to build your private one.
+If you want to have a discovery service that supports runtime reconfiguration, the best choice is to build your private one.

Documentation/security.md

@@ -97,7 +97,7 @@ $ curl --cacert /path/to/ca.crt --cert /path/to/client.crt --key /path/to/client
   -L https://127.0.0.1:2379/v2/keys/foo -XPUT -d value=bar -v
 ```
 
-You should able to see:
+You should be able to see:
 
 ```
 ...
@@ -138,7 +138,7 @@ $ etcd -name infra1 -data-dir infra1 \
 
 # member2
 $ etcd -name infra2 -data-dir infra2 \
-  -peer-client-cert-atuh -peer-trusted-ca-file=/path/to/ca.crt -peer-cert-file=/path/to/member2.crt -peer-key-file=/path/to/member2.key \
+  -peer-client-cert-auth -peer-trusted-ca-file=/path/to/ca.crt -peer-cert-file=/path/to/member2.crt -peer-key-file=/path/to/member2.key \
   -initial-advertise-peer-urls=https://10.0.1.11:2380 -listen-peer-urls=https://10.0.1.11:2380 \
   -discovery ${DISCOVERY_URL}
 ```
@@ -152,7 +152,7 @@ The etcd members will form a cluster and all communication between members in th
 The internal protocol of etcd v2.0.x uses a lot of short-lived HTTP connections.
 So, when enabling TLS you may need to increase the heartbeat interval and election timeouts to reduce internal cluster connection churn.
 A reasonable place to start are these values: ` --heartbeat-interval 500 --election-timeout 2500`.
-This issues is resolved in the etcd v2.1.x series of releases which uses fewer connections.
+These issues are resolved in the etcd v2.1.x series of releases which uses fewer connections.
 
 ### I'm seeing a SSLv3 alert handshake failure when using SSL client authentication?
 

Documentation/tuning.md

@@ -1,16 +1,16 @@
-## Tuning
+# Tuning
 
 The default settings in etcd should work well for installations on a local network where the average network latency is low.
 However, when using etcd across multiple data centers or over networks with high latency you may need to tweak the heartbeat interval and election timeout settings.
 
 The network isn't the only source of latency. Each request and response may be impacted by slow disks on both the leader and follower. Each of these timeouts represents the total time from request to successful response from the other machine.
 
-### Time Parameters
+## Time Parameters
 
 The underlying distributed consensus protocol relies on two separate time parameters to ensure that nodes can handoff leadership if one stalls or goes offline.
 The first parameter is called the *Heartbeat Interval*.
 This is the frequency with which the leader will notify followers that it is still the leader.
-For best pratices, the parameter should be set around round-trip time between members.
+For best practices, the parameter should be set around round-trip time between members.
 By default, etcd uses a `100ms` heartbeat interval.
 
 The second parameter is the *Election Timeout*.
@@ -27,11 +27,14 @@ The election timeout should be set based on the heartbeat interval and average r
 Election timeouts must be at least 10 times the round-trip time so it can account for variance in your network.
 For example, if the round-trip time between your members is 10ms then you should have at least a 100ms election timeout.
 
-The upper limit of election timeout is 50000ms, which should only be used when deploying global etcd cluster. First, 5s is the upper limit of average global round-trip time. A reasonable round-trip time for the continental united states is 130ms, and the time between US and japan is around 350-400ms. Because package gets delayed a lot, and network situation may be terrible, 5s is a safe value for it. Then, because election timeout should be an order of magnitude bigger than broadcast time, 50s becomes its maximum.
-
 You should also set your election timeout to at least 5 to 10 times your heartbeat interval to account for variance in leader replication.
 For a heartbeat interval of 50ms you should set your election timeout to at least 250ms - 500ms.
 
+The upper limit of election timeout is 50000ms (50s), which should only be used when deploying a globally-distributed etcd cluster.
+A reasonable round-trip time for the continental United States is 130ms, and the time between US and Japan is around 350-400ms.
+If your network has uneven performance or regular packet delays/loss then it is possible that a couple of retries may be necessary to successfully send a packet. So 5s is a safe upper limit of global round-trip time.
+As the election timeout should be an order of magnitude bigger than broadcast time, in the case of ~5s for a globally distributed cluster, then 50 seconds becomes a reasonable maximum.
+
 The heartbeat interval and election timeout value should be the same for all members in one cluster. Setting different values for etcd members may disrupt cluster stability.
 
 You can override the default values on the command line:
@@ -46,7 +49,7 @@ $ ETCD_HEARTBEAT_INTERVAL=100 ETCD_ELECTION_TIMEOUT=500 etcd
 
 The values are specified in milliseconds.
 
-### Snapshots
+## Snapshots
 
 etcd appends all key changes to a log file.
 This log grows forever and is a complete linear history of every change made to the keys.

Documentation/upgrade_2_1.md

@@ -1,4 +1,4 @@
-## Upgrade etcd to 2.1
+# Upgrade etcd to 2.1
 
 In the general case, upgrading from etcd 2.0 to 2.1 can be a zero-downtime, rolling upgrade:
  - one by one, stop the etcd v2.0 processes and replace them with etcd v2.1 processes
@@ -6,15 +6,15 @@ In the general case, upgrading from etcd 2.0 to 2.1 can be a zero-downtime, roll
 
 Before [starting an upgrade](#upgrade-procedure), read through the rest of this guide to prepare.
 
-### Upgrade Checklists
+## Upgrade Checklists
 
-#### Upgrade Requirement
+### Upgrade Requirements
 
 To upgrade an existing etcd deployment to 2.1, you must be running 2.0. If you’re running a version of etcd before 2.0, you must upgrade to [2.0](https://github.com/coreos/etcd/releases/tag/v2.0.13) before upgrading to 2.1.
 
 Also, to ensure a smooth rolling upgrade, your running cluster must be healthy. You can check the health of the cluster by using `etcdctl cluster-health` command. 
 
-#### Preparedness 
+### Preparedness 
 
 Before upgrading etcd, always test the services relying on etcd in a staging environment before deploying the upgrade to the production environment. 
 
@@ -22,13 +22,13 @@ You might also want to [backup your data directory](admin_guide.md#backing-up-th
 
 etcd 2.1 introduces a new [authentication](auth_api.md) feature, which is disabled by default. If your deployment depends on these, you may want to test the auth features before enabling them in production.
 
-#### Mixed Versions
+### Mixed Versions
 
 While upgrading, an etcd cluster supports mixed versions of etcd members. The cluster is only considered upgraded once all its members are upgraded to 2.1.
 
 Internally, etcd members negotiate with each other to determine the overall etcd cluster version, which controls the reported cluster version and the supported features. For example, if you are mid-upgrade, any 2.1 features (such as the the authentication feature mentioned above) won’t be available.
 
-#### Limitations
+### Limitations
 
 If you encounter any issues during the upgrade, you can attempt to restart the etcd process in trouble using a newer v2.1 binary to solve the problem. One known issue is that etcd v2.0.0 and v2.0.2 may panic during rolling upgrades due to an existing bug, which has been fixed since etcd v2.0.3.
 
@@ -36,7 +36,7 @@ It might take up to 2 minutes for the newly upgraded member to catch up with the
 
 If you have even more data, this might take more time. If you have a data size larger than 100MB you should contact us before upgrading, so we can make sure the upgrades work smoothly.
 
-#### Downgrade
+### Downgrade
 
 If all members have been upgraded to v2.1, the cluster will be upgraded to v2.1, and downgrade is **not possible**. If any member is still v2.0, the cluster will remain in v2.0, and you can go back to use v2.0 binary. 
 

Documentation/upgrade_2_2.md

@@ -1,4 +1,4 @@
-## Upgrade etcd from 2.1 to 2.2
+# Upgrade etcd from 2.1 to 2.2
 
 In the general case, upgrading from etcd 2.1 to 2.2 can be a zero-downtime, rolling upgrade:
 
@@ -7,33 +7,33 @@ In the general case, upgrading from etcd 2.1 to 2.2 can be a zero-downtime, roll
 
 Before [starting an upgrade](#upgrade-procedure), read through the rest of this guide to prepare.
 
-### Upgrade Checklists
+## Upgrade Checklists
 
-#### Upgrade Requirement
+### Upgrade Requirement
 
 To upgrade an existing etcd deployment to 2.2, you must be running 2.1. If you’re running a version of etcd before 2.1, you must upgrade to [2.1](https://github.com/coreos/etcd/releases/tag/v2.1.2) before upgrading to 2.2.
 
 Also, to ensure a smooth rolling upgrade, your running cluster must be healthy. You can check the health of the cluster by using `etcdctl cluster-health` command. 
 
-#### Preparedness 
+### Preparedness 
 
 Before upgrading etcd, always test the services relying on etcd in a staging environment before deploying the upgrade to the production environment. 
 
 You might also want to [backup your data directory](admin_guide.md#backing-up-the-datastore) for a potential [downgrade](#downgrade).
 
-#### Mixed Versions
+### Mixed Versions
 
 While upgrading, an etcd cluster supports mixed versions of etcd members. The cluster is only considered upgraded once all its members are upgraded to 2.2.
 
 Internally, etcd members negotiate with each other to determine the overall etcd cluster version, which controls the reported cluster version and the supported features.
 
-#### Limitations
+### Limitations
 
 If you have a data size larger than 100MB you should contact us before upgrading, so we can make sure the upgrades work smoothly.
 
 Every etcd 2.2 member will do health checking across the cluster periodically. etcd 2.1 member does not support health checking. During the upgrade, etcd 2.2 member will log warning about the unhealthy state of etcd 2.1 member. You can ignore the warning. 
 
-#### Downgrade
+### Downgrade
 
 If all members have been upgraded to v2.2, the cluster will be upgraded to v2.2, and downgrade is **not possible**. If any member is still v2.1, the cluster will remain in v2.1, and you can go back to use v2.1 binary. 
 

Godeps/Godeps.json

@@ -1,6 +1,6 @@
 {
 	"ImportPath": "github.com/coreos/etcd",
-	"GoVersion": "go1.4.2",
+	"GoVersion": "go1.5",
 	"Packages": [
 		"./..."
 	],
@@ -10,6 +10,10 @@
 			"Comment": "null-5",
 			"Rev": "75cd24fc2f2c2a2088577d12123ddee5f54e0675"
 		},
+		{
+			"ImportPath": "github.com/akrennmair/gopcap",
+			"Rev": "00e11033259acb75598ba416495bb708d864a010"
+		},
 		{
 			"ImportPath": "github.com/beorn7/perks/quantile",
 			"Rev": "b965b613227fddccbfffe13eae360ed3fa822f8d"
@@ -27,6 +31,10 @@
 			"ImportPath": "github.com/bradfitz/http2",
 			"Rev": "3e36af6d3af0e56fa3da71099f864933dea3d9fb"
 		},
+		{
+			"ImportPath": "github.com/cheggaaa/pb",
+			"Rev": "da1f27ad1d9509b16f65f52fd9d8138b0f2dc7b2"
+		},
 		{
 			"ImportPath": "github.com/codegangsta/cli",
 			"Comment": "1.2.0-26-gf7ebb76",
@@ -79,20 +87,10 @@
 			"ImportPath": "github.com/matttproud/golang_protobuf_extensions/pbutil",
 			"Rev": "fc2b8d3a73c4867e51861bbdd5ae3c1f0869dd6a"
 		},
-		{
-			"ImportPath": "github.com/prometheus/client_golang/model",
-			"Comment": "0.5.0-10-ga842dc1",
-			"Rev": "a842dc11e0621c34a71cab634d1d0190a59802a8"
-		},
 		{
 			"ImportPath": "github.com/prometheus/client_golang/prometheus",
-			"Comment": "0.5.0-10-ga842dc1",
-			"Rev": "a842dc11e0621c34a71cab634d1d0190a59802a8"
-		},
-		{
-			"ImportPath": "github.com/prometheus/client_golang/text",
-			"Comment": "0.5.0-10-ga842dc1",
-			"Rev": "a842dc11e0621c34a71cab634d1d0190a59802a8"
+			"Comment": "0.7.0-52-ge51041b",
+			"Rev": "e51041b3fa41cece0dca035740ba6411905be473"
 		},
 		{
 			"ImportPath": "github.com/prometheus/client_model/go",
@@ -100,12 +98,20 @@
 			"Rev": "fa8ad6fec33561be4280a8f0514318c79d7f6cb6"
 		},
 		{
-			"ImportPath": "github.com/prometheus/procfs",
-			"Rev": "ee2372b58cee877abe07cde670d04d3b3bac5ee6"
+			"ImportPath": "github.com/prometheus/common/expfmt",
+			"Rev": "ffe929a3f4c4faeaa10f2b9535c2b1be3ad15650"
 		},
 		{
-			"ImportPath": "github.com/rakyll/pb",
-			"Rev": "dc507ad06b7462501281bb4691ee43f0b1d1ec37"
+			"ImportPath": "github.com/prometheus/common/model",
+			"Rev": "ffe929a3f4c4faeaa10f2b9535c2b1be3ad15650"
+		},
+		{
+			"ImportPath": "github.com/prometheus/procfs",
+			"Rev": "454a56f35412459b5e684fd5ec0f9211b94f002a"
+		},
+		{
+			"ImportPath": "github.com/spacejam/loghisto",
+			"Rev": "323309774dec8b7430187e46cd0793974ccca04a"
 		},
 		{
 			"ImportPath": "github.com/stretchr/testify/assert",
@@ -113,7 +119,7 @@
 		},
 		{
 			"ImportPath": "github.com/ugorji/go/codec",
-			"Rev": "5abd4e96a45c386928ed2ca2a7ef63e2533e18ec"
+			"Rev": "f1f1a805ed361a0e078bb537e4ea78cd37dcf065"
 		},
 		{
 			"ImportPath": "github.com/xiang90/probing",

Godeps/_workspace/src/github.com/akrennmair/gopcap/.gitignore

@@ -0,0 +1,5 @@
+#*
+*~
+/tools/pass/pass
+/tools/pcaptest/pcaptest
+/tools/tcpdump/tcpdump

Godeps/_workspace/src/github.com/akrennmair/gopcap/LICENSE

@@ -0,0 +1,27 @@
+Copyright (c) 2009-2011 Andreas Krennmair. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+   * Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+   * Redistributions in binary form must reproduce the above
+copyright notice, this list of conditions and the following disclaimer
+in the documentation and/or other materials provided with the
+distribution.
+   * Neither the name of Andreas Krennmair nor the names of its
+contributors may be used to endorse or promote products derived from
+this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Godeps/_workspace/src/github.com/akrennmair/gopcap/README.mkd

@@ -0,0 +1,11 @@
+# PCAP
+
+This is a simple wrapper around libpcap for Go.  Originally written by Andreas
+Krennmair <ak@synflood.at> and only minorly touched up by Mark Smith <mark@qq.is>.
+
+Please see the included pcaptest.go and tcpdump.go programs for instructions on
+how to use this library.
+
+Miek Gieben <miek@miek.nl> has created a more Go-like package and replaced functionality
+with standard functions from the standard library. The package has also been renamed to
+pcap.

Godeps/_workspace/src/github.com/akrennmair/gopcap/decode.go

@@ -0,0 +1,527 @@
+package pcap
+
+import (
+	"encoding/binary"
+	"fmt"
+	"net"
+	"reflect"
+	"strings"
+)
+
+const (
+	TYPE_IP   = 0x0800
+	TYPE_ARP  = 0x0806
+	TYPE_IP6  = 0x86DD
+	TYPE_VLAN = 0x8100
+
+	IP_ICMP = 1
+	IP_INIP = 4
+	IP_TCP  = 6
+	IP_UDP  = 17
+)
+
+const (
+	ERRBUF_SIZE = 256
+
+	// According to pcap-linktype(7).
+	LINKTYPE_NULL             = 0
+	LINKTYPE_ETHERNET         = 1
+	LINKTYPE_TOKEN_RING       = 6
+	LINKTYPE_ARCNET           = 7
+	LINKTYPE_SLIP             = 8
+	LINKTYPE_PPP              = 9
+	LINKTYPE_FDDI             = 10
+	LINKTYPE_ATM_RFC1483      = 100
+	LINKTYPE_RAW              = 101
+	LINKTYPE_PPP_HDLC         = 50
+	LINKTYPE_PPP_ETHER        = 51
+	LINKTYPE_C_HDLC           = 104
+	LINKTYPE_IEEE802_11       = 105
+	LINKTYPE_FRELAY           = 107
+	LINKTYPE_LOOP             = 108
+	LINKTYPE_LINUX_SLL        = 113
+	LINKTYPE_LTALK            = 104
+	LINKTYPE_PFLOG            = 117
+	LINKTYPE_PRISM_HEADER     = 119
+	LINKTYPE_IP_OVER_FC       = 122
+	LINKTYPE_SUNATM           = 123
+	LINKTYPE_IEEE802_11_RADIO = 127
+	LINKTYPE_ARCNET_LINUX     = 129
+	LINKTYPE_LINUX_IRDA       = 144
+	LINKTYPE_LINUX_LAPD       = 177
+)
+
+type addrHdr interface {
+	SrcAddr() string
+	DestAddr() string
+	Len() int
+}
+
+type addrStringer interface {
+	String(addr addrHdr) string
+}
+
+func decodemac(pkt []byte) uint64 {
+	mac := uint64(0)
+	for i := uint(0); i < 6; i++ {
+		mac = (mac << 8) + uint64(pkt[i])
+	}
+	return mac
+}
+
+// Decode decodes the headers of a Packet.
+func (p *Packet) Decode() {
+	if len(p.Data) <= 14 {
+		return
+	}
+
+	p.Type = int(binary.BigEndian.Uint16(p.Data[12:14]))
+	p.DestMac = decodemac(p.Data[0:6])
+	p.SrcMac = decodemac(p.Data[6:12])
+
+	if len(p.Data) >= 15 {
+		p.Payload = p.Data[14:]
+	}
+
+	switch p.Type {
+	case TYPE_IP:
+		p.decodeIp()
+	case TYPE_IP6:
+		p.decodeIp6()
+	case TYPE_ARP:
+		p.decodeArp()
+	case TYPE_VLAN:
+		p.decodeVlan()
+	}
+}
+
+func (p *Packet) headerString(headers []interface{}) string {
+	// If there's just one header, return that.
+	if len(headers) == 1 {
+		if hdr, ok := headers[0].(fmt.Stringer); ok {
+			return hdr.String()
+		}
+	}
+	// If there are two headers (IPv4/IPv6 -> TCP/UDP/IP..)
+	if len(headers) == 2 {
+		// Commonly the first header is an address.
+		if addr, ok := p.Headers[0].(addrHdr); ok {
+			if hdr, ok := p.Headers[1].(addrStringer); ok {
+				return fmt.Sprintf("%s %s", p.Time, hdr.String(addr))
+			}
+		}
+	}
+	// For IP in IP, we do a recursive call.
+	if len(headers) >= 2 {
+		if addr, ok := headers[0].(addrHdr); ok {
+			if _, ok := headers[1].(addrHdr); ok {
+				return fmt.Sprintf("%s > %s IP in IP: ",
+					addr.SrcAddr(), addr.DestAddr(), p.headerString(headers[1:]))
+			}
+		}
+	}
+
+	var typeNames []string
+	for _, hdr := range headers {
+		typeNames = append(typeNames, reflect.TypeOf(hdr).String())
+	}
+
+	return fmt.Sprintf("unknown [%s]", strings.Join(typeNames, ","))
+}
+
+// String prints a one-line representation of the packet header.
+// The output is suitable for use in a tcpdump program.
+func (p *Packet) String() string {
+	// If there are no headers, print "unsupported protocol".
+	if len(p.Headers) == 0 {
+		return fmt.Sprintf("%s unsupported protocol %d", p.Time, int(p.Type))
+	}
+	return fmt.Sprintf("%s %s", p.Time, p.headerString(p.Headers))
+}
+
+// Arphdr is a ARP packet header.
+type Arphdr struct {
+	Addrtype          uint16
+	Protocol          uint16
+	HwAddressSize     uint8
+	ProtAddressSize   uint8
+	Operation         uint16
+	SourceHwAddress   []byte
+	SourceProtAddress []byte
+	DestHwAddress     []byte
+	DestProtAddress   []byte
+}
+
+func (arp *Arphdr) String() (s string) {
+	switch arp.Operation {
+	case 1:
+		s = "ARP request"
+	case 2:
+		s = "ARP Reply"
+	}
+	if arp.Addrtype == LINKTYPE_ETHERNET && arp.Protocol == TYPE_IP {
+		s = fmt.Sprintf("%012x (%s) > %012x (%s)",
+			decodemac(arp.SourceHwAddress), arp.SourceProtAddress,
+			decodemac(arp.DestHwAddress), arp.DestProtAddress)
+	} else {
+		s = fmt.Sprintf("addrtype = %d protocol = %d", arp.Addrtype, arp.Protocol)
+	}
+	return
+}
+
+func (p *Packet) decodeArp() {
+	if len(p.Payload) < 8 {
+		return
+	}
+
+	pkt := p.Payload
+	arp := new(Arphdr)
+	arp.Addrtype = binary.BigEndian.Uint16(pkt[0:2])
+	arp.Protocol = binary.BigEndian.Uint16(pkt[2:4])
+	arp.HwAddressSize = pkt[4]
+	arp.ProtAddressSize = pkt[5]
+	arp.Operation = binary.BigEndian.Uint16(pkt[6:8])
+
+	if len(pkt) < int(8+2*arp.HwAddressSize+2*arp.ProtAddressSize) {
+		return
+	}
+	arp.SourceHwAddress = pkt[8 : 8+arp.HwAddressSize]
+	arp.SourceProtAddress = pkt[8+arp.HwAddressSize : 8+arp.HwAddressSize+arp.ProtAddressSize]
+	arp.DestHwAddress = pkt[8+arp.HwAddressSize+arp.ProtAddressSize : 8+2*arp.HwAddressSize+arp.ProtAddressSize]
+	arp.DestProtAddress = pkt[8+2*arp.HwAddressSize+arp.ProtAddressSize : 8+2*arp.HwAddressSize+2*arp.ProtAddressSize]
+
+	p.Headers = append(p.Headers, arp)
+
+	if len(pkt) >= int(8+2*arp.HwAddressSize+2*arp.ProtAddressSize) {
+		p.Payload = p.Payload[8+2*arp.HwAddressSize+2*arp.ProtAddressSize:]
+	}
+}
+
+// IPadr is the header of an IP packet.
+type Iphdr struct {
+	Version    uint8
+	Ihl        uint8
+	Tos        uint8
+	Length     uint16
+	Id         uint16
+	Flags      uint8
+	FragOffset uint16
+	Ttl        uint8
+	Protocol   uint8
+	Checksum   uint16
+	SrcIp      []byte
+	DestIp     []byte
+}
+
+func (p *Packet) decodeIp() {
+	if len(p.Payload) < 20 {
+		return
+	}
+
+	pkt := p.Payload
+	ip := new(Iphdr)
+
+	ip.Version = uint8(pkt[0]) >> 4
+	ip.Ihl = uint8(pkt[0]) & 0x0F
+	ip.Tos = pkt[1]
+	ip.Length = binary.BigEndian.Uint16(pkt[2:4])
+	ip.Id = binary.BigEndian.Uint16(pkt[4:6])
+	flagsfrags := binary.BigEndian.Uint16(pkt[6:8])
+	ip.Flags = uint8(flagsfrags >> 13)
+	ip.FragOffset = flagsfrags & 0x1FFF
+	ip.Ttl = pkt[8]
+	ip.Protocol = pkt[9]
+	ip.Checksum = binary.BigEndian.Uint16(pkt[10:12])
+	ip.SrcIp = pkt[12:16]
+	ip.DestIp = pkt[16:20]
+
+	pEnd := int(ip.Length)
+	if pEnd > len(pkt) {
+		pEnd = len(pkt)
+	}
+
+	if len(pkt) >= pEnd && int(ip.Ihl*4) < pEnd {
+		p.Payload = pkt[ip.Ihl*4 : pEnd]
+	} else {
+		p.Payload = []byte{}
+	}
+
+	p.Headers = append(p.Headers, ip)
+	p.IP = ip
+
+	switch ip.Protocol {
+	case IP_TCP:
+		p.decodeTcp()
+	case IP_UDP:
+		p.decodeUdp()
+	case IP_ICMP:
+		p.decodeIcmp()
+	case IP_INIP:
+		p.decodeIp()
+	}
+}
+
+func (ip *Iphdr) SrcAddr() string  { return net.IP(ip.SrcIp).String() }
+func (ip *Iphdr) DestAddr() string { return net.IP(ip.DestIp).String() }
+func (ip *Iphdr) Len() int         { return int(ip.Length) }
+
+type Vlanhdr struct {
+	Priority       byte
+	DropEligible   bool
+	VlanIdentifier int
+	Type           int // Not actually part of the vlan header, but the type of the actual packet
+}
+
+func (v *Vlanhdr) String() {
+	fmt.Sprintf("VLAN Priority:%d Drop:%v Tag:%d", v.Priority, v.DropEligible, v.VlanIdentifier)
+}
+
+func (p *Packet) decodeVlan() {
+	pkt := p.Payload
+	vlan := new(Vlanhdr)
+	if len(pkt) < 4 {
+		return
+	}
+
+	vlan.Priority = (pkt[2] & 0xE0) >> 13
+	vlan.DropEligible = pkt[2]&0x10 != 0
+	vlan.VlanIdentifier = int(binary.BigEndian.Uint16(pkt[:2])) & 0x0FFF
+	vlan.Type = int(binary.BigEndian.Uint16(p.Payload[2:4]))
+	p.Headers = append(p.Headers, vlan)
+
+	if len(pkt) >= 5 {
+		p.Payload = p.Payload[4:]
+	}
+
+	switch vlan.Type {
+	case TYPE_IP:
+		p.decodeIp()
+	case TYPE_IP6:
+		p.decodeIp6()
+	case TYPE_ARP:
+		p.decodeArp()
+	}
+}
+
+type Tcphdr struct {
+	SrcPort    uint16
+	DestPort   uint16
+	Seq        uint32
+	Ack        uint32
+	DataOffset uint8
+	Flags      uint16
+	Window     uint16
+	Checksum   uint16
+	Urgent     uint16
+	Data       []byte
+}
+
+const (
+	TCP_FIN = 1 << iota
+	TCP_SYN
+	TCP_RST
+	TCP_PSH
+	TCP_ACK
+	TCP_URG
+	TCP_ECE
+	TCP_CWR
+	TCP_NS
+)
+
+func (p *Packet) decodeTcp() {
+	if len(p.Payload) < 20 {
+		return
+	}
+
+	pkt := p.Payload
+	tcp := new(Tcphdr)
+	tcp.SrcPort = binary.BigEndian.Uint16(pkt[0:2])
+	tcp.DestPort = binary.BigEndian.Uint16(pkt[2:4])
+	tcp.Seq = binary.BigEndian.Uint32(pkt[4:8])
+	tcp.Ack = binary.BigEndian.Uint32(pkt[8:12])
+	tcp.DataOffset = (pkt[12] & 0xF0) >> 4
+	tcp.Flags = binary.BigEndian.Uint16(pkt[12:14]) & 0x1FF
+	tcp.Window = binary.BigEndian.Uint16(pkt[14:16])
+	tcp.Checksum = binary.BigEndian.Uint16(pkt[16:18])
+	tcp.Urgent = binary.BigEndian.Uint16(pkt[18:20])
+	if len(pkt) >= int(tcp.DataOffset*4) {
+		p.Payload = pkt[tcp.DataOffset*4:]
+	}
+	p.Headers = append(p.Headers, tcp)
+	p.TCP = tcp
+}
+
+func (tcp *Tcphdr) String(hdr addrHdr) string {
+	return fmt.Sprintf("TCP %s:%d > %s:%d %s SEQ=%d ACK=%d LEN=%d",
+		hdr.SrcAddr(), int(tcp.SrcPort), hdr.DestAddr(), int(tcp.DestPort),
+		tcp.FlagsString(), int64(tcp.Seq), int64(tcp.Ack), hdr.Len())
+}
+
+func (tcp *Tcphdr) FlagsString() string {
+	var sflags []string
+	if 0 != (tcp.Flags & TCP_SYN) {
+		sflags = append(sflags, "syn")
+	}
+	if 0 != (tcp.Flags & TCP_FIN) {
+		sflags = append(sflags, "fin")
+	}
+	if 0 != (tcp.Flags & TCP_ACK) {
+		sflags = append(sflags, "ack")
+	}
+	if 0 != (tcp.Flags & TCP_PSH) {
+		sflags = append(sflags, "psh")
+	}
+	if 0 != (tcp.Flags & TCP_RST) {
+		sflags = append(sflags, "rst")
+	}
+	if 0 != (tcp.Flags & TCP_URG) {
+		sflags = append(sflags, "urg")
+	}
+	if 0 != (tcp.Flags & TCP_NS) {
+		sflags = append(sflags, "ns")
+	}
+	if 0 != (tcp.Flags & TCP_CWR) {
+		sflags = append(sflags, "cwr")
+	}
+	if 0 != (tcp.Flags & TCP_ECE) {
+		sflags = append(sflags, "ece")
+	}
+	return fmt.Sprintf("[%s]", strings.Join(sflags, " "))
+}
+
+type Udphdr struct {
+	SrcPort  uint16
+	DestPort uint16
+	Length   uint16
+	Checksum uint16
+}
+
+func (p *Packet) decodeUdp() {
+	if len(p.Payload) < 8 {
+		return
+	}
+
+	pkt := p.Payload
+	udp := new(Udphdr)
+	udp.SrcPort = binary.BigEndian.Uint16(pkt[0:2])
+	udp.DestPort = binary.BigEndian.Uint16(pkt[2:4])
+	udp.Length = binary.BigEndian.Uint16(pkt[4:6])
+	udp.Checksum = binary.BigEndian.Uint16(pkt[6:8])
+	p.Headers = append(p.Headers, udp)
+	p.UDP = udp
+	if len(p.Payload) >= 8 {
+		p.Payload = pkt[8:]
+	}
+}
+
+func (udp *Udphdr) String(hdr addrHdr) string {
+	return fmt.Sprintf("UDP %s:%d > %s:%d LEN=%d CHKSUM=%d",
+		hdr.SrcAddr(), int(udp.SrcPort), hdr.DestAddr(), int(udp.DestPort),
+		int(udp.Length), int(udp.Checksum))
+}
+
+type Icmphdr struct {
+	Type     uint8
+	Code     uint8
+	Checksum uint16
+	Id       uint16
+	Seq      uint16
+	Data     []byte
+}
+
+func (p *Packet) decodeIcmp() *Icmphdr {
+	if len(p.Payload) < 8 {
+		return nil
+	}
+
+	pkt := p.Payload
+	icmp := new(Icmphdr)
+	icmp.Type = pkt[0]
+	icmp.Code = pkt[1]
+	icmp.Checksum = binary.BigEndian.Uint16(pkt[2:4])
+	icmp.Id = binary.BigEndian.Uint16(pkt[4:6])
+	icmp.Seq = binary.BigEndian.Uint16(pkt[6:8])
+	p.Payload = pkt[8:]
+	p.Headers = append(p.Headers, icmp)
+	return icmp
+}
+
+func (icmp *Icmphdr) String(hdr addrHdr) string {
+	return fmt.Sprintf("ICMP %s > %s Type = %d Code = %d ",
+		hdr.SrcAddr(), hdr.DestAddr(), icmp.Type, icmp.Code)
+}
+
+func (icmp *Icmphdr) TypeString() (result string) {
+	switch icmp.Type {
+	case 0:
+		result = fmt.Sprintf("Echo reply seq=%d", icmp.Seq)
+	case 3:
+		switch icmp.Code {
+		case 0:
+			result = "Network unreachable"
+		case 1:
+			result = "Host unreachable"
+		case 2:
+			result = "Protocol unreachable"
+		case 3:
+			result = "Port unreachable"
+		default:
+			result = "Destination unreachable"
+		}
+	case 8:
+		result = fmt.Sprintf("Echo request seq=%d", icmp.Seq)
+	case 30:
+		result = "Traceroute"
+	}
+	return
+}
+
+type Ip6hdr struct {
+	// http://www.networksorcery.com/enp/protocol/ipv6.htm
+	Version      uint8  // 4 bits
+	TrafficClass uint8  // 8 bits
+	FlowLabel    uint32 // 20 bits
+	Length       uint16 // 16 bits
+	NextHeader   uint8  // 8 bits, same as Protocol in Iphdr
+	HopLimit     uint8  // 8 bits
+	SrcIp        []byte // 16 bytes
+	DestIp       []byte // 16 bytes
+}
+
+func (p *Packet) decodeIp6() {
+	if len(p.Payload) < 40 {
+		return
+	}
+
+	pkt := p.Payload
+	ip6 := new(Ip6hdr)
+	ip6.Version = uint8(pkt[0]) >> 4
+	ip6.TrafficClass = uint8((binary.BigEndian.Uint16(pkt[0:2]) >> 4) & 0x00FF)
+	ip6.FlowLabel = binary.BigEndian.Uint32(pkt[0:4]) & 0x000FFFFF
+	ip6.Length = binary.BigEndian.Uint16(pkt[4:6])
+	ip6.NextHeader = pkt[6]
+	ip6.HopLimit = pkt[7]
+	ip6.SrcIp = pkt[8:24]
+	ip6.DestIp = pkt[24:40]
+
+	if len(p.Payload) >= 40 {
+		p.Payload = pkt[40:]
+	}
+
+	p.Headers = append(p.Headers, ip6)
+
+	switch ip6.NextHeader {
+	case IP_TCP:
+		p.decodeTcp()
+	case IP_UDP:
+		p.decodeUdp()
+	case IP_ICMP:
+		p.decodeIcmp()
+	case IP_INIP:
+		p.decodeIp()
+	}
+}
+
+func (ip6 *Ip6hdr) SrcAddr() string  { return net.IP(ip6.SrcIp).String() }
+func (ip6 *Ip6hdr) DestAddr() string { return net.IP(ip6.DestIp).String() }
+func (ip6 *Ip6hdr) Len() int         { return int(ip6.Length) }

Godeps/_workspace/src/github.com/akrennmair/gopcap/decode_test.go

@@ -0,0 +1,247 @@
+package pcap
+
+import (
+	"bytes"
+	"testing"
+	"time"
+)
+
+var testSimpleTcpPacket *Packet = &Packet{
+	Data: []byte{
+		0x00, 0x00, 0x0c, 0x9f, 0xf0, 0x20, 0xbc, 0x30, 0x5b, 0xe8, 0xd3, 0x49,
+		0x08, 0x00, 0x45, 0x00, 0x01, 0xa4, 0x39, 0xdf, 0x40, 0x00, 0x40, 0x06,
+		0x55, 0x5a, 0xac, 0x11, 0x51, 0x49, 0xad, 0xde, 0xfe, 0xe1, 0xc5, 0xf7,
+		0x00, 0x50, 0xc5, 0x7e, 0x0e, 0x48, 0x49, 0x07, 0x42, 0x32, 0x80, 0x18,
+		0x00, 0x73, 0xab, 0xb1, 0x00, 0x00, 0x01, 0x01, 0x08, 0x0a, 0x03, 0x77,
+		0x37, 0x9c, 0x42, 0x77, 0x5e, 0x3a, 0x47, 0x45, 0x54, 0x20, 0x2f, 0x20,
+		0x48, 0x54, 0x54, 0x50, 0x2f, 0x31, 0x2e, 0x31, 0x0d, 0x0a, 0x48, 0x6f,
+		0x73, 0x74, 0x3a, 0x20, 0x77, 0x77, 0x77, 0x2e, 0x66, 0x69, 0x73, 0x68,
+		0x2e, 0x63, 0x6f, 0x6d, 0x0d, 0x0a, 0x43, 0x6f, 0x6e, 0x6e, 0x65, 0x63,
+		0x74, 0x69, 0x6f, 0x6e, 0x3a, 0x20, 0x6b, 0x65, 0x65, 0x70, 0x2d, 0x61,
+		0x6c, 0x69, 0x76, 0x65, 0x0d, 0x0a, 0x55, 0x73, 0x65, 0x72, 0x2d, 0x41,
+		0x67, 0x65, 0x6e, 0x74, 0x3a, 0x20, 0x4d, 0x6f, 0x7a, 0x69, 0x6c, 0x6c,
+		0x61, 0x2f, 0x35, 0x2e, 0x30, 0x20, 0x28, 0x58, 0x31, 0x31, 0x3b, 0x20,
+		0x4c, 0x69, 0x6e, 0x75, 0x78, 0x20, 0x78, 0x38, 0x36, 0x5f, 0x36, 0x34,
+		0x29, 0x20, 0x41, 0x70, 0x70, 0x6c, 0x65, 0x57, 0x65, 0x62, 0x4b, 0x69,
+		0x74, 0x2f, 0x35, 0x33, 0x35, 0x2e, 0x32, 0x20, 0x28, 0x4b, 0x48, 0x54,
+		0x4d, 0x4c, 0x2c, 0x20, 0x6c, 0x69, 0x6b, 0x65, 0x20, 0x47, 0x65, 0x63,
+		0x6b, 0x6f, 0x29, 0x20, 0x43, 0x68, 0x72, 0x6f, 0x6d, 0x65, 0x2f, 0x31,
+		0x35, 0x2e, 0x30, 0x2e, 0x38, 0x37, 0x34, 0x2e, 0x31, 0x32, 0x31, 0x20,
+		0x53, 0x61, 0x66, 0x61, 0x72, 0x69, 0x2f, 0x35, 0x33, 0x35, 0x2e, 0x32,
+		0x0d, 0x0a, 0x41, 0x63, 0x63, 0x65, 0x70, 0x74, 0x3a, 0x20, 0x74, 0x65,
+		0x78, 0x74, 0x2f, 0x68, 0x74, 0x6d, 0x6c, 0x2c, 0x61, 0x70, 0x70, 0x6c,
+		0x69, 0x63, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x2f, 0x78, 0x68, 0x74, 0x6d,
+		0x6c, 0x2b, 0x78, 0x6d, 0x6c, 0x2c, 0x61, 0x70, 0x70, 0x6c, 0x69, 0x63,
+		0x61, 0x74, 0x69, 0x6f, 0x6e, 0x2f, 0x78, 0x6d, 0x6c, 0x3b, 0x71, 0x3d,
+		0x30, 0x2e, 0x39, 0x2c, 0x2a, 0x2f, 0x2a, 0x3b, 0x71, 0x3d, 0x30, 0x2e,
+		0x38, 0x0d, 0x0a, 0x41, 0x63, 0x63, 0x65, 0x70, 0x74, 0x2d, 0x45, 0x6e,
+		0x63, 0x6f, 0x64, 0x69, 0x6e, 0x67, 0x3a, 0x20, 0x67, 0x7a, 0x69, 0x70,
+		0x2c, 0x64, 0x65, 0x66, 0x6c, 0x61, 0x74, 0x65, 0x2c, 0x73, 0x64, 0x63,
+		0x68, 0x0d, 0x0a, 0x41, 0x63, 0x63, 0x65, 0x70, 0x74, 0x2d, 0x4c, 0x61,
+		0x6e, 0x67, 0x75, 0x61, 0x67, 0x65, 0x3a, 0x20, 0x65, 0x6e, 0x2d, 0x55,
+		0x53, 0x2c, 0x65, 0x6e, 0x3b, 0x71, 0x3d, 0x30, 0x2e, 0x38, 0x0d, 0x0a,
+		0x41, 0x63, 0x63, 0x65, 0x70, 0x74, 0x2d, 0x43, 0x68, 0x61, 0x72, 0x73,
+		0x65, 0x74, 0x3a, 0x20, 0x49, 0x53, 0x4f, 0x2d, 0x38, 0x38, 0x35, 0x39,
+		0x2d, 0x31, 0x2c, 0x75, 0x74, 0x66, 0x2d, 0x38, 0x3b, 0x71, 0x3d, 0x30,
+		0x2e, 0x37, 0x2c, 0x2a, 0x3b, 0x71, 0x3d, 0x30, 0x2e, 0x33, 0x0d, 0x0a,
+		0x0d, 0x0a,
+	}}
+
+func BenchmarkDecodeSimpleTcpPacket(b *testing.B) {
+	for i := 0; i < b.N; i++ {
+		testSimpleTcpPacket.Decode()
+	}
+}
+
+func TestDecodeSimpleTcpPacket(t *testing.T) {
+	p := testSimpleTcpPacket
+	p.Decode()
+	if p.DestMac != 0x00000c9ff020 {
+		t.Error("Dest mac", p.DestMac)
+	}
+	if p.SrcMac != 0xbc305be8d349 {
+		t.Error("Src mac", p.SrcMac)
+	}
+	if len(p.Headers) != 2 {
+		t.Error("Incorrect number of headers", len(p.Headers))
+		return
+	}
+	if ip, ipOk := p.Headers[0].(*Iphdr); ipOk {
+		if ip.Version != 4 {
+			t.Error("ip Version", ip.Version)
+		}
+		if ip.Ihl != 5 {
+			t.Error("ip header length", ip.Ihl)
+		}
+		if ip.Tos != 0 {
+			t.Error("ip TOS", ip.Tos)
+		}
+		if ip.Length != 420 {
+			t.Error("ip Length", ip.Length)
+		}
+		if ip.Id != 14815 {
+			t.Error("ip ID", ip.Id)
+		}
+		if ip.Flags != 0x02 {
+			t.Error("ip Flags", ip.Flags)
+		}
+		if ip.FragOffset != 0 {
+			t.Error("ip Fragoffset", ip.FragOffset)
+		}
+		if ip.Ttl != 64 {
+			t.Error("ip TTL", ip.Ttl)
+		}
+		if ip.Protocol != 6 {
+			t.Error("ip Protocol", ip.Protocol)
+		}
+		if ip.Checksum != 0x555A {
+			t.Error("ip Checksum", ip.Checksum)
+		}
+		if !bytes.Equal(ip.SrcIp, []byte{172, 17, 81, 73}) {
+			t.Error("ip Src", ip.SrcIp)
+		}
+		if !bytes.Equal(ip.DestIp, []byte{173, 222, 254, 225}) {
+			t.Error("ip Dest", ip.DestIp)
+		}
+		if tcp, tcpOk := p.Headers[1].(*Tcphdr); tcpOk {
+			if tcp.SrcPort != 50679 {
+				t.Error("tcp srcport", tcp.SrcPort)
+			}
+			if tcp.DestPort != 80 {
+				t.Error("tcp destport", tcp.DestPort)
+			}
+			if tcp.Seq != 0xc57e0e48 {
+				t.Error("tcp seq", tcp.Seq)
+			}
+			if tcp.Ack != 0x49074232 {
+				t.Error("tcp ack", tcp.Ack)
+			}
+			if tcp.DataOffset != 8 {
+				t.Error("tcp dataoffset", tcp.DataOffset)
+			}
+			if tcp.Flags != 0x18 {
+				t.Error("tcp flags", tcp.Flags)
+			}
+			if tcp.Window != 0x73 {
+				t.Error("tcp window", tcp.Window)
+			}
+			if tcp.Checksum != 0xabb1 {
+				t.Error("tcp checksum", tcp.Checksum)
+			}
+			if tcp.Urgent != 0 {
+				t.Error("tcp urgent", tcp.Urgent)
+			}
+		} else {
+			t.Error("Second header is not TCP header")
+		}
+	} else {
+		t.Error("First header is not IP header")
+	}
+	if string(p.Payload) != "GET / HTTP/1.1\r\nHost: www.fish.com\r\nConnection: keep-alive\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Encoding: gzip,deflate,sdch\r\nAccept-Language: en-US,en;q=0.8\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3\r\n\r\n" {
+		t.Error("--- PAYLOAD STRING ---\n", string(p.Payload), "\n--- PAYLOAD BYTES ---\n", p.Payload)
+	}
+}
+
+// Makes sure packet payload doesn't display the 6 trailing null of this packet
+// as part of the payload.  They're actually the ethernet trailer.
+func TestDecodeSmallTcpPacketHasEmptyPayload(t *testing.T) {
+	p := &Packet{
+		// This packet is only 54 bits (an empty TCP RST), thus 6 trailing null
+		// bytes are added by the ethernet layer to make it the minimum packet size.
+		Data: []byte{
+			0xbc, 0x30, 0x5b, 0xe8, 0xd3, 0x49, 0xb8, 0xac, 0x6f, 0x92, 0xd5, 0xbf,
+			0x08, 0x00, 0x45, 0x00, 0x00, 0x28, 0x00, 0x00, 0x40, 0x00, 0x40, 0x06,
+			0x3f, 0x9f, 0xac, 0x11, 0x51, 0xc5, 0xac, 0x11, 0x51, 0x49, 0x00, 0x63,
+			0x9a, 0xef, 0x00, 0x00, 0x00, 0x00, 0x2e, 0xc1, 0x27, 0x83, 0x50, 0x14,
+			0x00, 0x00, 0xc3, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+		}}
+	p.Decode()
+	if p.Payload == nil {
+		t.Error("Nil payload")
+	}
+	if len(p.Payload) != 0 {
+		t.Error("Non-empty payload:", p.Payload)
+	}
+}
+
+func TestDecodeVlanPacket(t *testing.T) {
+	p := &Packet{
+		Data: []byte{
+			0x00, 0x10, 0xdb, 0xff, 0x10, 0x00, 0x00, 0x15, 0x2c, 0x9d, 0xcc, 0x00, 0x81, 0x00, 0x01, 0xf7,
+			0x08, 0x00, 0x45, 0x00, 0x00, 0x28, 0x29, 0x8d, 0x40, 0x00, 0x7d, 0x06, 0x83, 0xa0, 0xac, 0x1b,
+			0xca, 0x8e, 0x45, 0x16, 0x94, 0xe2, 0xd4, 0x0a, 0x00, 0x50, 0xdf, 0xab, 0x9c, 0xc6, 0xcd, 0x1e,
+			0xe5, 0xd1, 0x50, 0x10, 0x01, 0x00, 0x5a, 0x74, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+		}}
+	p.Decode()
+	if p.Type != TYPE_VLAN {
+		t.Error("Didn't detect vlan")
+	}
+	if len(p.Headers) != 3 {
+		t.Error("Incorrect number of headers:", len(p.Headers))
+		for i, h := range p.Headers {
+			t.Errorf("Header %d: %#v", i, h)
+		}
+		t.FailNow()
+	}
+	if _, ok := p.Headers[0].(*Vlanhdr); !ok {
+		t.Errorf("First header isn't vlan: %q", p.Headers[0])
+	}
+	if _, ok := p.Headers[1].(*Iphdr); !ok {
+		t.Errorf("Second header isn't IP: %q", p.Headers[1])
+	}
+	if _, ok := p.Headers[2].(*Tcphdr); !ok {
+		t.Errorf("Third header isn't TCP: %q", p.Headers[2])
+	}
+}
+
+func TestDecodeFuzzFallout(t *testing.T) {
+	testData := []struct {
+		Data []byte
+	}{
+		{[]byte("000000000000\x81\x000")},
+		{[]byte("000000000000\x81\x00000")},
+		{[]byte("000000000000\x86\xdd0")},
+		{[]byte("000000000000\b\x000")},
+		{[]byte("000000000000\b\x060")},
+		{[]byte{}},
+		{[]byte("000000000000\b\x0600000000")},
+		{[]byte("000000000000\x86\xdd000000\x01000000000000000000000000000000000")},
+		{[]byte("000000000000\x81\x0000\b\x0600000000")},
+		{[]byte("000000000000\b\x00n0000000000000000000")},
+		{[]byte("000000000000\x86\xdd000000\x0100000000000000000000000000000000000")},
+		{[]byte("000000000000\x81\x0000\b\x00g0000000000000000000")},
+		//{[]byte()},
+		{[]byte("000000000000\b\x00400000000\x110000000000")},
+		{[]byte("0nMء\xfe\x13\x13\x81\x00gr\b\x00&x\xc9\xe5b'\x1e0\x00\x04\x00\x0020596224")},
+		{[]byte("000000000000\x81\x0000\b\x00400000000\x110000000000")},
+		{[]byte("000000000000\b\x00000000000\x0600\xff0000000")},
+		{[]byte("000000000000\x86\xdd000000\x06000000000000000000000000000000000")},
+		{[]byte("000000000000\x81\x0000\b\x00000000000\x0600b0000000")},
+		{[]byte("000000000000\x81\x0000\b\x00400000000\x060000000000")},
+		{[]byte("000000000000\x86\xdd000000\x11000000000000000000000000000000000")},
+		{[]byte("000000000000\x86\xdd000000\x0600000000000000000000000000000000000000000000M")},
+		{[]byte("000000000000\b\x00500000000\x0600000000000")},
+		{[]byte("0nM\xd80\xfe\x13\x13\x81\x00gr\b\x00&x\xc9\xe5b'\x1e0\x00\x04\x00\x0020596224")},
+	}
+
+	for _, entry := range testData {
+		pkt := &Packet{
+			Time:   time.Now(),
+			Caplen: uint32(len(entry.Data)),
+			Len:    uint32(len(entry.Data)),
+			Data:   entry.Data,
+		}
+
+		pkt.Decode()
+		/*
+			func() {
+				defer func() {
+					if err := recover(); err != nil {
+						t.Fatalf("%d. %q failed: %v", idx, string(entry.Data), err)
+					}
+				}()
+				pkt.Decode()
+			}()
+		*/
+	}
+}

Godeps/_workspace/src/github.com/akrennmair/gopcap/io.go

@@ -0,0 +1,206 @@
+package pcap
+
+import (
+	"encoding/binary"
+	"fmt"
+	"io"
+	"time"
+)
+
+// FileHeader is the parsed header of a pcap file.
+// http://wiki.wireshark.org/Development/LibpcapFileFormat
+type FileHeader struct {
+	MagicNumber  uint32
+	VersionMajor uint16
+	VersionMinor uint16
+	TimeZone     int32
+	SigFigs      uint32
+	SnapLen      uint32
+	Network      uint32
+}
+
+type PacketTime struct {
+	Sec  int32
+	Usec int32
+}
+
+// Convert the PacketTime to a go Time struct.
+func (p *PacketTime) Time() time.Time {
+	return time.Unix(int64(p.Sec), int64(p.Usec)*1000)
+}
+
+// Packet is a single packet parsed from a pcap file.
+//
+// Convenient access to IP, TCP, and UDP headers is provided after Decode()
+// is called if the packet is of the appropriate type.
+type Packet struct {
+	Time   time.Time // packet send/receive time
+	Caplen uint32    // bytes stored in the file (caplen <= len)
+	Len    uint32    // bytes sent/received
+	Data   []byte    // packet data
+
+	Type    int // protocol type, see LINKTYPE_*
+	DestMac uint64
+	SrcMac  uint64
+
+	Headers []interface{} // decoded headers, in order
+	Payload []byte        // remaining non-header bytes
+
+	IP  *Iphdr  // IP header (for IP packets, after decoding)
+	TCP *Tcphdr // TCP header (for TCP packets, after decoding)
+	UDP *Udphdr // UDP header (for UDP packets after decoding)
+}
+
+// Reader parses pcap files.
+type Reader struct {
+	flip         bool
+	buf          io.Reader
+	err          error
+	fourBytes    []byte
+	twoBytes     []byte
+	sixteenBytes []byte
+	Header       FileHeader
+}
+
+// NewReader reads pcap data from an io.Reader.
+func NewReader(reader io.Reader) (*Reader, error) {
+	r := &Reader{
+		buf:          reader,
+		fourBytes:    make([]byte, 4),
+		twoBytes:     make([]byte, 2),
+		sixteenBytes: make([]byte, 16),
+	}
+	switch magic := r.readUint32(); magic {
+	case 0xa1b2c3d4:
+		r.flip = false
+	case 0xd4c3b2a1:
+		r.flip = true
+	default:
+		return nil, fmt.Errorf("pcap: bad magic number: %0x", magic)
+	}
+	r.Header = FileHeader{
+		MagicNumber:  0xa1b2c3d4,
+		VersionMajor: r.readUint16(),
+		VersionMinor: r.readUint16(),
+		TimeZone:     r.readInt32(),
+		SigFigs:      r.readUint32(),
+		SnapLen:      r.readUint32(),
+		Network:      r.readUint32(),
+	}
+	return r, nil
+}
+
+// Next returns the next packet or nil if no more packets can be read.
+func (r *Reader) Next() *Packet {
+	d := r.sixteenBytes
+	r.err = r.read(d)
+	if r.err != nil {
+		return nil
+	}
+	timeSec := asUint32(d[0:4], r.flip)
+	timeUsec := asUint32(d[4:8], r.flip)
+	capLen := asUint32(d[8:12], r.flip)
+	origLen := asUint32(d[12:16], r.flip)
+
+	data := make([]byte, capLen)
+	if r.err = r.read(data); r.err != nil {
+		return nil
+	}
+	return &Packet{
+		Time:   time.Unix(int64(timeSec), int64(timeUsec)),
+		Caplen: capLen,
+		Len:    origLen,
+		Data:   data,
+	}
+}
+
+func (r *Reader) read(data []byte) error {
+	var err error
+	n, err := r.buf.Read(data)
+	for err == nil && n != len(data) {
+		var chunk int
+		chunk, err = r.buf.Read(data[n:])
+		n += chunk
+	}
+	if len(data) == n {
+		return nil
+	}
+	return err
+}
+
+func (r *Reader) readUint32() uint32 {
+	data := r.fourBytes
+	if r.err = r.read(data); r.err != nil {
+		return 0
+	}
+	return asUint32(data, r.flip)
+}
+
+func (r *Reader) readInt32() int32 {
+	data := r.fourBytes
+	if r.err = r.read(data); r.err != nil {
+		return 0
+	}
+	return int32(asUint32(data, r.flip))
+}
+
+func (r *Reader) readUint16() uint16 {
+	data := r.twoBytes
+	if r.err = r.read(data); r.err != nil {
+		return 0
+	}
+	return asUint16(data, r.flip)
+}
+
+// Writer writes a pcap file.
+type Writer struct {
+	writer io.Writer
+	buf    []byte
+}
+
+// NewWriter creates a Writer that stores output in an io.Writer.
+// The FileHeader is written immediately.
+func NewWriter(writer io.Writer, header *FileHeader) (*Writer, error) {
+	w := &Writer{
+		writer: writer,
+		buf:    make([]byte, 24),
+	}
+	binary.LittleEndian.PutUint32(w.buf, header.MagicNumber)
+	binary.LittleEndian.PutUint16(w.buf[4:], header.VersionMajor)
+	binary.LittleEndian.PutUint16(w.buf[6:], header.VersionMinor)
+	binary.LittleEndian.PutUint32(w.buf[8:], uint32(header.TimeZone))
+	binary.LittleEndian.PutUint32(w.buf[12:], header.SigFigs)
+	binary.LittleEndian.PutUint32(w.buf[16:], header.SnapLen)
+	binary.LittleEndian.PutUint32(w.buf[20:], header.Network)
+	if _, err := writer.Write(w.buf); err != nil {
+		return nil, err
+	}
+	return w, nil
+}
+
+// Writer writes a packet to the underlying writer.
+func (w *Writer) Write(pkt *Packet) error {
+	binary.LittleEndian.PutUint32(w.buf, uint32(pkt.Time.Unix()))
+	binary.LittleEndian.PutUint32(w.buf[4:], uint32(pkt.Time.Nanosecond()))
+	binary.LittleEndian.PutUint32(w.buf[8:], uint32(pkt.Time.Unix()))
+	binary.LittleEndian.PutUint32(w.buf[12:], pkt.Len)
+	if _, err := w.writer.Write(w.buf[:16]); err != nil {
+		return err
+	}
+	_, err := w.writer.Write(pkt.Data)
+	return err
+}
+
+func asUint32(data []byte, flip bool) uint32 {
+	if flip {
+		return binary.BigEndian.Uint32(data)
+	}
+	return binary.LittleEndian.Uint32(data)
+}
+
+func asUint16(data []byte, flip bool) uint16 {
+	if flip {
+		return binary.BigEndian.Uint16(data)
+	}
+	return binary.LittleEndian.Uint16(data)
+}

Godeps/_workspace/src/github.com/akrennmair/gopcap/pcap.go

@@ -0,0 +1,266 @@
+// Interface to both live and offline pcap parsing.
+package pcap
+
+/*
+#cgo linux LDFLAGS: -lpcap
+#cgo freebsd LDFLAGS: -lpcap
+#cgo darwin LDFLAGS: -lpcap
+#cgo windows CFLAGS: -I C:/WpdPack/Include
+#cgo windows,386 LDFLAGS: -L C:/WpdPack/Lib -lwpcap
+#cgo windows,amd64 LDFLAGS: -L C:/WpdPack/Lib/x64 -lwpcap
+#include <stdlib.h>
+#include <pcap.h>
+
+// Workaround for not knowing how to cast to const u_char**
+int hack_pcap_next_ex(pcap_t *p, struct pcap_pkthdr **pkt_header,
+                      u_char **pkt_data) {
+    return pcap_next_ex(p, pkt_header, (const u_char **)pkt_data);
+}
+*/
+import "C"
+import (
+	"errors"
+	"net"
+	"syscall"
+	"time"
+	"unsafe"
+)
+
+type Pcap struct {
+	cptr *C.pcap_t
+}
+
+type Stat struct {
+	PacketsReceived  uint32
+	PacketsDropped   uint32
+	PacketsIfDropped uint32
+}
+
+type Interface struct {
+	Name        string
+	Description string
+	Addresses   []IFAddress
+	// TODO: add more elements
+}
+
+type IFAddress struct {
+	IP      net.IP
+	Netmask net.IPMask
+	// TODO: add broadcast + PtP dst ?
+}
+
+func (p *Pcap) Next() (pkt *Packet) {
+	rv, _ := p.NextEx()
+	return rv
+}
+
+// Openlive opens a device and returns a *Pcap handler
+func Openlive(device string, snaplen int32, promisc bool, timeout_ms int32) (handle *Pcap, err error) {
+	var buf *C.char
+	buf = (*C.char)(C.calloc(ERRBUF_SIZE, 1))
+	h := new(Pcap)
+	var pro int32
+	if promisc {
+		pro = 1
+	}
+
+	dev := C.CString(device)
+	defer C.free(unsafe.Pointer(dev))
+
+	h.cptr = C.pcap_open_live(dev, C.int(snaplen), C.int(pro), C.int(timeout_ms), buf)
+	if nil == h.cptr {
+		handle = nil
+		err = errors.New(C.GoString(buf))
+	} else {
+		handle = h
+	}
+	C.free(unsafe.Pointer(buf))
+	return
+}
+
+func Openoffline(file string) (handle *Pcap, err error) {
+	var buf *C.char
+	buf = (*C.char)(C.calloc(ERRBUF_SIZE, 1))
+	h := new(Pcap)
+
+	cf := C.CString(file)
+	defer C.free(unsafe.Pointer(cf))
+
+	h.cptr = C.pcap_open_offline(cf, buf)
+	if nil == h.cptr {
+		handle = nil
+		err = errors.New(C.GoString(buf))
+	} else {
+		handle = h
+	}
+	C.free(unsafe.Pointer(buf))
+	return
+}
+
+func (p *Pcap) NextEx() (pkt *Packet, result int32) {
+	var pkthdr *C.struct_pcap_pkthdr
+
+	var buf_ptr *C.u_char
+	var buf unsafe.Pointer
+	result = int32(C.hack_pcap_next_ex(p.cptr, &pkthdr, &buf_ptr))
+
+	buf = unsafe.Pointer(buf_ptr)
+	if nil == buf {
+		return
+	}
+
+	pkt = new(Packet)
+	pkt.Time = time.Unix(int64(pkthdr.ts.tv_sec), int64(pkthdr.ts.tv_usec)*1000)
+	pkt.Caplen = uint32(pkthdr.caplen)
+	pkt.Len = uint32(pkthdr.len)
+	pkt.Data = C.GoBytes(buf, C.int(pkthdr.caplen))
+	return
+}
+
+func (p *Pcap) Close() {
+	C.pcap_close(p.cptr)
+}
+
+func (p *Pcap) Geterror() error {
+	return errors.New(C.GoString(C.pcap_geterr(p.cptr)))
+}
+
+func (p *Pcap) Getstats() (stat *Stat, err error) {
+	var cstats _Ctype_struct_pcap_stat
+	if -1 == C.pcap_stats(p.cptr, &cstats) {
+		return nil, p.Geterror()
+	}
+	stats := new(Stat)
+	stats.PacketsReceived = uint32(cstats.ps_recv)
+	stats.PacketsDropped = uint32(cstats.ps_drop)
+	stats.PacketsIfDropped = uint32(cstats.ps_ifdrop)
+
+	return stats, nil
+}
+
+func (p *Pcap) Setfilter(expr string) (err error) {
+	var bpf _Ctype_struct_bpf_program
+	cexpr := C.CString(expr)
+	defer C.free(unsafe.Pointer(cexpr))
+
+	if -1 == C.pcap_compile(p.cptr, &bpf, cexpr, 1, 0) {
+		return p.Geterror()
+	}
+
+	if -1 == C.pcap_setfilter(p.cptr, &bpf) {
+		C.pcap_freecode(&bpf)
+		return p.Geterror()
+	}
+	C.pcap_freecode(&bpf)
+	return nil
+}
+
+func Version() string {
+	return C.GoString(C.pcap_lib_version())
+}
+
+func (p *Pcap) Datalink() int {
+	return int(C.pcap_datalink(p.cptr))
+}
+
+func (p *Pcap) Setdatalink(dlt int) error {
+	if -1 == C.pcap_set_datalink(p.cptr, C.int(dlt)) {
+		return p.Geterror()
+	}
+	return nil
+}
+
+func DatalinkValueToName(dlt int) string {
+	if name := C.pcap_datalink_val_to_name(C.int(dlt)); name != nil {
+		return C.GoString(name)
+	}
+	return ""
+}
+
+func DatalinkValueToDescription(dlt int) string {
+	if desc := C.pcap_datalink_val_to_description(C.int(dlt)); desc != nil {
+		return C.GoString(desc)
+	}
+	return ""
+}
+
+func Findalldevs() (ifs []Interface, err error) {
+	var buf *C.char
+	buf = (*C.char)(C.calloc(ERRBUF_SIZE, 1))
+	defer C.free(unsafe.Pointer(buf))
+	var alldevsp *C.pcap_if_t
+
+	if -1 == C.pcap_findalldevs((**C.pcap_if_t)(&alldevsp), buf) {
+		return nil, errors.New(C.GoString(buf))
+	}
+	defer C.pcap_freealldevs((*C.pcap_if_t)(alldevsp))
+	dev := alldevsp
+	var i uint32
+	for i = 0; dev != nil; dev = (*C.pcap_if_t)(dev.next) {
+		i++
+	}
+	ifs = make([]Interface, i)
+	dev = alldevsp
+	for j := uint32(0); dev != nil; dev = (*C.pcap_if_t)(dev.next) {
+		var iface Interface
+		iface.Name = C.GoString(dev.name)
+		iface.Description = C.GoString(dev.description)
+		iface.Addresses = findalladdresses(dev.addresses)
+		// TODO: add more elements
+		ifs[j] = iface
+		j++
+	}
+	return
+}
+
+func findalladdresses(addresses *_Ctype_struct_pcap_addr) (retval []IFAddress) {
+	// TODO - make it support more than IPv4 and IPv6?
+	retval = make([]IFAddress, 0, 1)
+	for curaddr := addresses; curaddr != nil; curaddr = (*_Ctype_struct_pcap_addr)(curaddr.next) {
+		var a IFAddress
+		var err error
+		if a.IP, err = sockaddr_to_IP((*syscall.RawSockaddr)(unsafe.Pointer(curaddr.addr))); err != nil {
+			continue
+		}
+		if a.Netmask, err = sockaddr_to_IP((*syscall.RawSockaddr)(unsafe.Pointer(curaddr.addr))); err != nil {
+			continue
+		}
+		retval = append(retval, a)
+	}
+	return
+}
+
+func sockaddr_to_IP(rsa *syscall.RawSockaddr) (IP []byte, err error) {
+	switch rsa.Family {
+	case syscall.AF_INET:
+		pp := (*syscall.RawSockaddrInet4)(unsafe.Pointer(rsa))
+		IP = make([]byte, 4)
+		for i := 0; i < len(IP); i++ {
+			IP[i] = pp.Addr[i]
+		}
+		return
+	case syscall.AF_INET6:
+		pp := (*syscall.RawSockaddrInet6)(unsafe.Pointer(rsa))
+		IP = make([]byte, 16)
+		for i := 0; i < len(IP); i++ {
+			IP[i] = pp.Addr[i]
+		}
+		return
+	}
+	err = errors.New("Unsupported address type")
+	return
+}
+
+func (p *Pcap) Inject(data []byte) (err error) {
+	buf := (*C.char)(C.malloc((C.size_t)(len(data))))
+
+	for i := 0; i < len(data); i++ {
+		*(*byte)(unsafe.Pointer(uintptr(unsafe.Pointer(buf)) + uintptr(i))) = data[i]
+	}
+
+	if -1 == C.pcap_sendpacket(p.cptr, (*C.u_char)(unsafe.Pointer(buf)), (C.int)(len(data))) {
+		err = p.Geterror()
+	}
+	C.free(unsafe.Pointer(buf))
+	return
+}

Godeps/_workspace/src/github.com/akrennmair/gopcap/tools/benchmark/benchmark.go

@@ -0,0 +1,49 @@
+package main
+
+import (
+	"flag"
+	"fmt"
+	"os"
+	"runtime/pprof"
+	"time"
+
+	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/akrennmair/gopcap"
+)
+
+func main() {
+	var filename *string = flag.String("file", "", "filename")
+	var decode *bool = flag.Bool("d", false, "If true, decode each packet")
+	var cpuprofile *string = flag.String("cpuprofile", "", "filename")
+
+	flag.Parse()
+
+	h, err := pcap.Openoffline(*filename)
+	if err != nil {
+		fmt.Printf("Couldn't create pcap reader: %v", err)
+	}
+
+	if *cpuprofile != "" {
+		if out, err := os.Create(*cpuprofile); err == nil {
+			pprof.StartCPUProfile(out)
+			defer func() {
+				pprof.StopCPUProfile()
+				out.Close()
+			}()
+		} else {
+			panic(err)
+		}
+	}
+
+	i, nilPackets := 0, 0
+	start := time.Now()
+	for pkt, code := h.NextEx(); code != -2; pkt, code = h.NextEx() {
+		if pkt == nil {
+			nilPackets++
+		} else if *decode {
+			pkt.Decode()
+		}
+		i++
+	}
+	duration := time.Since(start)
+	fmt.Printf("Took %v to process %v packets, %v per packet, %d nil packets\n", duration, i, duration/time.Duration(i), nilPackets)
+}

Godeps/_workspace/src/github.com/akrennmair/gopcap/tools/pass/pass.go

@@ -0,0 +1,96 @@
+package main
+
+// Parses a pcap file, writes it back to disk, then verifies the files
+// are the same.
+import (
+	"bufio"
+	"flag"
+	"fmt"
+	"io"
+	"os"
+
+	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/akrennmair/gopcap"
+)
+
+var input *string = flag.String("input", "", "input file")
+var output *string = flag.String("output", "", "output file")
+var decode *bool = flag.Bool("decode", false, "print decoded packets")
+
+func copyPcap(dest, src string) {
+	f, err := os.Open(src)
+	if err != nil {
+		fmt.Printf("couldn't open %q: %v\n", src, err)
+		return
+	}
+	defer f.Close()
+	reader, err := pcap.NewReader(bufio.NewReader(f))
+	if err != nil {
+		fmt.Printf("couldn't create reader: %v\n", err)
+		return
+	}
+	w, err := os.Create(dest)
+	if err != nil {
+		fmt.Printf("couldn't open %q: %v\n", dest, err)
+		return
+	}
+	defer w.Close()
+	buf := bufio.NewWriter(w)
+	writer, err := pcap.NewWriter(buf, &reader.Header)
+	if err != nil {
+		fmt.Printf("couldn't create writer: %v\n", err)
+		return
+	}
+	for {
+		pkt := reader.Next()
+		if pkt == nil {
+			break
+		}
+		if *decode {
+			pkt.Decode()
+			fmt.Println(pkt.String())
+		}
+		writer.Write(pkt)
+	}
+	buf.Flush()
+}
+
+func check(dest, src string) {
+	f, err := os.Open(src)
+	if err != nil {
+		fmt.Printf("couldn't open %q: %v\n", src, err)
+		return
+	}
+	defer f.Close()
+	freader := bufio.NewReader(f)
+
+	g, err := os.Open(dest)
+	if err != nil {
+		fmt.Printf("couldn't open %q: %v\n", src, err)
+		return
+	}
+	defer g.Close()
+	greader := bufio.NewReader(g)
+
+	for {
+		fb, ferr := freader.ReadByte()
+		gb, gerr := greader.ReadByte()
+
+		if ferr == io.EOF && gerr == io.EOF {
+			break
+		}
+		if fb == gb {
+			continue
+		}
+		fmt.Println("FAIL")
+		return
+	}
+
+	fmt.Println("PASS")
+}
+
+func main() {
+	flag.Parse()
+
+	copyPcap(*output, *input)
+	check(*output, *input)
+}

Godeps/_workspace/src/github.com/akrennmair/gopcap/tools/pcaptest/pcaptest.go

@@ -0,0 +1,82 @@
+package main
+
+import (
+	"flag"
+	"fmt"
+	"time"
+
+	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/akrennmair/gopcap"
+)
+
+func min(x uint32, y uint32) uint32 {
+	if x < y {
+		return x
+	}
+	return y
+}
+
+func main() {
+	var device *string = flag.String("d", "", "device")
+	var file *string = flag.String("r", "", "file")
+	var expr *string = flag.String("e", "", "filter expression")
+
+	flag.Parse()
+
+	var h *pcap.Pcap
+	var err error
+
+	ifs, err := pcap.Findalldevs()
+	if len(ifs) == 0 {
+		fmt.Printf("Warning: no devices found : %s\n", err)
+	} else {
+		for i := 0; i < len(ifs); i++ {
+			fmt.Printf("dev %d: %s (%s)\n", i+1, ifs[i].Name, ifs[i].Description)
+		}
+	}
+
+	if *device != "" {
+		h, err = pcap.Openlive(*device, 65535, true, 0)
+		if h == nil {
+			fmt.Printf("Openlive(%s) failed: %s\n", *device, err)
+			return
+		}
+	} else if *file != "" {
+		h, err = pcap.Openoffline(*file)
+		if h == nil {
+			fmt.Printf("Openoffline(%s) failed: %s\n", *file, err)
+			return
+		}
+	} else {
+		fmt.Printf("usage: pcaptest [-d <device> | -r <file>]\n")
+		return
+	}
+	defer h.Close()
+
+	fmt.Printf("pcap version: %s\n", pcap.Version())
+
+	if *expr != "" {
+		fmt.Printf("Setting filter: %s\n", *expr)
+		err := h.Setfilter(*expr)
+		if err != nil {
+			fmt.Printf("Warning: setting filter failed: %s\n", err)
+		}
+	}
+
+	for pkt := h.Next(); pkt != nil; pkt = h.Next() {
+		fmt.Printf("time: %d.%06d (%s) caplen: %d len: %d\nData:",
+			int64(pkt.Time.Second()), int64(pkt.Time.Nanosecond()),
+			time.Unix(int64(pkt.Time.Second()), 0).String(), int64(pkt.Caplen), int64(pkt.Len))
+		for i := uint32(0); i < pkt.Caplen; i++ {
+			if i%32 == 0 {
+				fmt.Printf("\n")
+			}
+			if 32 <= pkt.Data[i] && pkt.Data[i] <= 126 {
+				fmt.Printf("%c", pkt.Data[i])
+			} else {
+				fmt.Printf(".")
+			}
+		}
+		fmt.Printf("\n\n")
+	}
+
+}

Godeps/_workspace/src/github.com/akrennmair/gopcap/tools/tcpdump/tcpdump.go

@@ -0,0 +1,121 @@
+package main
+
+import (
+	"bufio"
+	"flag"
+	"fmt"
+	"os"
+
+	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/akrennmair/gopcap"
+)
+
+const (
+	TYPE_IP  = 0x0800
+	TYPE_ARP = 0x0806
+	TYPE_IP6 = 0x86DD
+
+	IP_ICMP = 1
+	IP_INIP = 4
+	IP_TCP  = 6
+	IP_UDP  = 17
+)
+
+var out *bufio.Writer
+var errout *bufio.Writer
+
+func main() {
+	var device *string = flag.String("i", "", "interface")
+	var snaplen *int = flag.Int("s", 65535, "snaplen")
+	var hexdump *bool = flag.Bool("X", false, "hexdump")
+	expr := ""
+
+	out = bufio.NewWriter(os.Stdout)
+	errout = bufio.NewWriter(os.Stderr)
+
+	flag.Usage = func() {
+		fmt.Fprintf(errout, "usage: %s [ -i interface ] [ -s snaplen ] [ -X ] [ expression ]\n", os.Args[0])
+		errout.Flush()
+		os.Exit(1)
+	}
+
+	flag.Parse()
+
+	if len(flag.Args()) > 0 {
+		expr = flag.Arg(0)
+	}
+
+	if *device == "" {
+		devs, err := pcap.Findalldevs()
+		if err != nil {
+			fmt.Fprintf(errout, "tcpdump: couldn't find any devices: %s\n", err)
+		}
+		if 0 == len(devs) {
+			flag.Usage()
+		}
+		*device = devs[0].Name
+	}
+
+	h, err := pcap.Openlive(*device, int32(*snaplen), true, 0)
+	if h == nil {
+		fmt.Fprintf(errout, "tcpdump: %s\n", err)
+		errout.Flush()
+		return
+	}
+	defer h.Close()
+
+	if expr != "" {
+		ferr := h.Setfilter(expr)
+		if ferr != nil {
+			fmt.Fprintf(out, "tcpdump: %s\n", ferr)
+			out.Flush()
+		}
+	}
+
+	for pkt := h.Next(); pkt != nil; pkt = h.Next() {
+		pkt.Decode()
+		fmt.Fprintf(out, "%s\n", pkt.String())
+		if *hexdump {
+			Hexdump(pkt)
+		}
+		out.Flush()
+	}
+}
+
+func min(a, b int) int {
+	if a < b {
+		return a
+	}
+	return b
+}
+
+func Hexdump(pkt *pcap.Packet) {
+	for i := 0; i < len(pkt.Data); i += 16 {
+		Dumpline(uint32(i), pkt.Data[i:min(i+16, len(pkt.Data))])
+	}
+}
+
+func Dumpline(addr uint32, line []byte) {
+	fmt.Fprintf(out, "\t0x%04x: ", int32(addr))
+	var i uint16
+	for i = 0; i < 16 && i < uint16(len(line)); i++ {
+		if i%2 == 0 {
+			out.WriteString(" ")
+		}
+		fmt.Fprintf(out, "%02x", line[i])
+	}
+	for j := i; j <= 16; j++ {
+		if j%2 == 0 {
+			out.WriteString(" ")
+		}
+		out.WriteString("  ")
+	}
+	out.WriteString("  ")
+	for i = 0; i < 16 && i < uint16(len(line)); i++ {
+		if line[i] >= 32 && line[i] <= 126 {
+			fmt.Fprintf(out, "%c", line[i])
+		} else {
+			out.WriteString(".")
+		}
+	}
+	out.WriteString("\n")
+}

Godeps/_workspace/src/github.com/cheggaaa/pb/.travis.yml

@@ -0,0 +1,4 @@
+language: go
+go:
+- 1.4.2
+sudo: false

Godeps/_workspace/src/github.com/cheggaaa/pb/README.md

@@ -57,6 +57,12 @@ bar.ShowTimeLeft = true
 // show average speed    
 bar.ShowSpeed = true
 
+// sets the width of the progress bar
+bar.SetWidth(80)
+
+// sets the width of the progress bar, but if terminal size smaller will be ignored
+bar.SetMaxWidth(80)
+
 // convert output to readable format (like KB, MB)     
 bar.SetUnits(pb.U_BYTES)
 

Godeps/_workspace/src/github.com/cheggaaa/pb/example/copy/copy.go

@@ -1,14 +1,14 @@
 package main
 
 import (
-	"github.com/cheggaaa/pb"
-	"os"
 	"fmt"
+	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/cheggaaa/pb"
 	"io"
-	"time"
-	"strings"
 	"net/http"
+	"os"
 	"strconv"
+	"strings"
+	"time"
 )
 
 func main() {
@@ -18,12 +18,12 @@ func main() {
 		return
 	}
 	sourceName, destName := os.Args[1], os.Args[2]
-	
+
 	// check source
 	var source io.Reader
 	var sourceSize int64
 	if strings.HasPrefix(sourceName, "http://") {
-		// open as url	
+		// open as url
 		resp, err := http.Get(sourceName)
 		if err != nil {
 			fmt.Printf("Can't get %s: %v\n", sourceName, err)
@@ -54,9 +54,7 @@ func main() {
 		sourceSize = sourceStat.Size()
 		source = s
 	}
-	
-	
-	
+
 	// create dest
 	dest, err := os.Create(destName)
 	if err != nil {
@@ -64,15 +62,15 @@ func main() {
 		return
 	}
 	defer dest.Close()
-	
-	// create bar	
+
+	// create bar
 	bar := pb.New(int(sourceSize)).SetUnits(pb.U_BYTES).SetRefreshRate(time.Millisecond * 10)
 	bar.ShowSpeed = true
 	bar.Start()
-	
+
 	// create multi writer
 	writer := io.MultiWriter(dest, bar)
-	
+
 	// and copy
 	io.Copy(writer, source)
 	bar.Finish()

Godeps/_workspace/src/github.com/cheggaaa/pb/example/pb.go

@@ -1,7 +1,7 @@
 package main
 
 import (
-	"github.com/cheggaaa/pb"
+	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/cheggaaa/pb"
 	"time"
 )
 
@@ -13,7 +13,7 @@ func main() {
 	bar.ShowPercent = true
 
 	// show bar (by default already true)
-	bar.ShowPercent = true
+	bar.ShowBar = true
 
 	// no need counters
 	bar.ShowCounters = true

Godeps/_workspace/src/github.com/cheggaaa/pb/format.go

@@ -0,0 +1,45 @@
+package pb
+
+import (
+	"fmt"
+	"strconv"
+	"strings"
+)
+
+type Units int
+
+const (
+	// By default, without type handle
+	U_NO Units = iota
+	// Handle as b, Kb, Mb, etc
+	U_BYTES
+)
+
+// Format integer
+func Format(i int64, units Units) string {
+	switch units {
+	case U_BYTES:
+		return FormatBytes(i)
+	default:
+		// by default just convert to string
+		return strconv.FormatInt(i, 10)
+	}
+}
+
+// Convert bytes to human readable string. Like a 2 MB, 64.2 KB, 52 B
+func FormatBytes(i int64) (result string) {
+	switch {
+	case i > (1024 * 1024 * 1024 * 1024):
+		result = fmt.Sprintf("%.02f TB", float64(i)/1024/1024/1024/1024)
+	case i > (1024 * 1024 * 1024):
+		result = fmt.Sprintf("%.02f GB", float64(i)/1024/1024/1024)
+	case i > (1024 * 1024):
+		result = fmt.Sprintf("%.02f MB", float64(i)/1024/1024)
+	case i > 1024:
+		result = fmt.Sprintf("%.02f KB", float64(i)/1024)
+	default:
+		result = fmt.Sprintf("%d B", i)
+	}
+	result = strings.Trim(result, " ")
+	return
+}

Godeps/_workspace/src/github.com/cheggaaa/pb/format_test.go

@@ -0,0 +1,37 @@
+package pb
+
+import (
+	"fmt"
+	"strconv"
+	"testing"
+)
+
+func Test_DefaultsToInteger(t *testing.T) {
+	value := int64(1000)
+	expected := strconv.Itoa(int(value))
+	actual := Format(value, -1)
+
+	if actual != expected {
+		t.Error(fmt.Sprintf("Expected {%s} was {%s}", expected, actual))
+	}
+}
+
+func Test_CanFormatAsInteger(t *testing.T) {
+	value := int64(1000)
+	expected := strconv.Itoa(int(value))
+	actual := Format(value, U_NO)
+
+	if actual != expected {
+		t.Error(fmt.Sprintf("Expected {%s} was {%s}", expected, actual))
+	}
+}
+
+func Test_CanFormatAsBytes(t *testing.T) {
+	value := int64(1000)
+	expected := "1000 B"
+	actual := Format(value, U_BYTES)
+
+	if actual != expected {
+		t.Error(fmt.Sprintf("Expected {%s} was {%s}", expected, actual))
+	}
+}

Godeps/_workspace/src/github.com/cheggaaa/pb/pb.go

@@ -0,0 +1,367 @@
+package pb
+
+import (
+	"fmt"
+	"io"
+	"math"
+	"strings"
+	"sync"
+	"sync/atomic"
+	"time"
+	"unicode/utf8"
+)
+
+const (
+	// Default refresh rate - 200ms
+	DEFAULT_REFRESH_RATE = time.Millisecond * 200
+	FORMAT               = "[=>-]"
+)
+
+// DEPRECATED
+// variables for backward compatibility, from now do not work
+// use pb.Format and pb.SetRefreshRate
+var (
+	DefaultRefreshRate                         = DEFAULT_REFRESH_RATE
+	BarStart, BarEnd, Empty, Current, CurrentN string
+)
+
+// Create new progress bar object
+func New(total int) *ProgressBar {
+	return New64(int64(total))
+}
+
+// Create new progress bar object uding int64 as total
+func New64(total int64) *ProgressBar {
+	pb := &ProgressBar{
+		Total:         total,
+		RefreshRate:   DEFAULT_REFRESH_RATE,
+		ShowPercent:   true,
+		ShowCounters:  true,
+		ShowBar:       true,
+		ShowTimeLeft:  true,
+		ShowFinalTime: true,
+		Units:         U_NO,
+		ManualUpdate:  false,
+		isFinish:      make(chan struct{}),
+		currentValue:  -1,
+	}
+	return pb.Format(FORMAT)
+}
+
+// Create new object and start
+func StartNew(total int) *ProgressBar {
+	return New(total).Start()
+}
+
+// Callback for custom output
+// For example:
+// bar.Callback = func(s string) {
+//     mySuperPrint(s)
+// }
+//
+type Callback func(out string)
+
+type ProgressBar struct {
+	current int64 // current must be first member of struct (https://code.google.com/p/go/issues/detail?id=5278)
+
+	Total                            int64
+	RefreshRate                      time.Duration
+	ShowPercent, ShowCounters        bool
+	ShowSpeed, ShowTimeLeft, ShowBar bool
+	ShowFinalTime                    bool
+	Output                           io.Writer
+	Callback                         Callback
+	NotPrint                         bool
+	Units                            Units
+	Width                            int
+	ForceWidth                       bool
+	ManualUpdate                     bool
+
+	finishOnce sync.Once //Guards isFinish
+	isFinish   chan struct{}
+
+	startTime    time.Time
+	startValue   int64
+	currentValue int64
+
+	prefix, postfix string
+
+	BarStart string
+	BarEnd   string
+	Empty    string
+	Current  string
+	CurrentN string
+}
+
+// Start print
+func (pb *ProgressBar) Start() *ProgressBar {
+	pb.startTime = time.Now()
+	pb.startValue = pb.current
+	if pb.Total == 0 {
+		pb.ShowTimeLeft = false
+		pb.ShowPercent = false
+	}
+	if !pb.ManualUpdate {
+		go pb.writer()
+	}
+	return pb
+}
+
+// Increment current value
+func (pb *ProgressBar) Increment() int {
+	return pb.Add(1)
+}
+
+// Set current value
+func (pb *ProgressBar) Set(current int) *ProgressBar {
+	return pb.Set64(int64(current))
+}
+
+// Set64 sets the current value as int64
+func (pb *ProgressBar) Set64(current int64) *ProgressBar {
+	atomic.StoreInt64(&pb.current, current)
+	return pb
+}
+
+// Add to current value
+func (pb *ProgressBar) Add(add int) int {
+	return int(pb.Add64(int64(add)))
+}
+
+func (pb *ProgressBar) Add64(add int64) int64 {
+	return atomic.AddInt64(&pb.current, add)
+}
+
+// Set prefix string
+func (pb *ProgressBar) Prefix(prefix string) *ProgressBar {
+	pb.prefix = prefix
+	return pb
+}
+
+// Set postfix string
+func (pb *ProgressBar) Postfix(postfix string) *ProgressBar {
+	pb.postfix = postfix
+	return pb
+}
+
+// Set custom format for bar
+// Example: bar.Format("[=>_]")
+func (pb *ProgressBar) Format(format string) *ProgressBar {
+	formatEntries := strings.Split(format, "")
+	if len(formatEntries) == 5 {
+		pb.BarStart = formatEntries[0]
+		pb.BarEnd = formatEntries[4]
+		pb.Empty = formatEntries[3]
+		pb.Current = formatEntries[1]
+		pb.CurrentN = formatEntries[2]
+	}
+	return pb
+}
+
+// Set bar refresh rate
+func (pb *ProgressBar) SetRefreshRate(rate time.Duration) *ProgressBar {
+	pb.RefreshRate = rate
+	return pb
+}
+
+// Set units
+// bar.SetUnits(U_NO) - by default
+// bar.SetUnits(U_BYTES) - for Mb, Kb, etc
+func (pb *ProgressBar) SetUnits(units Units) *ProgressBar {
+	pb.Units = units
+	return pb
+}
+
+// Set max width, if width is bigger than terminal width, will be ignored
+func (pb *ProgressBar) SetMaxWidth(width int) *ProgressBar {
+	pb.Width = width
+	pb.ForceWidth = false
+	return pb
+}
+
+// Set bar width
+func (pb *ProgressBar) SetWidth(width int) *ProgressBar {
+	pb.Width = width
+	pb.ForceWidth = true
+	return pb
+}
+
+// End print
+func (pb *ProgressBar) Finish() {
+	//Protect multiple calls
+	pb.finishOnce.Do(func() {
+		close(pb.isFinish)
+		pb.write(atomic.LoadInt64(&pb.current))
+		if !pb.NotPrint {
+			fmt.Println()
+		}
+	})
+}
+
+// End print and write string 'str'
+func (pb *ProgressBar) FinishPrint(str string) {
+	pb.Finish()
+	fmt.Println(str)
+}
+
+// implement io.Writer
+func (pb *ProgressBar) Write(p []byte) (n int, err error) {
+	n = len(p)
+	pb.Add(n)
+	return
+}
+
+// implement io.Reader
+func (pb *ProgressBar) Read(p []byte) (n int, err error) {
+	n = len(p)
+	pb.Add(n)
+	return
+}
+
+// Create new proxy reader over bar
+func (pb *ProgressBar) NewProxyReader(r io.Reader) *Reader {
+	return &Reader{r, pb}
+}
+
+func (pb *ProgressBar) write(current int64) {
+	width := pb.GetWidth()
+
+	var percentBox, countersBox, timeLeftBox, speedBox, barBox, end, out string
+
+	// percents
+	if pb.ShowPercent {
+		percent := float64(current) / (float64(pb.Total) / float64(100))
+		percentBox = fmt.Sprintf(" %.02f %% ", percent)
+	}
+
+	// counters
+	if pb.ShowCounters {
+		if pb.Total > 0 {
+			countersBox = fmt.Sprintf("%s / %s ", Format(current, pb.Units), Format(pb.Total, pb.Units))
+		} else {
+			countersBox = Format(current, pb.Units) + " / ? "
+		}
+	}
+
+	// time left
+	fromStart := time.Now().Sub(pb.startTime)
+	currentFromStart := current - pb.startValue
+	select {
+	case <-pb.isFinish:
+		if pb.ShowFinalTime {
+			left := (fromStart / time.Second) * time.Second
+			timeLeftBox = left.String()
+		}
+	default:
+		if pb.ShowTimeLeft && currentFromStart > 0 {
+			perEntry := fromStart / time.Duration(currentFromStart)
+			left := time.Duration(pb.Total-currentFromStart) * perEntry
+			left = (left / time.Second) * time.Second
+			timeLeftBox = left.String()
+		}
+	}
+
+	// speed
+	if pb.ShowSpeed && currentFromStart > 0 {
+		fromStart := time.Now().Sub(pb.startTime)
+		speed := float64(currentFromStart) / (float64(fromStart) / float64(time.Second))
+		speedBox = Format(int64(speed), pb.Units) + "/s "
+	}
+
+	barWidth := utf8.RuneCountInString(countersBox + pb.BarStart + pb.BarEnd + percentBox + timeLeftBox + speedBox + pb.prefix + pb.postfix)
+	// bar
+	if pb.ShowBar {
+		size := width - barWidth
+		if size > 0 {
+			if pb.Total > 0 {
+				curCount := int(math.Ceil((float64(current) / float64(pb.Total)) * float64(size)))
+				emptCount := size - curCount
+				barBox = pb.BarStart
+				if emptCount < 0 {
+					emptCount = 0
+				}
+				if curCount > size {
+					curCount = size
+				}
+				if emptCount <= 0 {
+					barBox += strings.Repeat(pb.Current, curCount)
+				} else if curCount > 0 {
+					barBox += strings.Repeat(pb.Current, curCount-1) + pb.CurrentN
+				}
+
+				barBox += strings.Repeat(pb.Empty, emptCount) + pb.BarEnd
+			} else {
+
+				barBox = pb.BarStart
+				pos := size - int(current)%int(size)
+				if pos-1 > 0 {
+					barBox += strings.Repeat(pb.Empty, pos-1)
+				}
+				barBox += pb.Current
+				if size-pos-1 > 0 {
+					barBox += strings.Repeat(pb.Empty, size-pos-1)
+				}
+				barBox += pb.BarEnd
+			}
+		}
+	}
+
+	// check len
+	out = pb.prefix + countersBox + barBox + percentBox + speedBox + timeLeftBox + pb.postfix
+	if utf8.RuneCountInString(out) < width {
+		end = strings.Repeat(" ", width-utf8.RuneCountInString(out))
+	}
+
+	// and print!
+	switch {
+	case pb.Output != nil:
+		fmt.Fprint(pb.Output, "\r"+out+end)
+	case pb.Callback != nil:
+		pb.Callback(out + end)
+	case !pb.NotPrint:
+		fmt.Print("\r" + out + end)
+	}
+}
+
+func (pb *ProgressBar) GetWidth() int {
+	if pb.ForceWidth {
+		return pb.Width
+	}
+
+	width := pb.Width
+	termWidth, _ := terminalWidth()
+	if width == 0 || termWidth <= width {
+		width = termWidth
+	}
+
+	return width
+}
+
+// Write the current state of the progressbar
+func (pb *ProgressBar) Update() {
+	c := atomic.LoadInt64(&pb.current)
+	if c != pb.currentValue {
+		pb.write(c)
+		pb.currentValue = c
+	}
+}
+
+// Internal loop for writing progressbar
+func (pb *ProgressBar) writer() {
+	pb.Update()
+	for {
+		select {
+		case <-pb.isFinish:
+			return
+		case <-time.After(pb.RefreshRate):
+			pb.Update()
+		}
+	}
+}
+
+type window struct {
+	Row    uint16
+	Col    uint16
+	Xpixel uint16
+	Ypixel uint16
+}

Godeps/_workspace/src/github.com/cheggaaa/pb/pb_nix.go

@@ -0,0 +1,7 @@
+// +build linux darwin freebsd netbsd openbsd
+
+package pb
+
+import "syscall"
+
+const sys_ioctl = syscall.SYS_IOCTL

Godeps/_workspace/src/github.com/cheggaaa/pb/pb_solaris.go

@@ -0,0 +1,5 @@
+// +build solaris
+
+package pb
+
+const sys_ioctl = 54

Godeps/_workspace/src/github.com/cheggaaa/pb/pb_test.go

@@ -0,0 +1,37 @@
+package pb
+
+import (
+	"testing"
+)
+
+func Test_IncrementAddsOne(t *testing.T) {
+	count := 5000
+	bar := New(count)
+	expected := 1
+	actual := bar.Increment()
+
+	if actual != expected {
+		t.Errorf("Expected {%d} was {%d}", expected, actual)
+	}
+}
+
+func Test_Width(t *testing.T) {
+	count := 5000
+	bar := New(count)
+	width := 100
+	bar.SetWidth(100).Callback = func(out string) {
+		if len(out) != width {
+			t.Errorf("Bar width expected {%d} was {%d}", len(out), width)
+		}
+	}
+	bar.Start()
+	bar.Increment()
+	bar.Finish()
+}
+
+func Test_MultipleFinish(t *testing.T) {
+	bar := New(5000)
+	bar.Add(2000)
+	bar.Finish()
+	bar.Finish()
+}

Godeps/_workspace/src/github.com/cheggaaa/pb/pb_win.go

@@ -1,16 +1,16 @@
-// +build windows
-
-package pb
-
-import (
-	"github.com/olekukonko/ts"
-)
-
-func bold(str string) string {
-	return str
-}
-
-func terminalWidth() (int, error) {
-	size , err := ts.GetSize()
-	return size.Col() , err
-}
+// +build windows
+
+package pb
+
+import (
+	"github.com/olekukonko/ts"
+)
+
+func bold(str string) string {
+	return str
+}
+
+func terminalWidth() (int, error) {
+	size, err := ts.GetSize()
+	return size.Col(), err
+}

Godeps/_workspace/src/github.com/cheggaaa/pb/pb_x.go

@@ -1,8 +1,9 @@
-// +build linux darwin freebsd
+// +build linux darwin freebsd netbsd openbsd solaris
 
 package pb
 
 import (
+	"os"
 	"runtime"
 	"syscall"
 	"unsafe"
@@ -13,6 +14,16 @@ const (
 	TIOCGWINSZ_OSX = 1074295912
 )
 
+var tty *os.File
+
+func init() {
+	var err error
+	tty, err = os.Open("/dev/tty")
+	if err != nil {
+		tty = os.Stdin
+	}
+}
+
 func bold(str string) string {
 	return "\033[1m" + str + "\033[0m"
 }
@@ -23,8 +34,8 @@ func terminalWidth() (int, error) {
 	if runtime.GOOS == "darwin" {
 		tio = TIOCGWINSZ_OSX
 	}
-	res, _, err := syscall.Syscall(syscall.SYS_IOCTL,
-		uintptr(syscall.Stdin),
+	res, _, err := syscall.Syscall(sys_ioctl,
+		tty.Fd(),
 		uintptr(tio),
 		uintptr(unsafe.Pointer(w)),
 	)

Godeps/_workspace/src/github.com/cheggaaa/pb/reader.go

@@ -0,0 +1,17 @@
+package pb
+
+import (
+	"io"
+)
+
+// It's proxy reader, implement io.Reader
+type Reader struct {
+	io.Reader
+	bar *ProgressBar
+}
+
+func (r *Reader) Read(p []byte) (n int, err error) {
+	n, err = r.Reader.Read(p)
+	r.bar.Add(n)
+	return
+}

Godeps/_workspace/src/github.com/prometheus/client_golang/model/labelname_test.go

@@ -1,55 +0,0 @@
-// Copyright 2013 The Prometheus Authors
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package model
-
-import (
-	"sort"
-	"testing"
-)
-
-func testLabelNames(t testing.TB) {
-	var scenarios = []struct {
-		in  LabelNames
-		out LabelNames
-	}{
-		{
-			in:  LabelNames{"ZZZ", "zzz"},
-			out: LabelNames{"ZZZ", "zzz"},
-		},
-		{
-			in:  LabelNames{"aaa", "AAA"},
-			out: LabelNames{"AAA", "aaa"},
-		},
-	}
-
-	for i, scenario := range scenarios {
-		sort.Sort(scenario.in)
-
-		for j, expected := range scenario.out {
-			if expected != scenario.in[j] {
-				t.Errorf("%d.%d expected %s, got %s", i, j, expected, scenario.in[j])
-			}
-		}
-	}
-}
-
-func TestLabelNames(t *testing.T) {
-	testLabelNames(t)
-}
-
-func BenchmarkLabelNames(b *testing.B) {
-	for i := 0; i < b.N; i++ {
-		testLabelNames(b)
-	}
-}

Godeps/_workspace/src/github.com/prometheus/client_golang/model/labelset.go

@@ -1,64 +0,0 @@
-// Copyright 2013 The Prometheus Authors
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package model
-
-import (
-	"fmt"
-	"sort"
-	"strings"
-)
-
-// A LabelSet is a collection of LabelName and LabelValue pairs.  The LabelSet
-// may be fully-qualified down to the point where it may resolve to a single
-// Metric in the data store or not.  All operations that occur within the realm
-// of a LabelSet can emit a vector of Metric entities to which the LabelSet may
-// match.
-type LabelSet map[LabelName]LabelValue
-
-// Merge is a helper function to non-destructively merge two label sets.
-func (l LabelSet) Merge(other LabelSet) LabelSet {
-	result := make(LabelSet, len(l))
-
-	for k, v := range l {
-		result[k] = v
-	}
-
-	for k, v := range other {
-		result[k] = v
-	}
-
-	return result
-}
-
-func (l LabelSet) String() string {
-	labelStrings := make([]string, 0, len(l))
-	for label, value := range l {
-		labelStrings = append(labelStrings, fmt.Sprintf("%s=%q", label, value))
-	}
-
-	switch len(labelStrings) {
-	case 0:
-		return ""
-	default:
-		sort.Strings(labelStrings)
-		return fmt.Sprintf("{%s}", strings.Join(labelStrings, ", "))
-	}
-}
-
-// MergeFromMetric merges Metric into this LabelSet.
-func (l LabelSet) MergeFromMetric(m Metric) {
-	for k, v := range m {
-		l[k] = v
-	}
-}

Godeps/_workspace/src/github.com/prometheus/client_golang/model/metric.go

@@ -1,192 +0,0 @@
-// Copyright 2013 The Prometheus Authors
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package model
-
-import (
-	"encoding/json"
-	"fmt"
-	"sort"
-	"strings"
-)
-
-var separator = []byte{0}
-
-// A Metric is similar to a LabelSet, but the key difference is that a Metric is
-// a singleton and refers to one and only one stream of samples.
-type Metric map[LabelName]LabelValue
-
-// Equal compares the metrics.
-func (m Metric) Equal(o Metric) bool {
-	if len(m) != len(o) {
-		return false
-	}
-	for ln, lv := range m {
-		olv, ok := o[ln]
-		if !ok {
-			return false
-		}
-		if olv != lv {
-			return false
-		}
-	}
-	return true
-}
-
-// Before compares the metrics, using the following criteria:
-//
-// If m has fewer labels than o, it is before o. If it has more, it is not.
-//
-// If the number of labels is the same, the superset of all label names is
-// sorted alphanumerically. The first differing label pair found in that order
-// determines the outcome: If the label does not exist at all in m, then m is
-// before o, and vice versa. Otherwise the label value is compared
-// alphanumerically.
-//
-// If m and o are equal, the method returns false.
-func (m Metric) Before(o Metric) bool {
-	if len(m) < len(o) {
-		return true
-	}
-	if len(m) > len(o) {
-		return false
-	}
-
-	lns := make(LabelNames, 0, len(m)+len(o))
-	for ln := range m {
-		lns = append(lns, ln)
-	}
-	for ln := range o {
-		lns = append(lns, ln)
-	}
-	// It's probably not worth it to de-dup lns.
-	sort.Sort(lns)
-	for _, ln := range lns {
-		mlv, ok := m[ln]
-		if !ok {
-			return true
-		}
-		olv, ok := o[ln]
-		if !ok {
-			return false
-		}
-		if mlv < olv {
-			return true
-		}
-		if mlv > olv {
-			return false
-		}
-	}
-	return false
-}
-
-// String implements Stringer.
-func (m Metric) String() string {
-	metricName, hasName := m[MetricNameLabel]
-	numLabels := len(m) - 1
-	if !hasName {
-		numLabels = len(m)
-	}
-	labelStrings := make([]string, 0, numLabels)
-	for label, value := range m {
-		if label != MetricNameLabel {
-			labelStrings = append(labelStrings, fmt.Sprintf("%s=%q", label, value))
-		}
-	}
-
-	switch numLabels {
-	case 0:
-		if hasName {
-			return string(metricName)
-		}
-		return "{}"
-	default:
-		sort.Strings(labelStrings)
-		return fmt.Sprintf("%s{%s}", metricName, strings.Join(labelStrings, ", "))
-	}
-}
-
-// Fingerprint returns a Metric's Fingerprint.
-func (m Metric) Fingerprint() Fingerprint {
-	return metricToFingerprint(m)
-}
-
-// Fingerprint returns a Metric's Fingerprint calculated by a faster hashing
-// algorithm, which is, however, more susceptible to hash collisions.
-func (m Metric) FastFingerprint() Fingerprint {
-	return metricToFastFingerprint(m)
-}
-
-// Clone returns a copy of the Metric.
-func (m Metric) Clone() Metric {
-	clone := Metric{}
-	for k, v := range m {
-		clone[k] = v
-	}
-	return clone
-}
-
-// MergeFromLabelSet merges a label set into this Metric, prefixing a collision
-// prefix to the label names merged from the label set where required.
-func (m Metric) MergeFromLabelSet(labels LabelSet, collisionPrefix LabelName) {
-	for k, v := range labels {
-		if collisionPrefix != "" {
-			for {
-				if _, exists := m[k]; !exists {
-					break
-				}
-				k = collisionPrefix + k
-			}
-		}
-
-		m[k] = v
-	}
-}
-
-// COWMetric wraps a Metric to enable copy-on-write access patterns.
-type COWMetric struct {
-	Copied bool
-	Metric Metric
-}
-
-// Set sets a label name in the wrapped Metric to a given value and copies the
-// Metric initially, if it is not already a copy.
-func (m *COWMetric) Set(ln LabelName, lv LabelValue) {
-	m.doCOW()
-	m.Metric[ln] = lv
-}
-
-// Delete deletes a given label name from the wrapped Metric and copies the
-// Metric initially, if it is not already a copy.
-func (m *COWMetric) Delete(ln LabelName) {
-	m.doCOW()
-	delete(m.Metric, ln)
-}
-
-// doCOW copies the underlying Metric if it is not already a copy.
-func (m *COWMetric) doCOW() {
-	if !m.Copied {
-		m.Metric = m.Metric.Clone()
-		m.Copied = true
-	}
-}
-
-// String implements fmt.Stringer.
-func (m COWMetric) String() string {
-	return m.Metric.String()
-}
-
-// MarshalJSON implements json.Marshaler.
-func (m COWMetric) MarshalJSON() ([]byte, error) {
-	return json.Marshal(m.Metric)
-}

Godeps/_workspace/src/github.com/prometheus/client_golang/model/sample.go

@@ -1,79 +0,0 @@
-// Copyright 2013 The Prometheus Authors
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package model
-
-// Sample is a sample value with a timestamp and a metric.
-type Sample struct {
-	Metric    Metric
-	Value     SampleValue
-	Timestamp Timestamp
-}
-
-// Equal compares first the metrics, then the timestamp, then the value.
-func (s *Sample) Equal(o *Sample) bool {
-	if s == o {
-		return true
-	}
-
-	if !s.Metric.Equal(o.Metric) {
-		return false
-	}
-	if !s.Timestamp.Equal(o.Timestamp) {
-		return false
-	}
-	if !s.Value.Equal(o.Value) {
-		return false
-	}
-
-	return true
-}
-
-// Samples is a sortable Sample slice. It implements sort.Interface.
-type Samples []*Sample
-
-func (s Samples) Len() int {
-	return len(s)
-}
-
-// Less compares first the metrics, then the timestamp.
-func (s Samples) Less(i, j int) bool {
-	switch {
-	case s[i].Metric.Before(s[j].Metric):
-		return true
-	case s[j].Metric.Before(s[i].Metric):
-		return false
-	case s[i].Timestamp.Before(s[j].Timestamp):
-		return true
-	default:
-		return false
-	}
-}
-
-func (s Samples) Swap(i, j int) {
-	s[i], s[j] = s[j], s[i]
-}
-
-// Equal compares two sets of samples and returns true if they are equal.
-func (s Samples) Equal(o Samples) bool {
-	if len(s) != len(o) {
-		return false
-	}
-
-	for i, sample := range s {
-		if !sample.Equal(o[i]) {
-			return false
-		}
-	}
-	return true
-}

Godeps/_workspace/src/github.com/prometheus/client_golang/model/sample_test.go

@@ -1,114 +0,0 @@
-// Copyright 2013 The Prometheus Authors
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package model
-
-import (
-	"sort"
-	"testing"
-)
-
-func TestSamplesSort(t *testing.T) {
-	input := Samples{
-		&Sample{
-			Metric: Metric{
-				MetricNameLabel: "A",
-			},
-			Timestamp: 1,
-		},
-		&Sample{
-			Metric: Metric{
-				MetricNameLabel: "A",
-			},
-			Timestamp: 2,
-		},
-		&Sample{
-			Metric: Metric{
-				MetricNameLabel: "C",
-			},
-			Timestamp: 1,
-		},
-		&Sample{
-			Metric: Metric{
-				MetricNameLabel: "C",
-			},
-			Timestamp: 2,
-		},
-		&Sample{
-			Metric: Metric{
-				MetricNameLabel: "B",
-			},
-			Timestamp: 1,
-		},
-		&Sample{
-			Metric: Metric{
-				MetricNameLabel: "B",
-			},
-			Timestamp: 2,
-		},
-	}
-
-	expected := Samples{
-		&Sample{
-			Metric: Metric{
-				MetricNameLabel: "A",
-			},
-			Timestamp: 1,
-		},
-		&Sample{
-			Metric: Metric{
-				MetricNameLabel: "A",
-			},
-			Timestamp: 2,
-		},
-		&Sample{
-			Metric: Metric{
-				MetricNameLabel: "B",
-			},
-			Timestamp: 1,
-		},
-		&Sample{
-			Metric: Metric{
-				MetricNameLabel: "B",
-			},
-			Timestamp: 2,
-		},
-		&Sample{
-			Metric: Metric{
-				MetricNameLabel: "C",
-			},
-			Timestamp: 1,
-		},
-		&Sample{
-			Metric: Metric{
-				MetricNameLabel: "C",
-			},
-			Timestamp: 2,
-		},
-	}
-
-	sort.Sort(input)
-
-	for i, actual := range input {
-		actualFp := actual.Metric.Fingerprint()
-		expectedFp := expected[i].Metric.Fingerprint()
-
-		if !actualFp.Equal(expectedFp) {
-			t.Fatalf("%d. Incorrect fingerprint. Got %s; want %s", i, actualFp.String(), expectedFp.String())
-		}
-
-		if actual.Timestamp != expected[i].Timestamp {
-			t.Fatalf("%d. Incorrect timestamp. Got %s; want %s", i, actual.Timestamp, expected[i].Timestamp)
-		}
-	}
-}

Godeps/_workspace/src/github.com/prometheus/client_golang/model/samplevalue.go

@@ -1,37 +0,0 @@
-// Copyright 2013 The Prometheus Authors
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package model
-
-import (
-	"fmt"
-	"strconv"
-)
-
-// A SampleValue is a representation of a value for a given sample at a given
-// time.
-type SampleValue float64
-
-// Equal does a straight v==o.
-func (v SampleValue) Equal(o SampleValue) bool {
-	return v == o
-}
-
-// MarshalJSON implements json.Marshaler.
-func (v SampleValue) MarshalJSON() ([]byte, error) {
-	return []byte(fmt.Sprintf(`"%s"`, v)), nil
-}
-
-func (v SampleValue) String() string {
-	return strconv.FormatFloat(float64(v), 'f', -1, 64)
-}

Godeps/_workspace/src/github.com/prometheus/client_golang/model/timestamp.go

@@ -1,116 +0,0 @@
-// Copyright 2013 The Prometheus Authors
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package model
-
-import (
-	"math"
-	"strconv"
-
-	native_time "time"
-)
-
-// Timestamp is the number of milliseconds since the epoch
-// (1970-01-01 00:00 UTC) excluding leap seconds.
-type Timestamp int64
-
-const (
-	// MinimumTick is the minimum supported time resolution. This has to be
-	// at least native_time.Second in order for the code below to work.
-	MinimumTick = native_time.Millisecond
-	// second is the timestamp duration equivalent to one second.
-	second = int64(native_time.Second / MinimumTick)
-	// The number of nanoseconds per minimum tick.
-	nanosPerTick = int64(MinimumTick / native_time.Nanosecond)
-
-	// Earliest is the earliest timestamp representable. Handy for
-	// initializing a high watermark.
-	Earliest = Timestamp(math.MinInt64)
-	// Latest is the latest timestamp representable. Handy for initializing
-	// a low watermark.
-	Latest = Timestamp(math.MaxInt64)
-)
-
-// Equal reports whether two timestamps represent the same instant.
-func (t Timestamp) Equal(o Timestamp) bool {
-	return t == o
-}
-
-// Before reports whether the timestamp t is before o.
-func (t Timestamp) Before(o Timestamp) bool {
-	return t < o
-}
-
-// After reports whether the timestamp t is after o.
-func (t Timestamp) After(o Timestamp) bool {
-	return t > o
-}
-
-// Add returns the Timestamp t + d.
-func (t Timestamp) Add(d native_time.Duration) Timestamp {
-	return t + Timestamp(d/MinimumTick)
-}
-
-// Sub returns the Duration t - o.
-func (t Timestamp) Sub(o Timestamp) native_time.Duration {
-	return native_time.Duration(t-o) * MinimumTick
-}
-
-// Time returns the time.Time representation of t.
-func (t Timestamp) Time() native_time.Time {
-	return native_time.Unix(int64(t)/second, (int64(t)%second)*nanosPerTick)
-}
-
-// Unix returns t as a Unix time, the number of seconds elapsed
-// since January 1, 1970 UTC.
-func (t Timestamp) Unix() int64 {
-	return int64(t) / second
-}
-
-// UnixNano returns t as a Unix time, the number of nanoseconds elapsed
-// since January 1, 1970 UTC.
-func (t Timestamp) UnixNano() int64 {
-	return int64(t) * nanosPerTick
-}
-
-// String returns a string representation of the timestamp.
-func (t Timestamp) String() string {
-	return strconv.FormatFloat(float64(t)/float64(second), 'f', -1, 64)
-}
-
-// MarshalJSON implements the json.Marshaler interface.
-func (t Timestamp) MarshalJSON() ([]byte, error) {
-	return []byte(t.String()), nil
-}
-
-// Now returns the current time as a Timestamp.
-func Now() Timestamp {
-	return TimestampFromTime(native_time.Now())
-}
-
-// TimestampFromTime returns the Timestamp equivalent to the time.Time t.
-func TimestampFromTime(t native_time.Time) Timestamp {
-	return TimestampFromUnixNano(t.UnixNano())
-}
-
-// TimestampFromUnix returns the Timestamp equivalent to the Unix timestamp t
-// provided in seconds.
-func TimestampFromUnix(t int64) Timestamp {
-	return Timestamp(t * second)
-}
-
-// TimestampFromUnixNano returns the Timestamp equivalent to the Unix timestamp
-// t provided in nanoseconds.
-func TimestampFromUnixNano(t int64) Timestamp {
-	return Timestamp(t / nanosPerTick)
-}

Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/desc.go

@@ -9,18 +9,19 @@ import (
 	"sort"
 	"strings"
 
-	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_golang/model"
-
-	dto "github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_model/go"
-
 	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/golang/protobuf/proto"
+	dto "github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_model/go"
 )
 
 var (
 	metricNameRE = regexp.MustCompile(`^[a-zA-Z_][a-zA-Z0-9_:]*$`)
-	labelNameRE  = regexp.MustCompile(`^[a-zA-Z_][a-zA-Z0-9_]*$`)
+	labelNameRE  = regexp.MustCompile("^[a-zA-Z_][a-zA-Z0-9_]*$")
 )
 
+// reservedLabelPrefix is a prefix which is not legal in user-supplied
+// label names.
+const reservedLabelPrefix = "__"
+
 // Labels represents a collection of label name -> value mappings. This type is
 // commonly used with the With(Labels) and GetMetricWith(Labels) methods of
 // metric vector Collectors, e.g.:
@@ -134,7 +135,7 @@ func NewDesc(fqName, help string, variableLabels []string, constLabels Labels) *
 	for _, val := range labelValues {
 		b.Reset()
 		b.WriteString(val)
-		b.WriteByte(model.SeparatorByte)
+		b.WriteByte(separatorByte)
 		h.Write(b.Bytes())
 	}
 	d.id = h.Sum64()
@@ -145,12 +146,12 @@ func NewDesc(fqName, help string, variableLabels []string, constLabels Labels) *
 	h.Reset()
 	b.Reset()
 	b.WriteString(help)
-	b.WriteByte(model.SeparatorByte)
+	b.WriteByte(separatorByte)
 	h.Write(b.Bytes())
 	for _, labelName := range labelNames {
 		b.Reset()
 		b.WriteString(labelName)
-		b.WriteByte(model.SeparatorByte)
+		b.WriteByte(separatorByte)
 		h.Write(b.Bytes())
 	}
 	d.dimHash = h.Sum64()
@@ -195,5 +196,5 @@ func (d *Desc) String() string {
 
 func checkLabelName(l string) bool {
 	return labelNameRE.MatchString(l) &&
-		!strings.HasPrefix(l, model.ReservedLabelPrefix)
+		!strings.HasPrefix(l, reservedLabelPrefix)
 }

Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/doc.go

@@ -61,12 +61,13 @@
 // It also exports some stats about the HTTP usage of the /metrics
 // endpoint. (See the Handler function for more detail.)
 //
-// A more advanced metric type is the Summary.
+// Two more advanced metric types are the Summary and Histogram.
 //
-// In addition to the fundamental metric types Gauge, Counter, and Summary, a
-// very important part of the Prometheus data model is the partitioning of
-// samples along dimensions called labels, which results in metric vectors. The
-// fundamental types are GaugeVec, CounterVec, and SummaryVec.
+// In addition to the fundamental metric types Gauge, Counter, Summary, and
+// Histogram, a very important part of the Prometheus data model is the
+// partitioning of samples along dimensions called labels, which results in
+// metric vectors. The fundamental types are GaugeVec, CounterVec, SummaryVec,
+// and HistogramVec.
 //
 // Those are all the parts needed for basic usage. Detailed documentation and
 // examples are provided below.

Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/example_selfcollector_test.go

@@ -17,10 +17,8 @@ import (
 	"runtime"
 
 	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/golang/protobuf/proto"
-
-	dto "github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_model/go"
-
 	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus"
+	dto "github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_model/go"
 )
 
 func NewCallbackMetric(desc *prometheus.Desc, callback func() float64) *CallbackMetric {

Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/examples_test.go

@@ -23,11 +23,9 @@ import (
 	"sort"
 	"time"
 
-	dto "github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_model/go"
-
 	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/golang/protobuf/proto"
-
 	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus"
+	dto "github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_model/go"
 )
 
 func ExampleGauge() {
@@ -392,6 +390,9 @@ func ExampleSummaryVec() {
 		temps.WithLabelValues("lithobates-catesbeianus").Observe(32 + math.Floor(100*math.Cos(float64(i)*0.11))/10)
 	}
 
+	// Create a Summary without any observations.
+	temps.WithLabelValues("leiopelma-hochstetteri")
+
 	// Just for demonstration, let's check the state of the summary vector
 	// by (ab)using its Collect method and the Write method of its elements
 	// (which is usually only used by Prometheus internally - code like the
@@ -414,6 +415,26 @@ func ExampleSummaryVec() {
 	// Output:
 	// [label: <
 	//   name: "species"
+	//   value: "leiopelma-hochstetteri"
+	// >
+	// summary: <
+	//   sample_count: 0
+	//   sample_sum: 0
+	//   quantile: <
+	//     quantile: 0.5
+	//     value: nan
+	//   >
+	//   quantile: <
+	//     quantile: 0.9
+	//     value: nan
+	//   >
+	//   quantile: <
+	//     quantile: 0.99
+	//     value: nan
+	//   >
+	// >
+	//  label: <
+	//   name: "species"
 	//   value: "lithobates-catesbeianus"
 	// >
 	// summary: <

Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/expvar_test.go

@@ -19,9 +19,8 @@ import (
 	"sort"
 	"strings"
 
-	dto "github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_model/go"
-
 	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus"
+	dto "github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_model/go"
 )
 
 func ExampleExpvarCollector() {

Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/go_collector.go

@@ -1,6 +1,7 @@
 package prometheus
 
 import (
+	"fmt"
 	"runtime"
 	"runtime/debug"
 	"time"
@@ -9,6 +10,9 @@ import (
 type goCollector struct {
 	goroutines Gauge
 	gcDesc     *Desc
+
+	// metrics to describe and collect
+	metrics memStatsMetrics
 }
 
 // NewGoCollector returns a collector which exports metrics about the current
@@ -16,20 +20,216 @@ type goCollector struct {
 func NewGoCollector() *goCollector {
 	return &goCollector{
 		goroutines: NewGauge(GaugeOpts{
-			Name: "go_goroutines",
-			Help: "Number of goroutines that currently exist.",
+			Namespace: "go",
+			Name:      "goroutines",
+			Help:      "Number of goroutines that currently exist.",
 		}),
 		gcDesc: NewDesc(
 			"go_gc_duration_seconds",
 			"A summary of the GC invocation durations.",
 			nil, nil),
+		metrics: memStatsMetrics{
+			{
+				desc: NewDesc(
+					memstatNamespace("alloc_bytes"),
+					"Number of bytes allocated and still in use.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.Alloc) },
+				valType: GaugeValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("alloc_bytes_total"),
+					"Total number of bytes allocated, even if freed.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.TotalAlloc) },
+				valType: CounterValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("sys_bytes"),
+					"Number of bytes obtained by system. Sum of all system allocations.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.Sys) },
+				valType: GaugeValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("lookups_total"),
+					"Total number of pointer lookups.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.Lookups) },
+				valType: CounterValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("mallocs_total"),
+					"Total number of mallocs.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.Mallocs) },
+				valType: CounterValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("frees_total"),
+					"Total number of frees.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.Frees) },
+				valType: CounterValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("heap_alloc_bytes"),
+					"Number of heap bytes allocated and still in use.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.HeapAlloc) },
+				valType: GaugeValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("heap_sys_bytes"),
+					"Number of heap bytes obtained from system.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.HeapSys) },
+				valType: GaugeValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("heap_idle_bytes"),
+					"Number of heap bytes waiting to be used.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.HeapIdle) },
+				valType: GaugeValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("heap_inuse_bytes"),
+					"Number of heap bytes that are in use.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.HeapInuse) },
+				valType: GaugeValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("heap_released_bytes_total"),
+					"Total number of heap bytes released to OS.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.HeapReleased) },
+				valType: CounterValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("heap_objects"),
+					"Number of allocated objects.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.HeapObjects) },
+				valType: GaugeValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("stack_inuse_bytes"),
+					"Number of bytes in use by the stack allocator.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.StackInuse) },
+				valType: GaugeValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("stack_sys_bytes"),
+					"Number of bytes obtained from system for stack allocator.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.StackSys) },
+				valType: GaugeValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("mspan_inuse_bytes"),
+					"Number of bytes in use by mspan structures.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.MSpanInuse) },
+				valType: GaugeValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("mspan_sys_bytes"),
+					"Number of bytes used for mspan structures obtained from system.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.MSpanSys) },
+				valType: GaugeValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("mcache_inuse_bytes"),
+					"Number of bytes in use by mcache structures.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.MCacheInuse) },
+				valType: GaugeValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("mcache_sys_bytes"),
+					"Number of bytes used for mcache structures obtained from system.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.MCacheSys) },
+				valType: GaugeValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("buck_hash_sys_bytes"),
+					"Number of bytes used by the profiling bucket hash table.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.BuckHashSys) },
+				valType: GaugeValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("gc_sys_bytes"),
+					"Number of bytes used for garbage collection system metadata.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.GCSys) },
+				valType: GaugeValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("other_sys_bytes"),
+					"Number of bytes used for other system allocations.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.OtherSys) },
+				valType: GaugeValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("next_gc_bytes"),
+					"Number of heap bytes when next garbage collection will take place.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.NextGC) },
+				valType: GaugeValue,
+			}, {
+				desc: NewDesc(
+					memstatNamespace("last_gc_time_seconds"),
+					"Number of seconds since 1970 of last garbage collection.",
+					nil, nil,
+				),
+				eval:    func(ms *runtime.MemStats) float64 { return float64(ms.LastGC*10 ^ 9) },
+				valType: GaugeValue,
+			},
+		},
 	}
 }
 
+func memstatNamespace(s string) string {
+	return fmt.Sprintf("go_memstats_%s", s)
+}
+
 // Describe returns all descriptions of the collector.
 func (c *goCollector) Describe(ch chan<- *Desc) {
 	ch <- c.goroutines.Desc()
 	ch <- c.gcDesc
+
+	for _, i := range c.metrics {
+		ch <- i.desc
+	}
 }
 
 // Collect returns the current state of all metrics of the collector.
@@ -47,4 +247,17 @@ func (c *goCollector) Collect(ch chan<- Metric) {
 	}
 	quantiles[0.0] = stats.PauseQuantiles[0].Seconds()
 	ch <- MustNewConstSummary(c.gcDesc, uint64(stats.NumGC), float64(stats.PauseTotal.Seconds()), quantiles)
+
+	ms := &runtime.MemStats{}
+	runtime.ReadMemStats(ms)
+	for _, i := range c.metrics {
+		ch <- MustNewConstMetric(i.desc, i.valType, i.eval(ms))
+	}
+}
+
+// memStatsMetrics provide description, value, and value type for memstat metrics.
+type memStatsMetrics []struct {
+	desc    *Desc
+	eval    func(*runtime.MemStats) float64
+	valType ValueType
 }

Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/go_collector_test.go

@@ -50,6 +50,10 @@ func TestGoCollector(t *testing.T) {
 					t.Errorf("want 1 new goroutine, got %d", diff)
 				}
 
+				// GoCollector performs two sends per call.
+				// On line 27 we need to receive the second send
+				// to shut down cleanly.
+				<-ch
 				return
 			}
 		case <-time.After(1 * time.Second):

Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/histogram.go

@@ -21,8 +21,6 @@ import (
 	"sync/atomic"
 
 	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/golang/protobuf/proto"
-
-	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_golang/model"
 	dto "github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_model/go"
 )
 
@@ -49,6 +47,10 @@ type Histogram interface {
 	Observe(float64)
 }
 
+// bucketLabel is used for the label that defines the upper bound of a
+// bucket of a histogram ("le" -> "less or equal").
+const bucketLabel = "le"
+
 var (
 	// DefBuckets are the default Histogram buckets. The default buckets are
 	// tailored to broadly measure the response time (in seconds) of a
@@ -57,7 +59,7 @@ var (
 	DefBuckets = []float64{.005, .01, .025, .05, .1, .25, .5, 1, 2.5, 5, 10}
 
 	errBucketLabelNotAllowed = fmt.Errorf(
-		"%q is not allowed as label name in histograms", model.BucketLabel,
+		"%q is not allowed as label name in histograms", bucketLabel,
 	)
 )
 
@@ -171,12 +173,12 @@ func newHistogram(desc *Desc, opts HistogramOpts, labelValues ...string) Histogr
 	}
 
 	for _, n := range desc.variableLabels {
-		if n == model.BucketLabel {
+		if n == bucketLabel {
 			panic(errBucketLabelNotAllowed)
 		}
 	}
 	for _, lp := range desc.constLabelPairs {
-		if lp.GetName() == model.BucketLabel {
+		if lp.GetName() == bucketLabel {
 			panic(errBucketLabelNotAllowed)
 		}
 	}

Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/histogram_test.go

@@ -124,6 +124,10 @@ func BenchmarkHistogramWrite8(b *testing.B) {
 var testBuckets = []float64{-2, -1, -0.5, 0, 0.5, 1, 2, math.Inf(+1)}
 
 func TestHistogramConcurrency(t *testing.T) {
+	if testing.Short() {
+		t.Skip("Skipping test in short mode.")
+	}
+
 	rand.Seed(42)
 
 	it := func(n uint32) bool {
@@ -198,6 +202,10 @@ func TestHistogramConcurrency(t *testing.T) {
 }
 
 func TestHistogramVecConcurrency(t *testing.T) {
+	if testing.Short() {
+		t.Skip("Skipping test in short mode.")
+	}
+
 	rand.Seed(42)
 
 	objectives := make([]float64, 0, len(DefObjectives))

Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/http.go

@@ -14,6 +14,9 @@
 package prometheus
 
 import (
+	"bufio"
+	"io"
+	"net"
 	"net/http"
 	"strconv"
 	"strings"
@@ -141,7 +144,18 @@ func InstrumentHandlerFuncWithOpts(opts SummaryOpts, handlerFunc func(http.Respo
 			urlLen = len(r.URL.String())
 		}
 		go computeApproximateRequestSize(r, out, urlLen)
-		handlerFunc(delegate, r)
+
+		_, cn := w.(http.CloseNotifier)
+		_, fl := w.(http.Flusher)
+		_, hj := w.(http.Hijacker)
+		_, rf := w.(io.ReaderFrom)
+		var rw http.ResponseWriter
+		if cn && fl && hj && rf {
+			rw = &fancyResponseWriterDelegator{delegate}
+		} else {
+			rw = delegate
+		}
+		handlerFunc(rw, r)
 
 		elapsed := float64(time.Since(now)) / float64(time.Microsecond)
 
@@ -178,7 +192,7 @@ type responseWriterDelegator struct {
 
 	handler, method string
 	status          int
-	written         int
+	written         int64
 	wroteHeader     bool
 }
 
@@ -193,7 +207,32 @@ func (r *responseWriterDelegator) Write(b []byte) (int, error) {
 		r.WriteHeader(http.StatusOK)
 	}
 	n, err := r.ResponseWriter.Write(b)
-	r.written += n
+	r.written += int64(n)
+	return n, err
+}
+
+type fancyResponseWriterDelegator struct {
+	*responseWriterDelegator
+}
+
+func (f *fancyResponseWriterDelegator) CloseNotify() <-chan bool {
+	return f.ResponseWriter.(http.CloseNotifier).CloseNotify()
+}
+
+func (f *fancyResponseWriterDelegator) Flush() {
+	f.ResponseWriter.(http.Flusher).Flush()
+}
+
+func (f *fancyResponseWriterDelegator) Hijack() (net.Conn, *bufio.ReadWriter, error) {
+	return f.ResponseWriter.(http.Hijacker).Hijack()
+}
+
+func (f *fancyResponseWriterDelegator) ReadFrom(r io.Reader) (int64, error) {
+	if !f.wroteHeader {
+		f.WriteHeader(http.StatusOK)
+	}
+	n, err := f.ResponseWriter.(io.ReaderFrom).ReadFrom(r)
+	f.written += n
 	return n, err
 }
 

Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/metric.go

@@ -19,6 +19,8 @@ import (
 	dto "github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_model/go"
 )
 
+const separatorByte byte = 255
+
 // A Metric models a single sample value with its meta data being exported to
 // Prometheus. Implementers of Metric in this package inclued Gauge, Counter,
 // Untyped, and Summary. Users can implement their own Metric types, but that

Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/registry.go

@@ -33,13 +33,9 @@ import (
 	"strings"
 	"sync"
 
-	"github.com/coreos/etcd/Godeps/_workspace/src/bitbucket.org/ww/goautoneg"
 	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/golang/protobuf/proto"
-
 	dto "github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_model/go"
-
-	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_golang/model"
-	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_golang/text"
+	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/common/expfmt"
 )
 
 var (
@@ -170,6 +166,11 @@ func Unregister(c Collector) bool {
 // checks are performed, but no further consistency checks (which would require
 // knowledge of a metric descriptor).
 //
+// Sorting concerns: The caller is responsible for sorting the label pairs in
+// each metric. However, the order of metrics will be sorted by the registry as
+// it is required anyway after merging with the metric families collected
+// conventionally.
+//
 // The function must be callable at any time and concurrently.
 func SetMetricFamilyInjectionHook(hook func() []*dto.MetricFamily) {
 	defRegistry.metricFamilyInjectionHook = hook
@@ -341,7 +342,7 @@ func (r *registry) Push(job, instance, pushURL, method string) error {
 	}
 	buf := r.getBuf()
 	defer r.giveBuf(buf)
-	if _, err := r.writePB(buf, text.WriteProtoDelimited); err != nil {
+	if err := r.writePB(expfmt.NewEncoder(buf, expfmt.FmtProtoDelim)); err != nil {
 		if r.panicOnCollectError {
 			panic(err)
 		}
@@ -364,11 +365,11 @@ func (r *registry) Push(job, instance, pushURL, method string) error {
 }
 
 func (r *registry) ServeHTTP(w http.ResponseWriter, req *http.Request) {
-	enc, contentType := chooseEncoder(req)
+	contentType := expfmt.Negotiate(req.Header)
 	buf := r.getBuf()
 	defer r.giveBuf(buf)
 	writer, encoding := decorateWriter(req, buf)
-	if _, err := r.writePB(writer, enc); err != nil {
+	if err := r.writePB(expfmt.NewEncoder(writer, contentType)); err != nil {
 		if r.panicOnCollectError {
 			panic(err)
 		}
@@ -379,7 +380,7 @@ func (r *registry) ServeHTTP(w http.ResponseWriter, req *http.Request) {
 		closer.Close()
 	}
 	header := w.Header()
-	header.Set(contentTypeHeader, contentType)
+	header.Set(contentTypeHeader, string(contentType))
 	header.Set(contentLengthHeader, fmt.Sprint(buf.Len()))
 	if encoding != "" {
 		header.Set(contentEncodingHeader, encoding)
@@ -387,7 +388,7 @@ func (r *registry) ServeHTTP(w http.ResponseWriter, req *http.Request) {
 	w.Write(buf.Bytes())
 }
 
-func (r *registry) writePB(w io.Writer, writeEncoded encoder) (int, error) {
+func (r *registry) writePB(encoder expfmt.Encoder) error {
 	var metricHashes map[uint64]struct{}
 	if r.collectChecksEnabled {
 		metricHashes = make(map[uint64]struct{})
@@ -439,7 +440,7 @@ func (r *registry) writePB(w io.Writer, writeEncoded encoder) (int, error) {
 			// TODO: Consider different means of error reporting so
 			// that a single erroneous metric could be skipped
 			// instead of blowing up the whole collection.
-			return 0, fmt.Errorf("error collecting metric %v: %s", desc, err)
+			return fmt.Errorf("error collecting metric %v: %s", desc, err)
 		}
 		switch {
 		case metricFamily.Type != nil:
@@ -455,11 +456,11 @@ func (r *registry) writePB(w io.Writer, writeEncoded encoder) (int, error) {
 		case dtoMetric.Histogram != nil:
 			metricFamily.Type = dto.MetricType_HISTOGRAM.Enum()
 		default:
-			return 0, fmt.Errorf("empty metric collected: %s", dtoMetric)
+			return fmt.Errorf("empty metric collected: %s", dtoMetric)
 		}
 		if r.collectChecksEnabled {
 			if err := r.checkConsistency(metricFamily, dtoMetric, desc, metricHashes); err != nil {
-				return 0, err
+				return err
 			}
 		}
 		metricFamily.Metric = append(metricFamily.Metric, dtoMetric)
@@ -473,7 +474,7 @@ func (r *registry) writePB(w io.Writer, writeEncoded encoder) (int, error) {
 				if r.collectChecksEnabled {
 					for _, m := range mf.Metric {
 						if err := r.checkConsistency(mf, m, nil, metricHashes); err != nil {
-							return 0, err
+							return err
 						}
 					}
 				}
@@ -482,7 +483,7 @@ func (r *registry) writePB(w io.Writer, writeEncoded encoder) (int, error) {
 			for _, m := range mf.Metric {
 				if r.collectChecksEnabled {
 					if err := r.checkConsistency(existingMF, m, nil, metricHashes); err != nil {
-						return 0, err
+						return err
 					}
 				}
 				existingMF.Metric = append(existingMF.Metric, m)
@@ -503,15 +504,12 @@ func (r *registry) writePB(w io.Writer, writeEncoded encoder) (int, error) {
 	}
 	sort.Strings(names)
 
-	var written int
 	for _, name := range names {
-		w, err := writeEncoded(w, metricFamiliesByName[name])
-		written += w
-		if err != nil {
-			return written, err
+		if err := encoder.Encode(metricFamiliesByName[name]); err != nil {
+			return err
 		}
 	}
-	return written, nil
+	return nil
 }
 
 func (r *registry) checkConsistency(metricFamily *dto.MetricFamily, dtoMetric *dto.Metric, desc *Desc, metricHashes map[uint64]struct{}) error {
@@ -520,10 +518,11 @@ func (r *registry) checkConsistency(metricFamily *dto.MetricFamily, dtoMetric *d
 	if metricFamily.GetType() == dto.MetricType_GAUGE && dtoMetric.Gauge == nil ||
 		metricFamily.GetType() == dto.MetricType_COUNTER && dtoMetric.Counter == nil ||
 		metricFamily.GetType() == dto.MetricType_SUMMARY && dtoMetric.Summary == nil ||
+		metricFamily.GetType() == dto.MetricType_HISTOGRAM && dtoMetric.Histogram == nil ||
 		metricFamily.GetType() == dto.MetricType_UNTYPED && dtoMetric.Untyped == nil {
 		return fmt.Errorf(
-			"collected metric %q is not a %s",
-			dtoMetric, metricFamily.Type,
+			"collected metric %s %s is not a %s",
+			metricFamily.GetName(), dtoMetric, metricFamily.GetType(),
 		)
 	}
 
@@ -531,19 +530,24 @@ func (r *registry) checkConsistency(metricFamily *dto.MetricFamily, dtoMetric *d
 	h := fnv.New64a()
 	var buf bytes.Buffer
 	buf.WriteString(metricFamily.GetName())
-	buf.WriteByte(model.SeparatorByte)
+	buf.WriteByte(separatorByte)
 	h.Write(buf.Bytes())
+	// Make sure label pairs are sorted. We depend on it for the consistency
+	// check. Label pairs must be sorted by contract. But the point of this
+	// method is to check for contract violations. So we better do the sort
+	// now.
+	sort.Sort(LabelPairSorter(dtoMetric.Label))
 	for _, lp := range dtoMetric.Label {
 		buf.Reset()
 		buf.WriteString(lp.GetValue())
-		buf.WriteByte(model.SeparatorByte)
+		buf.WriteByte(separatorByte)
 		h.Write(buf.Bytes())
 	}
 	metricHash := h.Sum64()
 	if _, exists := metricHashes[metricHash]; exists {
 		return fmt.Errorf(
-			"collected metric %q was collected before with the same name and label values",
-			dtoMetric,
+			"collected metric %s %s was collected before with the same name and label values",
+			metricFamily.GetName(), dtoMetric,
 		)
 	}
 	metricHashes[metricHash] = struct{}{}
@@ -555,14 +559,14 @@ func (r *registry) checkConsistency(metricFamily *dto.MetricFamily, dtoMetric *d
 	// Desc consistency with metric family.
 	if metricFamily.GetName() != desc.fqName {
 		return fmt.Errorf(
-			"collected metric %q has name %q but should have %q",
-			dtoMetric, metricFamily.GetName(), desc.fqName,
+			"collected metric %s %s has name %q but should have %q",
+			metricFamily.GetName(), dtoMetric, metricFamily.GetName(), desc.fqName,
 		)
 	}
 	if metricFamily.GetHelp() != desc.help {
 		return fmt.Errorf(
-			"collected metric %q has help %q but should have %q",
-			dtoMetric, metricFamily.GetHelp(), desc.help,
+			"collected metric %s %s has help %q but should have %q",
+			metricFamily.GetName(), dtoMetric, metricFamily.GetHelp(), desc.help,
 		)
 	}
 
@@ -576,8 +580,8 @@ func (r *registry) checkConsistency(metricFamily *dto.MetricFamily, dtoMetric *d
 	}
 	if len(lpsFromDesc) != len(dtoMetric.Label) {
 		return fmt.Errorf(
-			"labels in collected metric %q are inconsistent with descriptor %s",
-			dtoMetric, desc,
+			"labels in collected metric %s %s are inconsistent with descriptor %s",
+			metricFamily.GetName(), dtoMetric, desc,
 		)
 	}
 	sort.Sort(LabelPairSorter(lpsFromDesc))
@@ -586,8 +590,8 @@ func (r *registry) checkConsistency(metricFamily *dto.MetricFamily, dtoMetric *d
 		if lpFromDesc.GetName() != lpFromMetric.GetName() ||
 			lpFromDesc.Value != nil && lpFromDesc.GetValue() != lpFromMetric.GetValue() {
 			return fmt.Errorf(
-				"labels in collected metric %q are inconsistent with descriptor %s",
-				dtoMetric, desc,
+				"labels in collected metric %s %s are inconsistent with descriptor %s",
+				metricFamily.GetName(), dtoMetric, desc,
 			)
 		}
 	}
@@ -597,7 +601,10 @@ func (r *registry) checkConsistency(metricFamily *dto.MetricFamily, dtoMetric *d
 
 	// Is the desc registered?
 	if _, exist := r.descIDs[desc.id]; !exist {
-		return fmt.Errorf("collected metric %q with unregistered descriptor %s", dtoMetric, desc)
+		return fmt.Errorf(
+			"collected metric %s %s with unregistered descriptor %s",
+			metricFamily.GetName(), dtoMetric, desc,
+		)
 	}
 
 	return nil
@@ -672,34 +679,6 @@ func newDefaultRegistry() *registry {
 	return r
 }
 
-func chooseEncoder(req *http.Request) (encoder, string) {
-	accepts := goautoneg.ParseAccept(req.Header.Get(acceptHeader))
-	for _, accept := range accepts {
-		switch {
-		case accept.Type == "application" &&
-			accept.SubType == "vnd.google.protobuf" &&
-			accept.Params["proto"] == "io.prometheus.client.MetricFamily":
-			switch accept.Params["encoding"] {
-			case "delimited":
-				return text.WriteProtoDelimited, DelimitedTelemetryContentType
-			case "text":
-				return text.WriteProtoText, ProtoTextTelemetryContentType
-			case "compact-text":
-				return text.WriteProtoCompactText, ProtoCompactTextTelemetryContentType
-			default:
-				continue
-			}
-		case accept.Type == "text" &&
-			accept.SubType == "plain" &&
-			(accept.Params["version"] == "0.0.4" || accept.Params["version"] == ""):
-			return text.MetricFamilyToText, TextTelemetryContentType
-		default:
-			continue
-		}
-	}
-	return text.MetricFamilyToText, TextTelemetryContentType
-}
-
 // decorateWriter wraps a writer to handle gzip compression if requested.  It
 // returns the decorated writer and the appropriate "Content-Encoding" header
 // (which is empty if no compression is enabled).

Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/registry_test.go

@@ -68,14 +68,14 @@ func testHandler(t testing.TB) {
 		Metric: []*dto.Metric{
 			{
 				Label: []*dto.LabelPair{
-					{
-						Name:  proto.String("externallabelname"),
-						Value: proto.String("externalval1"),
-					},
 					{
 						Name:  proto.String("externalconstname"),
 						Value: proto.String("externalconstvalue"),
 					},
+					{
+						Name:  proto.String("externallabelname"),
+						Value: proto.String("externalval1"),
+					},
 				},
 				Counter: &dto.Counter{
 					Value: proto.Float64(1),
@@ -100,27 +100,27 @@ func testHandler(t testing.TB) {
 	externalMetricFamilyAsBytes := externalBuf.Bytes()
 	externalMetricFamilyAsText := []byte(`# HELP externalname externaldocstring
 # TYPE externalname counter
-externalname{externallabelname="externalval1",externalconstname="externalconstvalue"} 1
+externalname{externalconstname="externalconstvalue",externallabelname="externalval1"} 1
 `)
 	externalMetricFamilyAsProtoText := []byte(`name: "externalname"
 help: "externaldocstring"
 type: COUNTER
 metric: <
-  label: <
-    name: "externallabelname"
-    value: "externalval1"
-  >
   label: <
     name: "externalconstname"
     value: "externalconstvalue"
   >
+  label: <
+    name: "externallabelname"
+    value: "externalval1"
+  >
   counter: <
     value: 1
   >
 >
 
 `)
-	externalMetricFamilyAsProtoCompactText := []byte(`name:"externalname" help:"externaldocstring" type:COUNTER metric:<label:<name:"externallabelname" value:"externalval1" > label:<name:"externalconstname" value:"externalconstvalue" > counter:<value:1 > > 
+	externalMetricFamilyAsProtoCompactText := []byte(`name:"externalname" help:"externaldocstring" type:COUNTER metric:<label:<name:"externalconstname" value:"externalconstvalue" > label:<name:"externallabelname" value:"externalval1" > counter:<value:1 > > 
 `)
 
 	expectedMetricFamily := &dto.MetricFamily{

Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/summary.go

@@ -23,12 +23,13 @@ import (
 
 	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/beorn7/perks/quantile"
 	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/golang/protobuf/proto"
-
 	dto "github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_model/go"
-
-	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_golang/model"
 )
 
+// quantileLabel is used for the label that defines the quantile in a
+// summary.
+const quantileLabel = "quantile"
+
 // A Summary captures individual observations from an event or sample stream and
 // summarizes them in a manner similar to traditional summary statistics: 1. sum
 // of observations, 2. observation count, 3. rank estimations.
@@ -57,7 +58,7 @@ var (
 	DefObjectives = map[float64]float64{0.5: 0.05, 0.9: 0.01, 0.99: 0.001}
 
 	errQuantileLabelNotAllowed = fmt.Errorf(
-		"%q is not allowed as label name in summaries", model.QuantileLabel,
+		"%q is not allowed as label name in summaries", quantileLabel,
 	)
 )
 
@@ -112,7 +113,9 @@ type SummaryOpts struct {
 	ConstLabels Labels
 
 	// Objectives defines the quantile rank estimates with their respective
-	// absolute error. The default value is DefObjectives.
+	// absolute error. If Objectives[q] = e, then the value reported
+	// for q will be the φ-quantile value for some φ between q-e and q+e.
+	// The default value is DefObjectives.
 	Objectives map[float64]float64
 
 	// MaxAge defines the duration for which an observation stays relevant
@@ -170,12 +173,12 @@ func newSummary(desc *Desc, opts SummaryOpts, labelValues ...string) Summary {
 	}
 
 	for _, n := range desc.variableLabels {
-		if n == model.QuantileLabel {
+		if n == quantileLabel {
 			panic(errQuantileLabelNotAllowed)
 		}
 	}
 	for _, lp := range desc.constLabelPairs {
-		if lp.GetName() == model.QuantileLabel {
+		if lp.GetName() == quantileLabel {
 			panic(errQuantileLabelNotAllowed)
 		}
 	}

Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/untyped.go

@@ -21,7 +21,7 @@ import "hash/fnv"
 // An Untyped metric works the same as a Gauge. The only difference is that to
 // no type information is implied.
 //
-// To create Gauge instances, use NewUntyped.
+// To create Untyped instances, use NewUntyped.
 type Untyped interface {
 	Metric
 	Collector

Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/value.go

@@ -20,9 +20,8 @@ import (
 	"sort"
 	"sync/atomic"
 
-	dto "github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_model/go"
-
 	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/golang/protobuf/proto"
+	dto "github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_model/go"
 )
 
 // ValueType is an enumeration of metric types that represent a simple value.

Godeps/_workspace/src/github.com/prometheus/client_golang/prometheus/vec.go

@@ -58,6 +58,11 @@ func (m *MetricVec) Collect(ch chan<- Metric) {
 // GetMetricWithLabelValues returns the Metric for the given slice of label
 // values (same order as the VariableLabels in Desc). If that combination of
 // label values is accessed for the first time, a new Metric is created.
+//
+// It is possible to call this method without using the returned Metric to only
+// create the new Metric but leave it at its start value (e.g. a Summary or
+// Histogram without any observations). See also the SummaryVec example.
+//
 // Keeping the Metric for later use is possible (and should be considered if
 // performance is critical), but keep in mind that Reset, DeleteLabelValues and
 // Delete can be used to delete the Metric from the MetricVec. In that case, the
@@ -87,8 +92,9 @@ func (m *MetricVec) GetMetricWithLabelValues(lvs ...string) (Metric, error) {
 
 // GetMetricWith returns the Metric for the given Labels map (the label names
 // must match those of the VariableLabels in Desc). If that label map is
-// accessed for the first time, a new Metric is created. Implications of keeping
-// the Metric are the same as for GetMetricWithLabelValues.
+// accessed for the first time, a new Metric is created. Implications of
+// creating a Metric without using it and keeping the Metric for later use are
+// the same as for GetMetricWithLabelValues.
 //
 // An error is returned if the number and names of the Labels are inconsistent
 // with those of the VariableLabels in Desc.

Godeps/_workspace/src/github.com/prometheus/client_golang/text/proto.go

@@ -1,43 +0,0 @@
-// Copyright 2014 The Prometheus Authors
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-
-package text
-
-import (
-	"fmt"
-	"io"
-
-	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/golang/protobuf/proto"
-	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/matttproud/golang_protobuf_extensions/pbutil"
-
-	dto "github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_model/go"
-)
-
-// WriteProtoDelimited writes the MetricFamily to the writer in delimited
-// protobuf format and returns the number of bytes written and any error
-// encountered.
-func WriteProtoDelimited(w io.Writer, p *dto.MetricFamily) (int, error) {
-	return pbutil.WriteDelimited(w, p)
-}
-
-// WriteProtoText writes the MetricFamily to the writer in text format and
-// returns the number of bytes written and any error encountered.
-func WriteProtoText(w io.Writer, p *dto.MetricFamily) (int, error) {
-	return fmt.Fprintf(w, "%s\n", proto.MarshalTextString(p))
-}
-
-// WriteProtoCompactText writes the MetricFamily to the writer in compact text
-// format and returns the number of bytes written and any error encountered.
-func WriteProtoCompactText(w io.Writer, p *dto.MetricFamily) (int, error) {
-	return fmt.Fprintf(w, "%s\n", p)
-}

Godeps/_workspace/src/github.com/prometheus/common/expfmt/bench_test.go

@@ -11,19 +11,21 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
-package text
+package expfmt
 
 import (
 	"bytes"
 	"compress/gzip"
-	dto "github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_model/go"
 	"io"
 	"io/ioutil"
 	"testing"
 
 	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/matttproud/golang_protobuf_extensions/pbutil"
+	dto "github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_model/go"
 )
 
+var parser TextParser
+
 // Benchmarks to show how much penalty text format parsing actually inflicts.
 //
 // Example results on Linux 3.13.0, Intel(R) Core(TM) i7-4700MQ CPU @ 2.40GHz, go1.4.

Godeps/_workspace/src/github.com/prometheus/common/expfmt/decode.go

@@ -0,0 +1,410 @@
+// Copyright 2015 The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package expfmt
+
+import (
+	"fmt"
+	"io"
+	"math"
+	"mime"
+	"net/http"
+
+	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/matttproud/golang_protobuf_extensions/pbutil"
+	dto "github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_model/go"
+	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/common/model"
+)
+
+// Decoder types decode an input stream into metric families.
+type Decoder interface {
+	Decode(*dto.MetricFamily) error
+}
+
+type DecodeOptions struct {
+	// Timestamp is added to each value from the stream that has no explicit timestamp set.
+	Timestamp model.Time
+}
+
+// ResponseFormat extracts the correct format from a HTTP response header.
+// If no matching format can be found FormatUnknown is returned.
+func ResponseFormat(h http.Header) Format {
+	ct := h.Get(hdrContentType)
+
+	mediatype, params, err := mime.ParseMediaType(ct)
+	if err != nil {
+		return FmtUnknown
+	}
+
+	const (
+		textType = "text/plain"
+		jsonType = "application/json"
+	)
+
+	switch mediatype {
+	case ProtoType:
+		if p, ok := params["proto"]; ok && p != ProtoProtocol {
+			return FmtUnknown
+		}
+		if e, ok := params["encoding"]; ok && e != "delimited" {
+			return FmtUnknown
+		}
+		return FmtProtoDelim
+
+	case textType:
+		if v, ok := params["version"]; ok && v != TextVersion {
+			return FmtUnknown
+		}
+		return FmtText
+
+	case jsonType:
+		var prometheusAPIVersion string
+
+		if params["schema"] == "prometheus/telemetry" && params["version"] != "" {
+			prometheusAPIVersion = params["version"]
+		} else {
+			prometheusAPIVersion = h.Get("X-Prometheus-API-Version")
+		}
+
+		switch prometheusAPIVersion {
+		case "0.0.2", "":
+			return fmtJSON2
+		default:
+			return FmtUnknown
+		}
+	}
+
+	return FmtUnknown
+}
+
+// NewDecoder returns a new decoder based on the given input format.
+// If the input format does not imply otherwise, a text format decoder is returned.
+func NewDecoder(r io.Reader, format Format) Decoder {
+	switch format {
+	case FmtProtoDelim:
+		return &protoDecoder{r: r}
+	case fmtJSON2:
+		return newJSON2Decoder(r)
+	}
+	return &textDecoder{r: r}
+}
+
+// protoDecoder implements the Decoder interface for protocol buffers.
+type protoDecoder struct {
+	r io.Reader
+}
+
+// Decode implements the Decoder interface.
+func (d *protoDecoder) Decode(v *dto.MetricFamily) error {
+	_, err := pbutil.ReadDelimited(d.r, v)
+	return err
+}
+
+// textDecoder implements the Decoder interface for the text protcol.
+type textDecoder struct {
+	r    io.Reader
+	p    TextParser
+	fams []*dto.MetricFamily
+}
+
+// Decode implements the Decoder interface.
+func (d *textDecoder) Decode(v *dto.MetricFamily) error {
+	// TODO(fabxc): Wrap this as a line reader to make streaming safer.
+	if len(d.fams) == 0 {
+		// No cached metric families, read everything and parse metrics.
+		fams, err := d.p.TextToMetricFamilies(d.r)
+		if err != nil {
+			return err
+		}
+		if len(fams) == 0 {
+			return io.EOF
+		}
+		d.fams = make([]*dto.MetricFamily, 0, len(fams))
+		for _, f := range fams {
+			d.fams = append(d.fams, f)
+		}
+	}
+
+	*v = *d.fams[0]
+	d.fams = d.fams[1:]
+
+	return nil
+}
+
+type SampleDecoder struct {
+	Dec  Decoder
+	Opts *DecodeOptions
+
+	f dto.MetricFamily
+}
+
+func (sd *SampleDecoder) Decode(s *model.Vector) error {
+	if err := sd.Dec.Decode(&sd.f); err != nil {
+		return err
+	}
+	*s = extractSamples(&sd.f, sd.Opts)
+	return nil
+}
+
+// Extract samples builds a slice of samples from the provided metric families.
+func ExtractSamples(o *DecodeOptions, fams ...*dto.MetricFamily) model.Vector {
+	var all model.Vector
+	for _, f := range fams {
+		all = append(all, extractSamples(f, o)...)
+	}
+	return all
+}
+
+func extractSamples(f *dto.MetricFamily, o *DecodeOptions) model.Vector {
+	switch f.GetType() {
+	case dto.MetricType_COUNTER:
+		return extractCounter(o, f)
+	case dto.MetricType_GAUGE:
+		return extractGauge(o, f)
+	case dto.MetricType_SUMMARY:
+		return extractSummary(o, f)
+	case dto.MetricType_UNTYPED:
+		return extractUntyped(o, f)
+	case dto.MetricType_HISTOGRAM:
+		return extractHistogram(o, f)
+	}
+	panic("expfmt.extractSamples: unknown metric family type")
+}
+
+func extractCounter(o *DecodeOptions, f *dto.MetricFamily) model.Vector {
+	samples := make(model.Vector, 0, len(f.Metric))
+
+	for _, m := range f.Metric {
+		if m.Counter == nil {
+			continue
+		}
+
+		lset := make(model.LabelSet, len(m.Label)+1)
+		for _, p := range m.Label {
+			lset[model.LabelName(p.GetName())] = model.LabelValue(p.GetValue())
+		}
+		lset[model.MetricNameLabel] = model.LabelValue(f.GetName())
+
+		smpl := &model.Sample{
+			Metric: model.Metric(lset),
+			Value:  model.SampleValue(m.Counter.GetValue()),
+		}
+
+		if m.TimestampMs != nil {
+			smpl.Timestamp = model.TimeFromUnixNano(*m.TimestampMs * 1000000)
+		} else {
+			smpl.Timestamp = o.Timestamp
+		}
+
+		samples = append(samples, smpl)
+	}
+
+	return samples
+}
+
+func extractGauge(o *DecodeOptions, f *dto.MetricFamily) model.Vector {
+	samples := make(model.Vector, 0, len(f.Metric))
+
+	for _, m := range f.Metric {
+		if m.Gauge == nil {
+			continue
+		}
+
+		lset := make(model.LabelSet, len(m.Label)+1)
+		for _, p := range m.Label {
+			lset[model.LabelName(p.GetName())] = model.LabelValue(p.GetValue())
+		}
+		lset[model.MetricNameLabel] = model.LabelValue(f.GetName())
+
+		smpl := &model.Sample{
+			Metric: model.Metric(lset),
+			Value:  model.SampleValue(m.Gauge.GetValue()),
+		}
+
+		if m.TimestampMs != nil {
+			smpl.Timestamp = model.TimeFromUnixNano(*m.TimestampMs * 1000000)
+		} else {
+			smpl.Timestamp = o.Timestamp
+		}
+
+		samples = append(samples, smpl)
+	}
+
+	return samples
+}
+
+func extractUntyped(o *DecodeOptions, f *dto.MetricFamily) model.Vector {
+	samples := make(model.Vector, 0, len(f.Metric))
+
+	for _, m := range f.Metric {
+		if m.Untyped == nil {
+			continue
+		}
+
+		lset := make(model.LabelSet, len(m.Label)+1)
+		for _, p := range m.Label {
+			lset[model.LabelName(p.GetName())] = model.LabelValue(p.GetValue())
+		}
+		lset[model.MetricNameLabel] = model.LabelValue(f.GetName())
+
+		smpl := &model.Sample{
+			Metric: model.Metric(lset),
+			Value:  model.SampleValue(m.Untyped.GetValue()),
+		}
+
+		if m.TimestampMs != nil {
+			smpl.Timestamp = model.TimeFromUnixNano(*m.TimestampMs * 1000000)
+		} else {
+			smpl.Timestamp = o.Timestamp
+		}
+
+		samples = append(samples, smpl)
+	}
+
+	return samples
+}
+
+func extractSummary(o *DecodeOptions, f *dto.MetricFamily) model.Vector {
+	samples := make(model.Vector, 0, len(f.Metric))
+
+	for _, m := range f.Metric {
+		if m.Summary == nil {
+			continue
+		}
+
+		timestamp := o.Timestamp
+		if m.TimestampMs != nil {
+			timestamp = model.TimeFromUnixNano(*m.TimestampMs * 1000000)
+		}
+
+		for _, q := range m.Summary.Quantile {
+			lset := make(model.LabelSet, len(m.Label)+2)
+			for _, p := range m.Label {
+				lset[model.LabelName(p.GetName())] = model.LabelValue(p.GetValue())
+			}
+			// BUG(matt): Update other names to "quantile".
+			lset[model.LabelName(model.QuantileLabel)] = model.LabelValue(fmt.Sprint(q.GetQuantile()))
+			lset[model.MetricNameLabel] = model.LabelValue(f.GetName())
+
+			samples = append(samples, &model.Sample{
+				Metric:    model.Metric(lset),
+				Value:     model.SampleValue(q.GetValue()),
+				Timestamp: timestamp,
+			})
+		}
+
+		lset := make(model.LabelSet, len(m.Label)+1)
+		for _, p := range m.Label {
+			lset[model.LabelName(p.GetName())] = model.LabelValue(p.GetValue())
+		}
+		lset[model.MetricNameLabel] = model.LabelValue(f.GetName() + "_sum")
+
+		samples = append(samples, &model.Sample{
+			Metric:    model.Metric(lset),
+			Value:     model.SampleValue(m.Summary.GetSampleSum()),
+			Timestamp: timestamp,
+		})
+
+		lset = make(model.LabelSet, len(m.Label)+1)
+		for _, p := range m.Label {
+			lset[model.LabelName(p.GetName())] = model.LabelValue(p.GetValue())
+		}
+		lset[model.MetricNameLabel] = model.LabelValue(f.GetName() + "_count")
+
+		samples = append(samples, &model.Sample{
+			Metric:    model.Metric(lset),
+			Value:     model.SampleValue(m.Summary.GetSampleCount()),
+			Timestamp: timestamp,
+		})
+	}
+
+	return samples
+}
+
+func extractHistogram(o *DecodeOptions, f *dto.MetricFamily) model.Vector {
+	samples := make(model.Vector, 0, len(f.Metric))
+
+	for _, m := range f.Metric {
+		if m.Histogram == nil {
+			continue
+		}
+
+		timestamp := o.Timestamp
+		if m.TimestampMs != nil {
+			timestamp = model.TimeFromUnixNano(*m.TimestampMs * 1000000)
+		}
+
+		infSeen := false
+
+		for _, q := range m.Histogram.Bucket {
+			lset := make(model.LabelSet, len(m.Label)+2)
+			for _, p := range m.Label {
+				lset[model.LabelName(p.GetName())] = model.LabelValue(p.GetValue())
+			}
+			lset[model.LabelName(model.BucketLabel)] = model.LabelValue(fmt.Sprint(q.GetUpperBound()))
+			lset[model.MetricNameLabel] = model.LabelValue(f.GetName() + "_bucket")
+
+			if math.IsInf(q.GetUpperBound(), +1) {
+				infSeen = true
+			}
+
+			samples = append(samples, &model.Sample{
+				Metric:    model.Metric(lset),
+				Value:     model.SampleValue(q.GetCumulativeCount()),
+				Timestamp: timestamp,
+			})
+		}
+
+		lset := make(model.LabelSet, len(m.Label)+1)
+		for _, p := range m.Label {
+			lset[model.LabelName(p.GetName())] = model.LabelValue(p.GetValue())
+		}
+		lset[model.MetricNameLabel] = model.LabelValue(f.GetName() + "_sum")
+
+		samples = append(samples, &model.Sample{
+			Metric:    model.Metric(lset),
+			Value:     model.SampleValue(m.Histogram.GetSampleSum()),
+			Timestamp: timestamp,
+		})
+
+		lset = make(model.LabelSet, len(m.Label)+1)
+		for _, p := range m.Label {
+			lset[model.LabelName(p.GetName())] = model.LabelValue(p.GetValue())
+		}
+		lset[model.MetricNameLabel] = model.LabelValue(f.GetName() + "_count")
+
+		count := &model.Sample{
+			Metric:    model.Metric(lset),
+			Value:     model.SampleValue(m.Histogram.GetSampleCount()),
+			Timestamp: timestamp,
+		}
+		samples = append(samples, count)
+
+		if !infSeen {
+			// Append an infinity bucket sample.
+			lset := make(model.LabelSet, len(m.Label)+2)
+			for _, p := range m.Label {
+				lset[model.LabelName(p.GetName())] = model.LabelValue(p.GetValue())
+			}
+			lset[model.LabelName(model.BucketLabel)] = model.LabelValue("+Inf")
+			lset[model.MetricNameLabel] = model.LabelValue(f.GetName() + "_bucket")
+
+			samples = append(samples, &model.Sample{
+				Metric:    model.Metric(lset),
+				Value:     count.Value,
+				Timestamp: timestamp,
+			})
+		}
+	}
+
+	return samples
+}

Godeps/_workspace/src/github.com/prometheus/common/expfmt/decode_test.go

@@ -0,0 +1,356 @@
+// Copyright 2015 The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package expfmt
+
+import (
+	"io"
+	"net/http"
+	"reflect"
+	"sort"
+	"strings"
+	"testing"
+
+	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/common/model"
+)
+
+func TestTextDecoder(t *testing.T) {
+	var (
+		ts = model.Now()
+		in = `
+# Only a quite simple scenario with two metric families.
+# More complicated tests of the parser itself can be found in the text package.
+# TYPE mf2 counter
+mf2 3
+mf1{label="value1"} -3.14 123456
+mf1{label="value2"} 42
+mf2 4
+`
+		out = model.Vector{
+			&model.Sample{
+				Metric: model.Metric{
+					model.MetricNameLabel: "mf1",
+					"label":               "value1",
+				},
+				Value:     -3.14,
+				Timestamp: 123456,
+			},
+			&model.Sample{
+				Metric: model.Metric{
+					model.MetricNameLabel: "mf1",
+					"label":               "value2",
+				},
+				Value:     42,
+				Timestamp: ts,
+			},
+			&model.Sample{
+				Metric: model.Metric{
+					model.MetricNameLabel: "mf2",
+				},
+				Value:     3,
+				Timestamp: ts,
+			},
+			&model.Sample{
+				Metric: model.Metric{
+					model.MetricNameLabel: "mf2",
+				},
+				Value:     4,
+				Timestamp: ts,
+			},
+		}
+	)
+
+	dec := &SampleDecoder{
+		Dec: &textDecoder{r: strings.NewReader(in)},
+		Opts: &DecodeOptions{
+			Timestamp: ts,
+		},
+	}
+	var all model.Vector
+	for {
+		var smpls model.Vector
+		err := dec.Decode(&smpls)
+		if err == io.EOF {
+			break
+		}
+		if err != nil {
+			t.Fatal(err)
+		}
+		all = append(all, smpls...)
+	}
+	sort.Sort(all)
+	sort.Sort(out)
+	if !reflect.DeepEqual(all, out) {
+		t.Fatalf("output does not match")
+	}
+}
+
+func TestProtoDecoder(t *testing.T) {
+
+	var testTime = model.Now()
+
+	scenarios := []struct {
+		in       string
+		expected model.Vector
+	}{
+		{
+			in: "",
+		},
+		{
+			in: "\x8f\x01\n\rrequest_count\x12\x12Number of requests\x18\x00\"0\n#\n\x0fsome_label_name\x12\x10some_label_value\x1a\t\t\x00\x00\x00\x00\x00\x00E\xc0\"6\n)\n\x12another_label_name\x12\x13another_label_value\x1a\t\t\x00\x00\x00\x00\x00\x00U@",
+			expected: model.Vector{
+				&model.Sample{
+					Metric: model.Metric{
+						model.MetricNameLabel: "request_count",
+						"some_label_name":     "some_label_value",
+					},
+					Value:     -42,
+					Timestamp: testTime,
+				},
+				&model.Sample{
+					Metric: model.Metric{
+						model.MetricNameLabel: "request_count",
+						"another_label_name":  "another_label_value",
+					},
+					Value:     84,
+					Timestamp: testTime,
+				},
+			},
+		},
+		{
+			in: "\xb9\x01\n\rrequest_count\x12\x12Number of requests\x18\x02\"O\n#\n\x0fsome_label_name\x12\x10some_label_value\"(\x1a\x12\t\xaeG\xe1z\x14\xae\xef?\x11\x00\x00\x00\x00\x00\x00E\xc0\x1a\x12\t+\x87\x16\xd9\xce\xf7\xef?\x11\x00\x00\x00\x00\x00\x00U\xc0\"A\n)\n\x12another_label_name\x12\x13another_label_value\"\x14\x1a\x12\t\x00\x00\x00\x00\x00\x00\xe0?\x11\x00\x00\x00\x00\x00\x00$@",
+			expected: model.Vector{
+				&model.Sample{
+					Metric: model.Metric{
+						model.MetricNameLabel: "request_count_count",
+						"some_label_name":     "some_label_value",
+					},
+					Value:     0,
+					Timestamp: testTime,
+				},
+				&model.Sample{
+					Metric: model.Metric{
+						model.MetricNameLabel: "request_count_sum",
+						"some_label_name":     "some_label_value",
+					},
+					Value:     0,
+					Timestamp: testTime,
+				},
+				&model.Sample{
+					Metric: model.Metric{
+						model.MetricNameLabel: "request_count",
+						"some_label_name":     "some_label_value",
+						"quantile":            "0.99",
+					},
+					Value:     -42,
+					Timestamp: testTime,
+				},
+				&model.Sample{
+					Metric: model.Metric{
+						model.MetricNameLabel: "request_count",
+						"some_label_name":     "some_label_value",
+						"quantile":            "0.999",
+					},
+					Value:     -84,
+					Timestamp: testTime,
+				},
+				&model.Sample{
+					Metric: model.Metric{
+						model.MetricNameLabel: "request_count_count",
+						"another_label_name":  "another_label_value",
+					},
+					Value:     0,
+					Timestamp: testTime,
+				},
+				&model.Sample{
+					Metric: model.Metric{
+						model.MetricNameLabel: "request_count_sum",
+						"another_label_name":  "another_label_value",
+					},
+					Value:     0,
+					Timestamp: testTime,
+				},
+				&model.Sample{
+					Metric: model.Metric{
+						model.MetricNameLabel: "request_count",
+						"another_label_name":  "another_label_value",
+						"quantile":            "0.5",
+					},
+					Value:     10,
+					Timestamp: testTime,
+				},
+			},
+		},
+		{
+			in: "\x8d\x01\n\x1drequest_duration_microseconds\x12\x15The response latency.\x18\x04\"S:Q\b\x85\x15\x11\xcd\xcc\xccL\x8f\xcb:A\x1a\v\b{\x11\x00\x00\x00\x00\x00\x00Y@\x1a\f\b\x9c\x03\x11\x00\x00\x00\x00\x00\x00^@\x1a\f\b\xd0\x04\x11\x00\x00\x00\x00\x00\x00b@\x1a\f\b\xf4\v\x11\x9a\x99\x99\x99\x99\x99e@\x1a\f\b\x85\x15\x11\x00\x00\x00\x00\x00\x00\xf0\u007f",
+			expected: model.Vector{
+				&model.Sample{
+					Metric: model.Metric{
+						model.MetricNameLabel: "request_duration_microseconds_bucket",
+						"le": "100",
+					},
+					Value:     123,
+					Timestamp: testTime,
+				},
+				&model.Sample{
+					Metric: model.Metric{
+						model.MetricNameLabel: "request_duration_microseconds_bucket",
+						"le": "120",
+					},
+					Value:     412,
+					Timestamp: testTime,
+				},
+				&model.Sample{
+					Metric: model.Metric{
+						model.MetricNameLabel: "request_duration_microseconds_bucket",
+						"le": "144",
+					},
+					Value:     592,
+					Timestamp: testTime,
+				},
+				&model.Sample{
+					Metric: model.Metric{
+						model.MetricNameLabel: "request_duration_microseconds_bucket",
+						"le": "172.8",
+					},
+					Value:     1524,
+					Timestamp: testTime,
+				},
+				&model.Sample{
+					Metric: model.Metric{
+						model.MetricNameLabel: "request_duration_microseconds_bucket",
+						"le": "+Inf",
+					},
+					Value:     2693,
+					Timestamp: testTime,
+				},
+				&model.Sample{
+					Metric: model.Metric{
+						model.MetricNameLabel: "request_duration_microseconds_sum",
+					},
+					Value:     1756047.3,
+					Timestamp: testTime,
+				},
+				&model.Sample{
+					Metric: model.Metric{
+						model.MetricNameLabel: "request_duration_microseconds_count",
+					},
+					Value:     2693,
+					Timestamp: testTime,
+				},
+			},
+		},
+		{
+			// The metric type is unset in this protobuf, which needs to be handled
+			// correctly by the decoder.
+			in: "\x1c\n\rrequest_count\"\v\x1a\t\t\x00\x00\x00\x00\x00\x00\xf0?",
+			expected: model.Vector{
+				&model.Sample{
+					Metric: model.Metric{
+						model.MetricNameLabel: "request_count",
+					},
+					Value:     1,
+					Timestamp: testTime,
+				},
+			},
+		},
+	}
+
+	for i, scenario := range scenarios {
+		dec := &SampleDecoder{
+			Dec: &protoDecoder{r: strings.NewReader(scenario.in)},
+			Opts: &DecodeOptions{
+				Timestamp: testTime,
+			},
+		}
+
+		var all model.Vector
+		for {
+			var smpls model.Vector
+			err := dec.Decode(&smpls)
+			if err == io.EOF {
+				break
+			}
+			if err != nil {
+				t.Fatal(err)
+			}
+			all = append(all, smpls...)
+		}
+		sort.Sort(all)
+		sort.Sort(scenario.expected)
+		if !reflect.DeepEqual(all, scenario.expected) {
+			t.Fatalf("%d. output does not match, want: %#v, got %#v", i, scenario.expected, all)
+		}
+	}
+}
+
+func testDiscriminatorHTTPHeader(t testing.TB) {
+	var scenarios = []struct {
+		input  map[string]string
+		output Format
+		err    error
+	}{
+		{
+			input:  map[string]string{"Content-Type": `application/vnd.google.protobuf; proto="io.prometheus.client.MetricFamily"; encoding="delimited"`},
+			output: FmtProtoDelim,
+		},
+		{
+			input:  map[string]string{"Content-Type": `application/vnd.google.protobuf; proto="illegal"; encoding="delimited"`},
+			output: FmtUnknown,
+		},
+		{
+			input:  map[string]string{"Content-Type": `application/vnd.google.protobuf; proto="io.prometheus.client.MetricFamily"; encoding="illegal"`},
+			output: FmtUnknown,
+		},
+		{
+			input:  map[string]string{"Content-Type": `text/plain; version=0.0.4`},
+			output: FmtText,
+		},
+		{
+			input:  map[string]string{"Content-Type": `text/plain`},
+			output: FmtText,
+		},
+		{
+			input:  map[string]string{"Content-Type": `text/plain; version=0.0.3`},
+			output: FmtUnknown,
+		},
+	}
+
+	for i, scenario := range scenarios {
+		var header http.Header
+
+		if len(scenario.input) > 0 {
+			header = http.Header{}
+		}
+
+		for key, value := range scenario.input {
+			header.Add(key, value)
+		}
+
+		actual := ResponseFormat(header)
+
+		if scenario.output != actual {
+			t.Errorf("%d. expected %s, got %s", i, scenario.output, actual)
+		}
+	}
+}
+
+func TestDiscriminatorHTTPHeader(t *testing.T) {
+	testDiscriminatorHTTPHeader(t)
+}
+
+func BenchmarkDiscriminatorHTTPHeader(b *testing.B) {
+	for i := 0; i < b.N; i++ {
+		testDiscriminatorHTTPHeader(b)
+	}
+}

Godeps/_workspace/src/github.com/prometheus/common/expfmt/encode.go

@@ -0,0 +1,87 @@
+// Copyright 2015 The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package expfmt
+
+import (
+	"fmt"
+	"io"
+	"net/http"
+
+	"github.com/coreos/etcd/Godeps/_workspace/src/bitbucket.org/ww/goautoneg"
+	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/golang/protobuf/proto"
+	"github.com/coreos/etcd/Godeps/_workspace/src/github.com/matttproud/golang_protobuf_extensions/pbutil"
+	dto "github.com/coreos/etcd/Godeps/_workspace/src/github.com/prometheus/client_model/go"
+)
+
+// Encoder types encode metric families into an underlying wire protocol.
+type Encoder interface {
+	Encode(*dto.MetricFamily) error
+}
+
+type encoder func(*dto.MetricFamily) error
+
+func (e encoder) Encode(v *dto.MetricFamily) error {
+	return e(v)
+}
+
+// Negotiate returns the Content-Type based on the given Accept header.
+// If no appropriate accepted type is found, FmtText is returned.
+func Negotiate(h http.Header) Format {
+	for _, ac := range goautoneg.ParseAccept(h.Get(hdrAccept)) {
+		// Check for protocol buffer
+		if ac.Type+"/"+ac.SubType == ProtoType && ac.Params["proto"] == ProtoProtocol {
+			switch ac.Params["encoding"] {
+			case "delimited":
+				return FmtProtoDelim
+			case "text":
+				return FmtProtoText
+			case "compact-text":
+				return FmtProtoCompact
+			}
+		}
+		// Check for text format.
+		ver := ac.Params["version"]
+		if ac.Type == "text" && ac.SubType == "plain" && (ver == TextVersion || ver == "") {
+			return FmtText
+		}
+	}
+	return FmtText
+}
+
+// NewEncoder returns a new encoder based on content type negotiation.
+func NewEncoder(w io.Writer, format Format) Encoder {
+	switch format {
+	case FmtProtoDelim:
+		return encoder(func(v *dto.MetricFamily) error {
+			_, err := pbutil.WriteDelimited(w, v)
+			return err
+		})
+	case FmtProtoCompact:
+		return encoder(func(v *dto.MetricFamily) error {
+			_, err := fmt.Fprintln(w, v.String())
+			return err
+		})
+	case FmtProtoText:
+		return encoder(func(v *dto.MetricFamily) error {
+			_, err := fmt.Fprintln(w, proto.MarshalTextString(v))
+			return err
+		})
+	case FmtText:
+		return encoder(func(v *dto.MetricFamily) error {
+			_, err := MetricFamilyToText(w, v)
+			return err
+		})
+	}
+	panic("expfmt.NewEncoder: unknown format")
+}

Godeps/_workspace/src/github.com/prometheus/common/expfmt/expfmt.go

@@ -0,0 +1,40 @@
+// Copyright 2015 The Prometheus Authors
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// A package for reading and writing Prometheus metrics.
+package expfmt
+
+type Format string
+
+const (
+	TextVersion = "0.0.4"
+
+	ProtoType     = `application/vnd.google.protobuf`
+	ProtoProtocol = `io.prometheus.client.MetricFamily`
+	ProtoFmt      = ProtoType + "; proto=" + ProtoProtocol + ";"
+
+	// The Content-Type values for the different wire protocols.
+	FmtUnknown      Format = `<unknown>`
+	FmtText         Format = `text/plain; version=` + TextVersion
+	FmtProtoDelim   Format = ProtoFmt + ` encoding=delimited`
+	FmtProtoText    Format = ProtoFmt + ` encoding=text`
+	FmtProtoCompact Format = ProtoFmt + ` encoding=compact-text`
+
+	// fmtJSON2 is hidden as it is deprecated.
+	fmtJSON2 Format = `application/json; version=0.0.2`
+)
+
+const (
+	hdrContentType = "Content-Type"
+	hdrAccept      = "Accept"
+)

Godeps/_workspace/src/github.com/prometheus/common/expfmt/fuzz.go

@@ -1,4 +1,4 @@
-// Copyright 2013 The Prometheus Authors
+// Copyright 2014 The Prometheus Authors
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
 // You may obtain a copy of the License at
@@ -11,26 +11,26 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.
 
-package model
+// Build only when actually fuzzing
+// +build gofuzz
 
-import (
-	"sort"
-)
+package expfmt
 
-// A LabelValue is an associated value for a LabelName.
-type LabelValue string
+import "bytes"
 
-// LabelValues is a sortable LabelValue slice. It implements sort.Interface.
-type LabelValues []LabelValue
+// Fuzz text metric parser with with github.com/dvyukov/go-fuzz:
+//
+//     go-fuzz-build github.com/prometheus/client_golang/text
+//     go-fuzz -bin text-fuzz.zip -workdir fuzz
+//
+// Further input samples should go in the folder fuzz/corpus.
+func Fuzz(in []byte) int {
+	parser := TextParser{}
+	_, err := parser.TextToMetricFamilies(bytes.NewReader(in))
 
-func (l LabelValues) Len() int {
-	return len(l)
-}
-
-func (l LabelValues) Less(i, j int) bool {
-	return sort.StringsAreSorted([]string{string(l[i]), string(l[j])})
-}
-
-func (l LabelValues) Swap(i, j int) {
-	l[i], l[j] = l[j], l[i]
+	if err != nil {
+		return 0
+	}
+
+	return 1
 }

Godeps/_workspace/src/github.com/prometheus/common/expfmt/fuzz/corpus/from_test_parse_0

@@ -0,0 +1,2 @@
+
+

Godeps/_workspace/src/github.com/prometheus/common/expfmt/fuzz/corpus/from_test_parse_1

@@ -0,0 +1,6 @@
+
+minimal_metric 1.234
+another_metric -3e3 103948
+# Even that:
+no_labels{} 3
+# HELP line for non-existing metric will be ignored.

Godeps/_workspace/src/github.com/prometheus/common/expfmt/fuzz/corpus/from_test_parse_2

@@ -0,0 +1,12 @@
+
+# A normal comment.
+#
+# TYPE name counter
+name{labelname="val1",basename="basevalue"} NaN
+name {labelname="val2",basename="base\"v\\al\nue"} 0.23 1234567890
+# HELP name two-line\n doc  str\\ing
+
+ # HELP  name2  	doc str"ing 2
+  #    TYPE    name2 gauge
+name2{labelname="val2"	,basename   =   "basevalue2"		} +Inf 54321
+name2{ labelname = "val1" , }-Inf

Godeps/_workspace/src/github.com/prometheus/common/expfmt/fuzz/corpus/from_test_parse_3

@@ -0,0 +1,22 @@
+
+# TYPE my_summary summary
+my_summary{n1="val1",quantile="0.5"} 110
+decoy -1 -2
+my_summary{n1="val1",quantile="0.9"} 140 1
+my_summary_count{n1="val1"} 42
+# Latest timestamp wins in case of a summary.
+my_summary_sum{n1="val1"} 4711 2
+fake_sum{n1="val1"} 2001
+# TYPE another_summary summary
+another_summary_count{n2="val2",n1="val1"} 20
+my_summary_count{n2="val2",n1="val1"} 5 5
+another_summary{n1="val1",n2="val2",quantile=".3"} -1.2
+my_summary_sum{n1="val2"} 08 15
+my_summary{n1="val3", quantile="0.2"} 4711
+  my_summary{n1="val1",n2="val2",quantile="-12.34",} NaN
+# some
+# funny comments
+# HELP 
+# HELP
+# HELP my_summary
+# HELP my_summary 

Godeps/_workspace/src/github.com/prometheus/common/expfmt/fuzz/corpus/from_test_parse_4

@@ -0,0 +1,10 @@
+
+# HELP request_duration_microseconds The response latency.
+# TYPE request_duration_microseconds histogram
+request_duration_microseconds_bucket{le="100"} 123
+request_duration_microseconds_bucket{le="120"} 412
+request_duration_microseconds_bucket{le="144"} 592
+request_duration_microseconds_bucket{le="172.8"} 1524
+request_duration_microseconds_bucket{le="+Inf"} 2693
+request_duration_microseconds_sum 1.7560473e+06
+request_duration_microseconds_count 2693

Godeps/_workspace/src/github.com/prometheus/common/expfmt/fuzz/corpus/from_test_parse_error_0

@@ -0,0 +1 @@
+bla 3.14

Godeps/_workspace/src/github.com/prometheus/common/expfmt/fuzz/corpus/from_test_parse_error_1

@@ -0,0 +1 @@
+metric{label="\t"} 3.14

Godeps/_workspace/src/github.com/prometheus/common/expfmt/fuzz/corpus/from_test_parse_error_10

@@ -0,0 +1 @@
+metric{label="bla"} 3.14 2 3

Godeps/_workspace/src/github.com/prometheus/common/expfmt/fuzz/corpus/from_test_parse_error_11

@@ -0,0 +1 @@
+metric{label="bla"} blubb

Godeps/_workspace/src/github.com/prometheus/common/expfmt/fuzz/corpus/from_test_parse_error_12

@@ -0,0 +1,3 @@
+
+# HELP metric one
+# HELP metric two