Merge branch 'bc/credential-scheme-enhancement'

The credential helper protocol, together with the HTTP layer, have
been enhanced to support authentication schemes different from
username & password pair, like Bearer and NTLM.

* bc/credential-scheme-enhancement:
  credential: add method for querying capabilities
  credential-cache: implement authtype capability
  t: add credential tests for authtype
  credential: add support for multistage credential rounds
  t5563: refactor for multi-stage authentication
  docs: set a limit on credential line length
  credential: enable state capability
  credential: add an argument to keep state
  http: add support for authtype and credential
  docs: indicate new credential protocol fields
  credential: add a field called "ephemeral"
  credential: gate new fields on capability
  credential: add a field for pre-encoded credentials
  http: use new headers for each object request
  remote-curl: reset headers on new request
  credential: add an authtype field

Documentation/git-credential.txt

@@ -8,7 +8,7 @@ git-credential - Retrieve and store user credentials
 SYNOPSIS
 --------
 ------------------
-'git credential' (fill|approve|reject)
+'git credential' (fill|approve|reject|capability)
 ------------------
 
 DESCRIPTION
@@ -41,6 +41,9 @@ If the action is `reject`, git-credential will send the description to
 any configured credential helpers, which may erase any stored
 credentials matching the description.
 
+If the action is `capability`, git-credential will announce any capabilities
+it supports to standard output.
+
 If the action is `approve` or `reject`, no output should be emitted.
 
 TYPICAL USE OF GIT CREDENTIAL
@@ -111,7 +114,9 @@ attribute per line. Each attribute is specified by a key-value pair,
 separated by an `=` (equals) sign, followed by a newline.
 
 The key may contain any bytes except `=`, newline, or NUL. The value may
-contain any bytes except newline or NUL.
+contain any bytes except newline or NUL.  A line, including the trailing
+newline, may not exceed 65535 bytes in order to allow implementations to
+parse efficiently.
 
 Attributes with keys that end with C-style array brackets `[]` can have
 multiple values. Each instance of a multi-valued attribute forms an
@@ -178,6 +183,61 @@ empty string.
 Components which are missing from the URL (e.g., there is no
 username in the example above) will be left unset.
 
+`authtype`::
+	This indicates that the authentication scheme in question should be used.
+	Common values for HTTP and HTTPS include `basic`, `bearer`, and `digest`,
+	although the latter is insecure and should not be used.  If `credential`
+	is used, this may be set to an arbitrary string suitable for the protocol in
+	question (usually HTTP).
++
+This value should not be sent unless the appropriate capability (see below) is
+provided on input.
+
+`credential`::
+	The pre-encoded credential, suitable for the protocol in question (usually
+	HTTP).  If this key is sent, `authtype` is mandatory, and `username` and
+	`password` are not used.  For HTTP, Git concatenates the `authtype` value and
+	this value with a single space to determine the `Authorization` header.
++
+This value should not be sent unless the appropriate capability (see below) is
+provided on input.
+
+`ephemeral`::
+	This boolean value indicates, if true, that the value in the `credential`
+	field should not be saved by the credential helper because its usefulness is
+	limited in time.  For example, an HTTP Digest `credential` value is computed
+	using a nonce and reusing it will not result in successful authentication.
+	This may also be used for situations with short duration (e.g., 24-hour)
+	credentials.  The default value is false.
++
+The credential helper will still be invoked with `store` or `erase` so that it
+can determine whether the operation was successful.
++
+This value should not be sent unless the appropriate capability (see below) is
+provided on input.
+
+`state[]`::
+	This value provides an opaque state that will be passed back to this helper
+	if it is called again.  Each different credential helper may specify this
+	once.  The value should include a prefix unique to the credential helper and
+	should ignore values that don't match its prefix.
++
+This value should not be sent unless the appropriate capability (see below) is
+provided on input.
+
+`continue`::
+	This is a boolean value, which, if enabled, indicates that this
+	authentication is a non-final part of a multistage authentication step. This
+	is common in protocols such as NTLM and Kerberos, where two rounds of client
+	authentication are required, and setting this flag allows the credential
+	helper to implement the multistage authentication step.  This flag should
+	only be sent if a further stage is required; that is, if another round of
+	authentication is expected.
++
+This value should not be sent unless the appropriate capability (see below) is
+provided on input.  This attribute is 'one-way' from a credential helper to
+pass information to Git (or other programs invoking `git credential`).
+
 `wwwauth[]`::
 
 	When an HTTP response is received by Git that includes one or more
@@ -189,7 +249,45 @@ attribute 'wwwauth[]', where the order of the attributes is the same as
 they appear in the HTTP response. This attribute is 'one-way' from Git
 to pass additional information to credential helpers.
 
-Unrecognised attributes are silently discarded.
+`capability[]`::
+	This signals that Git, or the helper, as appropriate, supports the capability
+	in question.  This can be used to provide better, more specific data as part
+	of the protocol.  A `capability[]` directive must precede any value depending
+	on it and these directives _should_ be the first item announced in the
+	protocol.
++
+There are two currently supported capabilities.  The first is `authtype`, which
+indicates that the `authtype`, `credential`, and `ephemeral` values are
+understood.  The second is `state`, which indicates that the `state[]` and
+`continue` values are understood.
++
+It is not obligatory to use the additional features just because the capability
+is supported, but they should not be provided without the capability.
+
+Unrecognised attributes and capabilities are silently discarded.
+
+[[CAPA-IOFMT]]
+CAPABILITY INPUT/OUTPUT FORMAT
+------------------------------
+
+For `git credential capability`, the format is slightly different. First, a
+`version 0` announcement is made to indicate the current version of the
+protocol, and then each capability is announced with a line like `capability
+authtype`. Credential helpers may also implement this format, again with the
+`capability` argument. Additional lines may be added in the future; callers
+should ignore lines which they don't understand.
+
+Because this is a new part of the credential helper protocol, older versions of
+Git, as well as some credential helpers, may not support it.  If a non-zero
+exit status is received, or if the first line doesn't start with the word
+`version` and a space, callers should assume that no capabilities are supported.
+
+The intention of this format is to differentiate it from the credential output
+in an unambiguous way.  It is possible to use very simple credential helpers
+(e.g., inline shell scripts) which always produce identical output.  Using a
+distinct format allows users to continue to use this syntax without having to
+worry about correctly implementing capability advertisements or accidentally
+confusing callers querying for capabilities.
 
 GIT
 ---

builtin/credential-cache--daemon.c

@@ -115,7 +115,9 @@ static int read_request(FILE *fh, struct credential *c,
 		return error("client sent bogus timeout line: %s", item.buf);
 	*timeout = atoi(p);
 
-	if (credential_read(c, fh) < 0)
+	credential_set_all_capabilities(c, CREDENTIAL_OP_INITIAL);
+
+	if (credential_read(c, fh, CREDENTIAL_OP_HELPER) < 0)
 		return -1;
 	return 0;
 }
@@ -131,8 +133,18 @@ static void serve_one_client(FILE *in, FILE *out)
 	else if (!strcmp(action.buf, "get")) {
 		struct credential_cache_entry *e = lookup_credential(&c);
 		if (e) {
-			fprintf(out, "username=%s\n", e->item.username);
+			e->item.capa_authtype.request_initial = 1;
-			fprintf(out, "password=%s\n", e->item.password);
+			e->item.capa_authtype.request_helper = 1;
+
+			fprintf(out, "capability[]=authtype\n");
+			if (e->item.username)
+				fprintf(out, "username=%s\n", e->item.username);
+			if (e->item.password)
+				fprintf(out, "password=%s\n", e->item.password);
+			if (credential_has_capability(&c.capa_authtype, CREDENTIAL_OP_HELPER) && e->item.authtype)
+				fprintf(out, "authtype=%s\n", e->item.authtype);
+			if (credential_has_capability(&c.capa_authtype, CREDENTIAL_OP_HELPER) && e->item.credential)
+				fprintf(out, "credential=%s\n", e->item.credential);
 			if (e->item.password_expiry_utc != TIME_MAX)
 				fprintf(out, "password_expiry_utc=%"PRItime"\n",
 					e->item.password_expiry_utc);
@@ -157,8 +169,10 @@ static void serve_one_client(FILE *in, FILE *out)
 	else if (!strcmp(action.buf, "store")) {
 		if (timeout < 0)
 			warning("cache client didn't specify a timeout");
-		else if (!c.username || !c.password)
+		else if ((!c.username || !c.password) && (!c.authtype && !c.credential))
 			warning("cache client gave us a partial credential");
+		else if (c.ephemeral)
+			warning("not storing ephemeral credential");
 		else {
 			remove_credential(&c, 0);
 			cache_credential(&c, timeout);

builtin/credential-cache.c

@@ -1,4 +1,5 @@
 #include "builtin.h"
+#include "credential.h"
 #include "gettext.h"
 #include "parse-options.h"
 #include "path.h"
@@ -127,6 +128,13 @@ static char *get_socket_path(void)
 	return socket;
 }
 
+static void announce_capabilities(void)
+{
+	struct credential c = CREDENTIAL_INIT;
+	c.capa_authtype.request_initial = 1;
+	credential_announce_capabilities(&c, stdout);
+}
+
 int cmd_credential_cache(int argc, const char **argv, const char *prefix)
 {
 	char *socket_path = NULL;
@@ -163,6 +171,8 @@ int cmd_credential_cache(int argc, const char **argv, const char *prefix)
 		do_cache(socket_path, op, timeout, FLAG_RELAY);
 	else if (!strcmp(op, "store"))
 		do_cache(socket_path, op, timeout, FLAG_RELAY|FLAG_SPAWN);
+	else if (!strcmp(op, "capability"))
+		announce_capabilities();
 	else
 		; /* ignore unknown operation */
 

builtin/credential-store.c

@@ -205,7 +205,7 @@ int cmd_credential_store(int argc, const char **argv, const char *prefix)
 	if (!fns.nr)
 		die("unable to set up default path; use --file");
 
-	if (credential_read(&c, stdin) < 0)
+	if (credential_read(&c, stdin, CREDENTIAL_OP_HELPER) < 0)
 		die("unable to read credential");
 
 	if (!strcmp(op, "get"))

builtin/credential.c

@@ -17,15 +17,24 @@ int cmd_credential(int argc, const char **argv, const char *prefix UNUSED)
 		usage(usage_msg);
 	op = argv[1];
 
-	if (credential_read(&c, stdin) < 0)
+	if (!strcmp(op, "capability")) {
+		credential_set_all_capabilities(&c, CREDENTIAL_OP_INITIAL);
+		credential_announce_capabilities(&c, stdout);
+		return 0;
+	}
+
+	if (credential_read(&c, stdin, CREDENTIAL_OP_INITIAL) < 0)
 		die("unable to read credential from stdin");
 
 	if (!strcmp(op, "fill")) {
-		credential_fill(&c);
+		credential_fill(&c, 0);
-		credential_write(&c, stdout);
+		credential_next_state(&c);
+		credential_write(&c, stdout, CREDENTIAL_OP_RESPONSE);
 	} else if (!strcmp(op, "approve")) {
+		credential_set_all_capabilities(&c, CREDENTIAL_OP_HELPER);
 		credential_approve(&c);
 	} else if (!strcmp(op, "reject")) {
+		credential_set_all_capabilities(&c, CREDENTIAL_OP_HELPER);
 		credential_reject(&c);
 	} else {
 		usage(usage_msg);

credential.c

@@ -25,13 +25,64 @@ void credential_clear(struct credential *c)
 	free(c->path);
 	free(c->username);
 	free(c->password);
+	free(c->credential);
 	free(c->oauth_refresh_token);
+	free(c->authtype);
 	string_list_clear(&c->helpers, 0);
 	strvec_clear(&c->wwwauth_headers);
+	strvec_clear(&c->state_headers);
+	strvec_clear(&c->state_headers_to_send);
 
 	credential_init(c);
 }
 
+void credential_next_state(struct credential *c)
+{
+	strvec_clear(&c->state_headers_to_send);
+	SWAP(c->state_headers, c->state_headers_to_send);
+}
+
+void credential_clear_secrets(struct credential *c)
+{
+	FREE_AND_NULL(c->password);
+	FREE_AND_NULL(c->credential);
+}
+
+static void credential_set_capability(struct credential_capability *capa,
+				      enum credential_op_type op_type)
+{
+	switch (op_type) {
+	case CREDENTIAL_OP_INITIAL:
+		capa->request_initial = 1;
+		break;
+	case CREDENTIAL_OP_HELPER:
+		capa->request_helper = 1;
+		break;
+	case CREDENTIAL_OP_RESPONSE:
+		capa->response = 1;
+		break;
+	}
+}
+
+
+void credential_set_all_capabilities(struct credential *c,
+				     enum credential_op_type op_type)
+{
+	credential_set_capability(&c->capa_authtype, op_type);
+	credential_set_capability(&c->capa_state, op_type);
+}
+
+static void announce_one(struct credential_capability *cc, const char *name, FILE *fp) {
+	if (cc->request_initial)
+		fprintf(fp, "capability %s\n", name);
+}
+
+void credential_announce_capabilities(struct credential *c, FILE *fp) {
+	fprintf(fp, "version 0\n");
+	announce_one(&c->capa_authtype, "authtype", fp);
+	announce_one(&c->capa_state, "state", fp);
+}
+
 int credential_match(const struct credential *want,
 		     const struct credential *have, int match_password)
 {
@@ -40,7 +91,8 @@ int credential_match(const struct credential *want,
 	       CHECK(host) &&
 	       CHECK(path) &&
 	       CHECK(username) &&
-	       (!match_password || CHECK(password));
+	       (!match_password || CHECK(password)) &&
+	       (!match_password || CHECK(credential));
 #undef CHECK
 }
 
@@ -208,7 +260,26 @@ static void credential_getpass(struct credential *c)
 						 PROMPT_ASKPASS);
 }
 
-int credential_read(struct credential *c, FILE *fp)
+int credential_has_capability(const struct credential_capability *capa,
+			      enum credential_op_type op_type)
+{
+	/*
+	 * We're checking here if each previous step indicated that we had the
+	 * capability.  If it did, then we want to pass it along; conversely, if
+	 * it did not, we don't want to report that to our caller.
+	 */
+	switch (op_type) {
+	case CREDENTIAL_OP_HELPER:
+		return capa->request_initial;
+	case CREDENTIAL_OP_RESPONSE:
+		return capa->request_initial && capa->request_helper;
+	default:
+		return 0;
+	}
+}
+
+int credential_read(struct credential *c, FILE *fp,
+		    enum credential_op_type op_type)
 {
 	struct strbuf line = STRBUF_INIT;
 
@@ -233,6 +304,9 @@ int credential_read(struct credential *c, FILE *fp)
 		} else if (!strcmp(key, "password")) {
 			free(c->password);
 			c->password = xstrdup(value);
+		} else if (!strcmp(key, "credential")) {
+			free(c->credential);
+			c->credential = xstrdup(value);
 		} else if (!strcmp(key, "protocol")) {
 			free(c->protocol);
 			c->protocol = xstrdup(value);
@@ -242,8 +316,19 @@ int credential_read(struct credential *c, FILE *fp)
 		} else if (!strcmp(key, "path")) {
 			free(c->path);
 			c->path = xstrdup(value);
+		} else if (!strcmp(key, "ephemeral")) {
+			c->ephemeral = !!git_config_bool("ephemeral", value);
 		} else if (!strcmp(key, "wwwauth[]")) {
 			strvec_push(&c->wwwauth_headers, value);
+		} else if (!strcmp(key, "state[]")) {
+			strvec_push(&c->state_headers, value);
+		} else if (!strcmp(key, "capability[]")) {
+			if (!strcmp(value, "authtype"))
+				credential_set_capability(&c->capa_authtype, op_type);
+			else if (!strcmp(value, "state"))
+				credential_set_capability(&c->capa_state, op_type);
+		} else if (!strcmp(key, "continue")) {
+			c->multistage = !!git_config_bool("continue", value);
 		} else if (!strcmp(key, "password_expiry_utc")) {
 			errno = 0;
 			c->password_expiry_utc = parse_timestamp(value, NULL, 10);
@@ -252,6 +337,9 @@ int credential_read(struct credential *c, FILE *fp)
 		} else if (!strcmp(key, "oauth_refresh_token")) {
 			free(c->oauth_refresh_token);
 			c->oauth_refresh_token = xstrdup(value);
+		} else if (!strcmp(key, "authtype")) {
+			free(c->authtype);
+			c->authtype = xstrdup(value);
 		} else if (!strcmp(key, "url")) {
 			credential_from_url(c, value);
 		} else if (!strcmp(key, "quit")) {
@@ -280,8 +368,20 @@ static void credential_write_item(FILE *fp, const char *key, const char *value,
 	fprintf(fp, "%s=%s\n", key, value);
 }
 
-void credential_write(const struct credential *c, FILE *fp)
+void credential_write(const struct credential *c, FILE *fp,
+		      enum credential_op_type op_type)
 {
+	if (credential_has_capability(&c->capa_authtype, op_type))
+		credential_write_item(fp, "capability[]", "authtype", 0);
+	if (credential_has_capability(&c->capa_state, op_type))
+		credential_write_item(fp, "capability[]", "state", 0);
+
+	if (credential_has_capability(&c->capa_authtype, op_type)) {
+		credential_write_item(fp, "authtype", c->authtype, 0);
+		credential_write_item(fp, "credential", c->credential, 0);
+		if (c->ephemeral)
+			credential_write_item(fp, "ephemeral", "1", 0);
+	}
 	credential_write_item(fp, "protocol", c->protocol, 1);
 	credential_write_item(fp, "host", c->host, 1);
 	credential_write_item(fp, "path", c->path, 0);
@@ -295,6 +395,12 @@ void credential_write(const struct credential *c, FILE *fp)
 	}
 	for (size_t i = 0; i < c->wwwauth_headers.nr; i++)
 		credential_write_item(fp, "wwwauth[]", c->wwwauth_headers.v[i], 0);
+	if (credential_has_capability(&c->capa_state, op_type)) {
+		if (c->multistage)
+			credential_write_item(fp, "continue", "1", 0);
+		for (size_t i = 0; i < c->state_headers_to_send.nr; i++)
+			credential_write_item(fp, "state[]", c->state_headers_to_send.v[i], 0);
+	}
 }
 
 static int run_credential_helper(struct credential *c,
@@ -317,14 +423,14 @@ static int run_credential_helper(struct credential *c,
 
 	fp = xfdopen(helper.in, "w");
 	sigchain_push(SIGPIPE, SIG_IGN);
-	credential_write(c, fp);
+	credential_write(c, fp, want_output ? CREDENTIAL_OP_HELPER : CREDENTIAL_OP_RESPONSE);
 	fclose(fp);
 	sigchain_pop(SIGPIPE);
 
 	if (want_output) {
 		int r;
 		fp = xfdopen(helper.out, "r");
-		r = credential_read(c, fp);
+		r = credential_read(c, fp, CREDENTIAL_OP_HELPER);
 		fclose(fp);
 		if (r < 0) {
 			finish_command(&helper);
@@ -357,14 +463,19 @@ static int credential_do(struct credential *c, const char *helper,
 	return r;
 }
 
-void credential_fill(struct credential *c)
+void credential_fill(struct credential *c, int all_capabilities)
 {
 	int i;
 
-	if (c->username && c->password)
+	if ((c->username && c->password) || c->credential)
 		return;
 
+	credential_next_state(c);
+	c->multistage = 0;
+
 	credential_apply_config(c);
+	if (all_capabilities)
+		credential_set_all_capabilities(c, CREDENTIAL_OP_INITIAL);
 
 	for (i = 0; i < c->helpers.nr; i++) {
 		credential_do(c, c->helpers.items[i].string, "get");
@@ -374,15 +485,17 @@ void credential_fill(struct credential *c)
 			/* Reset expiry to maintain consistency */
 			c->password_expiry_utc = TIME_MAX;
 		}
-		if (c->username && c->password)
+		if ((c->username && c->password) || c->credential) {
+			strvec_clear(&c->wwwauth_headers);
 			return;
+		}
 		if (c->quit)
 			die("credential helper '%s' told us to quit",
 			    c->helpers.items[i].string);
 	}
 
 	credential_getpass(c);
-	if (!c->username && !c->password)
+	if (!c->username && !c->password && !c->credential)
 		die("unable to get password from user");
 }
 
@@ -392,9 +505,11 @@ void credential_approve(struct credential *c)
 
 	if (c->approved)
 		return;
-	if (!c->username || !c->password || c->password_expiry_utc < time(NULL))
+	if (((!c->username || !c->password) && !c->credential) || c->password_expiry_utc < time(NULL))
 		return;
 
+	credential_next_state(c);
+
 	credential_apply_config(c);
 
 	for (i = 0; i < c->helpers.nr; i++)
@@ -406,6 +521,8 @@ void credential_reject(struct credential *c)
 {
 	int i;
 
+	credential_next_state(c);
+
 	credential_apply_config(c);
 
 	for (i = 0; i < c->helpers.nr; i++)
@@ -413,6 +530,7 @@ void credential_reject(struct credential *c)
 
 	FREE_AND_NULL(c->username);
 	FREE_AND_NULL(c->password);
+	FREE_AND_NULL(c->credential);
 	FREE_AND_NULL(c->oauth_refresh_token);
 	c->password_expiry_utc = TIME_MAX;
 	c->approved = 0;

credential.h

@@ -93,6 +93,27 @@
  * -----------------------------------------------------------------------
  */
 
+/*
+ * These values define the kind of operation we're performing and the
+ * capabilities at each stage.  The first is either an external request (via git
+ * credential fill) or an internal request (e.g., via the HTTP) code.  The
+ * second is the call to the credential helper, and the third is the response
+ * we're providing.
+ *
+ * At each stage, we will emit the capability only if the previous stage
+ * supported it.
+ */
+enum credential_op_type {
+	CREDENTIAL_OP_INITIAL  = 1,
+	CREDENTIAL_OP_HELPER   = 2,
+	CREDENTIAL_OP_RESPONSE = 3,
+};
+
+struct credential_capability {
+	unsigned request_initial:1,
+		 request_helper:1,
+		 response:1;
+};
 
 /**
  * This struct represents a single username/password combination
@@ -123,6 +144,16 @@ struct credential {
 	 */
 	struct strvec wwwauth_headers;
 
+	/**
+	 * A `strvec` of state headers received from credential helpers.
+	 */
+	struct strvec state_headers;
+
+	/**
+	 * A `strvec` of state headers to send to credential helpers.
+	 */
+	struct strvec state_headers_to_send;
+
 	/**
 	 * Internal use only. Keeps track of if we previously matched against a
 	 * WWW-Authenticate header line in order to re-fold future continuation
@@ -131,24 +162,38 @@ struct credential {
 	unsigned header_is_last_match:1;
 
 	unsigned approved:1,
+		 ephemeral:1,
 		 configured:1,
+		 multistage: 1,
 		 quit:1,
 		 use_http_path:1,
 		 username_from_proto:1;
 
+	struct credential_capability capa_authtype;
+	struct credential_capability capa_state;
+
 	char *username;
 	char *password;
+	char *credential;
 	char *protocol;
 	char *host;
 	char *path;
 	char *oauth_refresh_token;
 	timestamp_t password_expiry_utc;
+
+	/**
+	 * The authorization scheme to use.  If this is NULL, libcurl is free to
+	 * negotiate any scheme it likes.
+	 */
+	char *authtype;
 };
 
 #define CREDENTIAL_INIT { \
 	.helpers = STRING_LIST_INIT_DUP, \
 	.password_expiry_utc = TIME_MAX, \
 	.wwwauth_headers = STRVEC_INIT, \
+	.state_headers = STRVEC_INIT, \
+	.state_headers_to_send = STRVEC_INIT, \
 }
 
 /* Initialize a credential structure, setting all fields to empty. */
@@ -167,8 +212,11 @@ void credential_clear(struct credential *);
  * returns, the username and password fields of the credential are
  * guaranteed to be non-NULL. If an error occurs, the function will
  * die().
+ *
+ * If all_capabilities is set, this is an internal user that is prepared
+ * to deal with all known capabilities, and we should advertise that fact.
  */
-void credential_fill(struct credential *);
+void credential_fill(struct credential *, int all_capabilities);
 
 /**
  * Inform the credential subsystem that the provided credentials
@@ -191,8 +239,46 @@ void credential_approve(struct credential *);
  */
 void credential_reject(struct credential *);
 
-int credential_read(struct credential *, FILE *);
+/**
-void credential_write(const struct credential *, FILE *);
+ * Enable all of the supported credential flags in this credential.
+ */
+void credential_set_all_capabilities(struct credential *c,
+				     enum credential_op_type op_type);
+
+/**
+ * Clear the secrets in this credential, but leave other data intact.
+ *
+ * This is useful for resetting credentials in preparation for a subsequent
+ * stage of filling.
+ */
+void credential_clear_secrets(struct credential *c);
+
+/**
+ * Print a list of supported capabilities and version numbers to standard
+ * output.
+ */
+void credential_announce_capabilities(struct credential *c, FILE *fp);
+
+/**
+ * Prepares the credential for the next iteration of the helper protocol by
+ * updating the state headers to send with the ones read by the last iteration
+ * of the protocol.
+ *
+ * Except for internal callers, this should be called exactly once between
+ * reading credentials with `credential_fill` and writing them.
+ */
+void credential_next_state(struct credential *c);
+
+/**
+ * Return true if the capability is enabled for an operation of op_type.
+ */
+int credential_has_capability(const struct credential_capability *capa,
+			      enum credential_op_type op_type);
+
+int credential_read(struct credential *, FILE *,
+		    enum credential_op_type);
+void credential_write(const struct credential *, FILE *,
+		      enum credential_op_type);
 
 /*
  * Parse a url into a credential struct, replacing any existing contents.

http.c

@@ -128,7 +128,6 @@ static unsigned long empty_auth_useless =
 	| CURLAUTH_DIGEST;
 
 static struct curl_slist *pragma_header;
-static struct curl_slist *no_pragma_header;
 static struct string_list extra_http_headers = STRING_LIST_INIT_DUP;
 
 static struct curl_slist *host_resolutions;
@@ -299,6 +298,11 @@ size_t fwrite_null(char *ptr UNUSED, size_t eltsize UNUSED, size_t nmemb,
 	return nmemb;
 }
 
+static struct curl_slist *object_request_headers(void)
+{
+	return curl_slist_append(http_copy_default_headers(), "Pragma:");
+}
+
 static void closedown_active_slot(struct active_request_slot *slot)
 {
 	active_requests--;
@@ -557,18 +561,34 @@ static int curl_empty_auth_enabled(void)
 	return 0;
 }
 
+struct curl_slist *http_append_auth_header(const struct credential *c,
+					   struct curl_slist *headers)
+{
+	if (c->authtype && c->credential) {
+		struct strbuf auth = STRBUF_INIT;
+		strbuf_addf(&auth, "Authorization: %s %s",
+			    c->authtype, c->credential);
+		headers = curl_slist_append(headers, auth.buf);
+		strbuf_release(&auth);
+	}
+	return headers;
+}
+
 static void init_curl_http_auth(CURL *result)
 {
-	if (!http_auth.username || !*http_auth.username) {
+	if ((!http_auth.username || !*http_auth.username) &&
+	    (!http_auth.credential || !*http_auth.credential)) {
 		if (curl_empty_auth_enabled())
 			curl_easy_setopt(result, CURLOPT_USERPWD, ":");
 		return;
 	}
 
-	credential_fill(&http_auth);
+	credential_fill(&http_auth, 1);
 
-	curl_easy_setopt(result, CURLOPT_USERNAME, http_auth.username);
+	if (http_auth.password) {
-	curl_easy_setopt(result, CURLOPT_PASSWORD, http_auth.password);
+		curl_easy_setopt(result, CURLOPT_USERNAME, http_auth.username);
+		curl_easy_setopt(result, CURLOPT_PASSWORD, http_auth.password);
+	}
 }
 
 /* *var must be free-able */
@@ -582,17 +602,22 @@ static void var_override(const char **var, char *value)
 
 static void set_proxyauth_name_password(CURL *result)
 {
+	if (proxy_auth.password) {
 		curl_easy_setopt(result, CURLOPT_PROXYUSERNAME,
 			proxy_auth.username);
 		curl_easy_setopt(result, CURLOPT_PROXYPASSWORD,
 			proxy_auth.password);
+	} else if (proxy_auth.authtype && proxy_auth.credential) {
+		curl_easy_setopt(result, CURLOPT_PROXYHEADER,
+				 http_append_auth_header(&proxy_auth, NULL));
+	}
 }
 
 static void init_curl_proxy_auth(CURL *result)
 {
 	if (proxy_auth.username) {
-		if (!proxy_auth.password)
+		if (!proxy_auth.password && !proxy_auth.credential)
-			credential_fill(&proxy_auth);
+			credential_fill(&proxy_auth, 1);
 		set_proxyauth_name_password(result);
 	}
 
@@ -626,7 +651,7 @@ static int has_cert_password(void)
 		cert_auth.host = xstrdup("");
 		cert_auth.username = xstrdup("");
 		cert_auth.path = xstrdup(ssl_cert);
-		credential_fill(&cert_auth);
+		credential_fill(&cert_auth, 0);
 	}
 	return 1;
 }
@@ -641,7 +666,7 @@ static int has_proxy_cert_password(void)
 		proxy_cert_auth.host = xstrdup("");
 		proxy_cert_auth.username = xstrdup("");
 		proxy_cert_auth.path = xstrdup(http_proxy_ssl_cert);
-		credential_fill(&proxy_cert_auth);
+		credential_fill(&proxy_cert_auth, 0);
 	}
 	return 1;
 }
@@ -1275,8 +1300,6 @@ void http_init(struct remote *remote, const char *url, int proactive_auth)
 
 	pragma_header = curl_slist_append(http_copy_default_headers(),
 		"Pragma: no-cache");
-	no_pragma_header = curl_slist_append(http_copy_default_headers(),
-		"Pragma:");
 
 	{
 		char *http_max_requests = getenv("GIT_HTTP_MAX_REQUESTS");
@@ -1360,9 +1383,6 @@ void http_cleanup(void)
 	curl_slist_free_all(pragma_header);
 	pragma_header = NULL;
 
-	curl_slist_free_all(no_pragma_header);
-	no_pragma_header = NULL;
-
 	curl_slist_free_all(host_resolutions);
 	host_resolutions = NULL;
 
@@ -1470,7 +1490,7 @@ struct active_request_slot *get_active_slot(void)
 
 	curl_easy_setopt(slot->curl, CURLOPT_IPRESOLVE, git_curl_ipresolve);
 	curl_easy_setopt(slot->curl, CURLOPT_HTTPAUTH, http_auth_methods);
-	if (http_auth.password || curl_empty_auth_enabled())
+	if (http_auth.password || http_auth.credential || curl_empty_auth_enabled())
 		init_curl_http_auth(slot->curl);
 
 	return slot;
@@ -1759,7 +1779,12 @@ static int handle_curl_result(struct slot_results *results)
 	} else if (missing_target(results))
 		return HTTP_MISSING_TARGET;
 	else if (results->http_code == 401) {
-		if (http_auth.username && http_auth.password) {
+		if ((http_auth.username && http_auth.password) ||\
+		    (http_auth.authtype && http_auth.credential)) {
+			if (http_auth.multistage) {
+				credential_clear_secrets(&http_auth);
+				return HTTP_REAUTH;
+			}
 			credential_reject(&http_auth);
 			return HTTP_NOAUTH;
 		} else {
@@ -2067,11 +2092,15 @@ static int http_request(const char *url,
 	/* Add additional headers here */
 	if (options && options->extra_headers) {
 		const struct string_list_item *item;
-		for_each_string_list_item(item, options->extra_headers) {
+		if (options && options->extra_headers) {
-			headers = curl_slist_append(headers, item->string);
+			for_each_string_list_item(item, options->extra_headers) {
+				headers = curl_slist_append(headers, item->string);
+			}
 		}
 	}
 
+	headers = http_append_auth_header(&http_auth, headers);
+
 	curl_easy_setopt(slot->curl, CURLOPT_URL, url);
 	curl_easy_setopt(slot->curl, CURLOPT_HTTPHEADER, headers);
 	curl_easy_setopt(slot->curl, CURLOPT_ENCODING, "");
@@ -2153,6 +2182,7 @@ static int http_request_reauth(const char *url,
 			       void *result, int target,
 			       struct http_get_options *options)
 {
+	int i = 3;
 	int ret = http_request(url, result, target, options);
 
 	if (ret != HTTP_OK && ret != HTTP_REAUTH)
@@ -2166,35 +2196,35 @@ static int http_request_reauth(const char *url,
 		}
 	}
 
-	if (ret != HTTP_REAUTH)
+	while (ret == HTTP_REAUTH && --i) {
-		return ret;
+		/*
+		 * The previous request may have put cruft into our output stream; we
+		 * should clear it out before making our next request.
+		 */
+		switch (target) {
+		case HTTP_REQUEST_STRBUF:
+			strbuf_reset(result);
+			break;
+		case HTTP_REQUEST_FILE:
+			if (fflush(result)) {
+				error_errno("unable to flush a file");
+				return HTTP_START_FAILED;
+			}
+			rewind(result);
+			if (ftruncate(fileno(result), 0) < 0) {
+				error_errno("unable to truncate a file");
+				return HTTP_START_FAILED;
+			}
+			break;
+		default:
+			BUG("Unknown http_request target");
+		}
 
-	/*
+		credential_fill(&http_auth, 1);
-	 * The previous request may have put cruft into our output stream; we
+
-	 * should clear it out before making our next request.
+		ret = http_request(url, result, target, options);
-	 */
-	switch (target) {
-	case HTTP_REQUEST_STRBUF:
-		strbuf_reset(result);
-		break;
-	case HTTP_REQUEST_FILE:
-		if (fflush(result)) {
-			error_errno("unable to flush a file");
-			return HTTP_START_FAILED;
-		}
-		rewind(result);
-		if (ftruncate(fileno(result), 0) < 0) {
-			error_errno("unable to truncate a file");
-			return HTTP_START_FAILED;
-		}
-		break;
-	default:
-		BUG("Unknown http_request target");
 	}
-
+	return ret;
-	credential_fill(&http_auth);
-
-	return http_request(url, result, target, options);
 }
 
 int http_get_strbuf(const char *url,
@@ -2371,6 +2401,7 @@ void release_http_pack_request(struct http_pack_request *preq)
 	}
 	preq->slot = NULL;
 	strbuf_release(&preq->tmpfile);
+	curl_slist_free_all(preq->headers);
 	free(preq->url);
 	free(preq);
 }
@@ -2455,11 +2486,11 @@ struct http_pack_request *new_direct_http_pack_request(
 	}
 
 	preq->slot = get_active_slot();
+	preq->headers = object_request_headers();
 	curl_easy_setopt(preq->slot->curl, CURLOPT_WRITEDATA, preq->packfile);
 	curl_easy_setopt(preq->slot->curl, CURLOPT_WRITEFUNCTION, fwrite);
 	curl_easy_setopt(preq->slot->curl, CURLOPT_URL, preq->url);
-	curl_easy_setopt(preq->slot->curl, CURLOPT_HTTPHEADER,
+	curl_easy_setopt(preq->slot->curl, CURLOPT_HTTPHEADER, preq->headers);
-		no_pragma_header);
 
 	/*
 	 * If there is data present from a previous transfer attempt,
@@ -2625,13 +2656,14 @@ struct http_object_request *new_http_object_request(const char *base_url,
 	}
 
 	freq->slot = get_active_slot();
+	freq->headers = object_request_headers();
 
 	curl_easy_setopt(freq->slot->curl, CURLOPT_WRITEDATA, freq);
 	curl_easy_setopt(freq->slot->curl, CURLOPT_FAILONERROR, 0);
 	curl_easy_setopt(freq->slot->curl, CURLOPT_WRITEFUNCTION, fwrite_sha1_file);
 	curl_easy_setopt(freq->slot->curl, CURLOPT_ERRORBUFFER, freq->errorstr);
 	curl_easy_setopt(freq->slot->curl, CURLOPT_URL, freq->url);
-	curl_easy_setopt(freq->slot->curl, CURLOPT_HTTPHEADER, no_pragma_header);
+	curl_easy_setopt(freq->slot->curl, CURLOPT_HTTPHEADER, freq->headers);
 
 	/*
 	 * If we have successfully processed data from a previous fetch
@@ -2719,5 +2751,6 @@ void release_http_object_request(struct http_object_request *freq)
 		release_active_slot(freq->slot);
 		freq->slot = NULL;
 	}
+	curl_slist_free_all(freq->headers);
 	strbuf_release(&freq->tmpfile);
 }

http.h

@@ -175,6 +175,9 @@ int http_get_file(const char *url, const char *filename,
 
 int http_fetch_ref(const char *base, struct ref *ref);
 
+struct curl_slist *http_append_auth_header(const struct credential *c,
+					   struct curl_slist *headers);
+
 /* Helpers for fetching packs */
 int http_get_info_packs(const char *base_url,
 			struct packed_git **packs_head);
@@ -196,6 +199,7 @@ struct http_pack_request {
 	FILE *packfile;
 	struct strbuf tmpfile;
 	struct active_request_slot *slot;
+	struct curl_slist *headers;
 };
 
 struct http_pack_request *new_http_pack_request(
@@ -229,6 +233,7 @@ struct http_object_request {
 	int zret;
 	int rename;
 	struct active_request_slot *slot;
+	struct curl_slist *headers;
 };
 
 struct http_object_request *new_http_object_request(

imap-send.c

@@ -917,7 +917,7 @@ static void server_fill_credential(struct imap_server_conf *srvc, struct credent
 	cred->username = xstrdup_or_null(srvc->user);
 	cred->password = xstrdup_or_null(srvc->pass);
 
-	credential_fill(cred);
+	credential_fill(cred, 1);
 
 	if (!srvc->user)
 		srvc->user = xstrdup(cred->username);

remote-curl.c

@@ -889,7 +889,7 @@ static curl_off_t xcurl_off_t(size_t len)
 static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_received)
 {
 	struct active_request_slot *slot;
-	struct curl_slist *headers = http_copy_default_headers();
+	struct curl_slist *headers = NULL;
 	int use_gzip = rpc->gzip_request;
 	char *gzip_body = NULL;
 	size_t gzip_size = 0;
@@ -922,20 +922,24 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece
 		do {
 			err = probe_rpc(rpc, &results);
 			if (err == HTTP_REAUTH)
-				credential_fill(&http_auth);
+				credential_fill(&http_auth, 0);
 		} while (err == HTTP_REAUTH);
 		if (err != HTTP_OK)
 			return -1;
 
-		if (results.auth_avail & CURLAUTH_GSSNEGOTIATE)
+		if (results.auth_avail & CURLAUTH_GSSNEGOTIATE || http_auth.authtype)
 			needs_100_continue = 1;
 	}
 
+retry:
+	headers = http_copy_default_headers();
 	headers = curl_slist_append(headers, rpc->hdr_content_type);
 	headers = curl_slist_append(headers, rpc->hdr_accept);
 	headers = curl_slist_append(headers, needs_100_continue ?
 		"Expect: 100-continue" : "Expect:");
 
+	headers = http_append_auth_header(&http_auth, headers);
+
 	/* Add Accept-Language header */
 	if (rpc->hdr_accept_language)
 		headers = curl_slist_append(headers, rpc->hdr_accept_language);
@@ -944,7 +948,6 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece
 	if (rpc->protocol_header)
 		headers = curl_slist_append(headers, rpc->protocol_header);
 
-retry:
 	slot = get_active_slot();
 
 	curl_easy_setopt(slot->curl, CURLOPT_NOBODY, 0);
@@ -1041,7 +1044,8 @@ static int post_rpc(struct rpc_state *rpc, int stateless_connect, int flush_rece
 	rpc->any_written = 0;
 	err = run_slot(slot, NULL);
 	if (err == HTTP_REAUTH && !large_request) {
-		credential_fill(&http_auth);
+		credential_fill(&http_auth, 0);
+		curl_slist_free_all(headers);
 		goto retry;
 	}
 	if (err != HTTP_OK)

t/lib-credential.sh

@@ -538,6 +538,129 @@ helper_test_oauth_refresh_token() {
 	'
 }
 
+helper_test_authtype() {
+	HELPER=$1
+
+	test_expect_success "helper ($HELPER) stores authtype and credential" '
+		check approve $HELPER <<-\EOF
+		capability[]=authtype
+		authtype=Bearer
+		credential=random-token
+		protocol=https
+		host=git.example.com
+		EOF
+	'
+
+	test_expect_success "helper ($HELPER) gets authtype and credential" '
+		check fill $HELPER <<-\EOF
+		capability[]=authtype
+		protocol=https
+		host=git.example.com
+		--
+		capability[]=authtype
+		authtype=Bearer
+		credential=random-token
+		protocol=https
+		host=git.example.com
+		--
+		EOF
+	'
+
+	test_expect_success "helper ($HELPER) stores authtype and credential with username" '
+		check approve $HELPER <<-\EOF
+		capability[]=authtype
+		authtype=Bearer
+		credential=other-token
+		protocol=https
+		host=git.example.com
+		username=foobar
+		EOF
+	'
+
+	test_expect_success "helper ($HELPER) gets authtype and credential with username" '
+		check fill $HELPER <<-\EOF
+		capability[]=authtype
+		protocol=https
+		host=git.example.com
+		username=foobar
+		--
+		capability[]=authtype
+		authtype=Bearer
+		credential=other-token
+		protocol=https
+		host=git.example.com
+		username=foobar
+		--
+		EOF
+	'
+
+	test_expect_success "helper ($HELPER) does not get authtype and credential with different username" '
+		check fill $HELPER <<-\EOF
+		capability[]=authtype
+		protocol=https
+		host=git.example.com
+		username=barbaz
+		--
+		protocol=https
+		host=git.example.com
+		username=barbaz
+		password=askpass-password
+		--
+		askpass: Password for '\''https://barbaz@git.example.com'\'':
+		EOF
+	'
+
+	test_expect_success "helper ($HELPER) does not store ephemeral authtype and credential" '
+		check approve $HELPER <<-\EOF &&
+		capability[]=authtype
+		authtype=Bearer
+		credential=git2-token
+		protocol=https
+		host=git2.example.com
+		ephemeral=1
+		EOF
+
+		check fill $HELPER <<-\EOF
+		capability[]=authtype
+		protocol=https
+		host=git2.example.com
+		--
+		protocol=https
+		host=git2.example.com
+		username=askpass-username
+		password=askpass-password
+		--
+		askpass: Username for '\''https://git2.example.com'\'':
+		askpass: Password for '\''https://askpass-username@git2.example.com'\'':
+		EOF
+	'
+
+	test_expect_success "helper ($HELPER) does not store ephemeral username and password" '
+		check approve $HELPER <<-\EOF &&
+		capability[]=authtype
+		protocol=https
+		host=git2.example.com
+		user=barbaz
+		password=secret
+		ephemeral=1
+		EOF
+
+		check fill $HELPER <<-\EOF
+		capability[]=authtype
+		protocol=https
+		host=git2.example.com
+		--
+		protocol=https
+		host=git2.example.com
+		username=askpass-username
+		password=askpass-password
+		--
+		askpass: Username for '\''https://git2.example.com'\'':
+		askpass: Password for '\''https://askpass-username@git2.example.com'\'':
+		EOF
+	'
+}
+
 write_script askpass <<\EOF
 echo >&2 askpass: $*
 what=$(echo $1 | cut -d" " -f1 | tr A-Z a-z | tr -cd a-z)

t/lib-httpd/nph-custom-auth.sh

@@ -19,21 +19,30 @@ CHALLENGE_FILE=custom-auth.challenge
 #
 
 if test -n "$HTTP_AUTHORIZATION" && \
-	grep -Fqsx "${HTTP_AUTHORIZATION}" "$VALID_CREDS_FILE"
+	grep -Fqs "creds=${HTTP_AUTHORIZATION}" "$VALID_CREDS_FILE"
 then
+	idno=$(grep -F "creds=${HTTP_AUTHORIZATION}" "$VALID_CREDS_FILE" | sed -e 's/^id=\([a-z0-9-][a-z0-9-]*\) .*$/\1/')
+	status=$(sed -ne "s/^id=$idno.*status=\\([0-9][0-9][0-9]\\).*\$/\\1/p" "$CHALLENGE_FILE" | head -n1)
 	# Note that although git-http-backend returns a status line, it
 	# does so using a CGI 'Status' header. Because this script is an
 	# No Parsed Headers (NPH) script, we must return a real HTTP
 	# status line.
 	# This is only a test script, so we don't bother to check for
 	# the actual status from git-http-backend and always return 200.
-	echo 'HTTP/1.1 200 OK'
+	echo "HTTP/1.1 $status Nonspecific Reason Phrase"
-	exec "$GIT_EXEC_PATH"/git-http-backend
+	if test "$status" -eq 200
+	then
+		exec "$GIT_EXEC_PATH"/git-http-backend
+	else
+		sed -ne "s/^id=$idno.*response=//p" "$CHALLENGE_FILE"
+		echo
+		exit
+	fi
 fi
 
 echo 'HTTP/1.1 401 Authorization Required'
 if test -f "$CHALLENGE_FILE"
 then
-	cat "$CHALLENGE_FILE"
+	sed -ne 's/^id=default.*response=//p' "$CHALLENGE_FILE"
 fi
 echo

t/t0300-credentials.sh

@@ -12,7 +12,13 @@ test_expect_success 'setup helper scripts' '
 	IFS==
 	while read key value; do
 		echo >&2 "$whoami: $key=$value"
-		eval "$key=$value"
+		if test -z "${key%%*\[\]}"
+		then
+			key=${key%%\[\]}
+			eval "$key=\"\$$key $value\""
+		else
+			eval "$key=$value"
+		fi
 	done
 	IFS=$OIFS
 	EOF
@@ -35,6 +41,30 @@ test_expect_success 'setup helper scripts' '
 	test -z "$pass" || echo password=$pass
 	EOF
 
+	write_script git-credential-verbatim-cred <<-\EOF &&
+	authtype=$1; shift
+	credential=$1; shift
+	. ./dump
+	echo capability[]=authtype
+	echo capability[]=state
+	test -z "${capability##*authtype*}" || exit 0
+	test -z "$authtype" || echo authtype=$authtype
+	test -z "$credential" || echo credential=$credential
+	test -z "${capability##*state*}" || exit 0
+	echo state[]=verbatim-cred:foo
+	EOF
+
+	write_script git-credential-verbatim-ephemeral <<-\EOF &&
+	authtype=$1; shift
+	credential=$1; shift
+	. ./dump
+	echo capability[]=authtype
+	test -z "${capability##*authtype*}" || exit 0
+	test -z "$authtype" || echo authtype=$authtype
+	test -z "$credential" || echo credential=$credential
+	echo "ephemeral=1"
+	EOF
+
 	write_script git-credential-verbatim-with-expiry <<-\EOF &&
 	user=$1; shift
 	pass=$1; shift
@@ -64,6 +94,67 @@ test_expect_success 'credential_fill invokes helper' '
 	EOF
 '
 
+test_expect_success 'credential_fill invokes helper with credential' '
+	check fill "verbatim-cred Bearer token" <<-\EOF
+	capability[]=authtype
+	protocol=http
+	host=example.com
+	--
+	capability[]=authtype
+	authtype=Bearer
+	credential=token
+	protocol=http
+	host=example.com
+	--
+	verbatim-cred: get
+	verbatim-cred: capability[]=authtype
+	verbatim-cred: protocol=http
+	verbatim-cred: host=example.com
+	EOF
+'
+
+test_expect_success 'credential_fill invokes helper with ephemeral credential' '
+	check fill "verbatim-ephemeral Bearer token" <<-\EOF
+	capability[]=authtype
+	protocol=http
+	host=example.com
+	--
+	capability[]=authtype
+	authtype=Bearer
+	credential=token
+	ephemeral=1
+	protocol=http
+	host=example.com
+	--
+	verbatim-ephemeral: get
+	verbatim-ephemeral: capability[]=authtype
+	verbatim-ephemeral: protocol=http
+	verbatim-ephemeral: host=example.com
+	EOF
+'
+test_expect_success 'credential_fill invokes helper with credential and state' '
+	check fill "verbatim-cred Bearer token" <<-\EOF
+	capability[]=authtype
+	capability[]=state
+	protocol=http
+	host=example.com
+	--
+	capability[]=authtype
+	capability[]=state
+	authtype=Bearer
+	credential=token
+	protocol=http
+	host=example.com
+	state[]=verbatim-cred:foo
+	--
+	verbatim-cred: get
+	verbatim-cred: capability[]=authtype
+	verbatim-cred: capability[]=state
+	verbatim-cred: protocol=http
+	verbatim-cred: host=example.com
+	EOF
+'
+
 test_expect_success 'credential_fill invokes multiple helpers' '
 	check fill useless "verbatim foo bar" <<-\EOF
 	protocol=http
@@ -83,6 +174,45 @@ test_expect_success 'credential_fill invokes multiple helpers' '
 	EOF
 '
 
+test_expect_success 'credential_fill response does not get capabilities when helpers are incapable' '
+	check fill useless "verbatim foo bar" <<-\EOF
+	capability[]=authtype
+	capability[]=state
+	protocol=http
+	host=example.com
+	--
+	protocol=http
+	host=example.com
+	username=foo
+	password=bar
+	--
+	useless: get
+	useless: capability[]=authtype
+	useless: capability[]=state
+	useless: protocol=http
+	useless: host=example.com
+	verbatim: get
+	verbatim: capability[]=authtype
+	verbatim: capability[]=state
+	verbatim: protocol=http
+	verbatim: host=example.com
+	EOF
+'
+
+test_expect_success 'credential_fill response does not get capabilities when caller is incapable' '
+	check fill "verbatim-cred Bearer token" <<-\EOF
+	protocol=http
+	host=example.com
+	--
+	protocol=http
+	host=example.com
+	--
+	verbatim-cred: get
+	verbatim-cred: protocol=http
+	verbatim-cred: host=example.com
+	EOF
+'
+
 test_expect_success 'credential_fill stops when we get a full response' '
 	check fill "verbatim one two" "verbatim three four" <<-\EOF
 	protocol=http
@@ -99,6 +229,25 @@ test_expect_success 'credential_fill stops when we get a full response' '
 	EOF
 '
 
+test_expect_success 'credential_fill thinks a credential is a full response' '
+	check fill "verbatim-cred Bearer token" "verbatim three four" <<-\EOF
+	capability[]=authtype
+	protocol=http
+	host=example.com
+	--
+	capability[]=authtype
+	authtype=Bearer
+	credential=token
+	protocol=http
+	host=example.com
+	--
+	verbatim-cred: get
+	verbatim-cred: capability[]=authtype
+	verbatim-cred: protocol=http
+	verbatim-cred: host=example.com
+	EOF
+'
+
 test_expect_success 'credential_fill continues through partial response' '
 	check fill "verbatim one \"\"" "verbatim two three" <<-\EOF
 	protocol=http
@@ -175,6 +324,20 @@ test_expect_success 'credential_fill passes along metadata' '
 	EOF
 '
 
+test_expect_success 'credential_fill produces no credential without capability' '
+	check fill "verbatim-cred Bearer token" <<-\EOF
+	protocol=http
+	host=example.com
+	--
+	protocol=http
+	host=example.com
+	--
+	verbatim-cred: get
+	verbatim-cred: protocol=http
+	verbatim-cred: host=example.com
+	EOF
+'
+
 test_expect_success 'credential_approve calls all helpers' '
 	check approve useless "verbatim one two" <<-\EOF
 	protocol=http

t/t0301-credential-cache.sh

@@ -39,6 +39,7 @@ test_atexit 'git credential-cache exit'
 helper_test cache
 helper_test_password_expiry_utc cache
 helper_test_oauth_refresh_token cache
+helper_test_authtype cache
 
 test_expect_success 'socket defaults to ~/.cache/git/credential/socket' '
 	test_when_finished "

t/t5563-simple-http-auth.sh

@@ -21,9 +21,17 @@ test_expect_success 'setup_credential_helper' '
 	CREDENTIAL_HELPER="$TRASH_DIRECTORY/bin/git-credential-test-helper" &&
 	write_script "$CREDENTIAL_HELPER" <<-\EOF
 	cmd=$1
-	teefile=$cmd-query.cred
+	teefile=$cmd-query-temp.cred
 	catfile=$cmd-reply.cred
 	sed -n -e "/^$/q" -e "p" >>$teefile
+	state=$(sed -ne "s/^state\[\]=helper://p" "$teefile")
+	if test -z "$state"
+	then
+		mv "$teefile" "$cmd-query.cred"
+	else
+		mv "$teefile" "$cmd-query-$state.cred"
+		catfile="$cmd-reply-$state.cred"
+	fi
 	if test "$cmd" = "get"
 	then
 		cat $catfile
@@ -32,13 +40,15 @@ test_expect_success 'setup_credential_helper' '
 '
 
 set_credential_reply () {
-	cat >"$TRASH_DIRECTORY/$1-reply.cred"
+	local suffix="$(test -n "$2" && echo "-$2")"
+	cat >"$TRASH_DIRECTORY/$1-reply$suffix.cred"
 }
 
 expect_credential_query () {
-	cat >"$TRASH_DIRECTORY/$1-expect.cred" &&
+	local suffix="$(test -n "$2" && echo "-$2")"
-	test_cmp "$TRASH_DIRECTORY/$1-expect.cred" \
+	cat >"$TRASH_DIRECTORY/$1-expect$suffix.cred" &&
-		 "$TRASH_DIRECTORY/$1-query.cred"
+	test_cmp "$TRASH_DIRECTORY/$1-expect$suffix.cred" \
+		 "$TRASH_DIRECTORY/$1-query$suffix.cred"
 }
 
 per_test_cleanup () {
@@ -63,17 +73,20 @@ test_expect_success 'access using basic auth' '
 
 	# Basic base64(alice:secret-passwd)
 	cat >"$HTTPD_ROOT_PATH/custom-auth.valid" <<-EOF &&
-	Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==
+	id=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==
 	EOF
 
 	cat >"$HTTPD_ROOT_PATH/custom-auth.challenge" <<-EOF &&
-	WWW-Authenticate: Basic realm="example.com"
+	id=1 status=200
+	id=default response=WWW-Authenticate: Basic realm="example.com"
 	EOF
 
 	test_config_global credential.helper test-helper &&
 	git ls-remote "$HTTPD_URL/custom_auth/repo.git" &&
 
 	expect_credential_query get <<-EOF &&
+	capability[]=authtype
+	capability[]=state
 	protocol=http
 	host=$HTTPD_DEST
 	wwwauth[]=Basic realm="example.com"
@@ -87,6 +100,45 @@ test_expect_success 'access using basic auth' '
 	EOF
 '
 
+test_expect_success 'access using basic auth via authtype' '
+	test_when_finished "per_test_cleanup" &&
+
+	set_credential_reply get <<-EOF &&
+	capability[]=authtype
+	authtype=Basic
+	credential=YWxpY2U6c2VjcmV0LXBhc3N3ZA==
+	EOF
+
+	# Basic base64(alice:secret-passwd)
+	cat >"$HTTPD_ROOT_PATH/custom-auth.valid" <<-EOF &&
+	id=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==
+	EOF
+
+	cat >"$HTTPD_ROOT_PATH/custom-auth.challenge" <<-EOF &&
+	id=1 status=200
+	id=default response=WWW-Authenticate: Basic realm="example.com"
+	EOF
+
+	test_config_global credential.helper test-helper &&
+	GIT_CURL_VERBOSE=1 git ls-remote "$HTTPD_URL/custom_auth/repo.git" &&
+
+	expect_credential_query get <<-EOF &&
+	capability[]=authtype
+	capability[]=state
+	protocol=http
+	host=$HTTPD_DEST
+	wwwauth[]=Basic realm="example.com"
+	EOF
+
+	expect_credential_query store <<-EOF
+	capability[]=authtype
+	authtype=Basic
+	credential=YWxpY2U6c2VjcmV0LXBhc3N3ZA==
+	protocol=http
+	host=$HTTPD_DEST
+	EOF
+'
+
 test_expect_success 'access using basic auth invalid credentials' '
 	test_when_finished "per_test_cleanup" &&
 
@@ -97,17 +149,20 @@ test_expect_success 'access using basic auth invalid credentials' '
 
 	# Basic base64(alice:secret-passwd)
 	cat >"$HTTPD_ROOT_PATH/custom-auth.valid" <<-EOF &&
-	Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==
+	id=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==
 	EOF
 
 	cat >"$HTTPD_ROOT_PATH/custom-auth.challenge" <<-EOF &&
-	WWW-Authenticate: Basic realm="example.com"
+	id=1 status=200
+	id=default response=WWW-Authenticate: Basic realm="example.com"
 	EOF
 
 	test_config_global credential.helper test-helper &&
 	test_must_fail git ls-remote "$HTTPD_URL/custom_auth/repo.git" &&
 
 	expect_credential_query get <<-EOF &&
+	capability[]=authtype
+	capability[]=state
 	protocol=http
 	host=$HTTPD_DEST
 	wwwauth[]=Basic realm="example.com"
@@ -132,19 +187,22 @@ test_expect_success 'access using basic auth with extra challenges' '
 
 	# Basic base64(alice:secret-passwd)
 	cat >"$HTTPD_ROOT_PATH/custom-auth.valid" <<-EOF &&
-	Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==
+	id=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==
 	EOF
 
 	cat >"$HTTPD_ROOT_PATH/custom-auth.challenge" <<-EOF &&
-	WWW-Authenticate: FooBar param1="value1" param2="value2"
+	id=1 status=200
-	WWW-Authenticate: Bearer authorize_uri="id.example.com" p=1 q=0
+	id=default response=WWW-Authenticate: FooBar param1="value1" param2="value2"
-	WWW-Authenticate: Basic realm="example.com"
+	id=default response=WWW-Authenticate: Bearer authorize_uri="id.example.com" p=1 q=0
+	id=default response=WWW-Authenticate: Basic realm="example.com"
 	EOF
 
 	test_config_global credential.helper test-helper &&
 	git ls-remote "$HTTPD_URL/custom_auth/repo.git" &&
 
 	expect_credential_query get <<-EOF &&
+	capability[]=authtype
+	capability[]=state
 	protocol=http
 	host=$HTTPD_DEST
 	wwwauth[]=FooBar param1="value1" param2="value2"
@@ -170,19 +228,22 @@ test_expect_success 'access using basic auth mixed-case wwwauth header name' '
 
 	# Basic base64(alice:secret-passwd)
 	cat >"$HTTPD_ROOT_PATH/custom-auth.valid" <<-EOF &&
-	Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==
+	id=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==
 	EOF
 
 	cat >"$HTTPD_ROOT_PATH/custom-auth.challenge" <<-EOF &&
-	www-authenticate: foobar param1="value1" param2="value2"
+	id=1 status=200
-	WWW-AUTHENTICATE: BEARER authorize_uri="id.example.com" p=1 q=0
+	id=default response=www-authenticate: foobar param1="value1" param2="value2"
-	WwW-aUtHeNtIcAtE: baSiC realm="example.com"
+	id=default response=WWW-AUTHENTICATE: BEARER authorize_uri="id.example.com" p=1 q=0
+	id=default response=WwW-aUtHeNtIcAtE: baSiC realm="example.com"
 	EOF
 
 	test_config_global credential.helper test-helper &&
 	git ls-remote "$HTTPD_URL/custom_auth/repo.git" &&
 
 	expect_credential_query get <<-EOF &&
+	capability[]=authtype
+	capability[]=state
 	protocol=http
 	host=$HTTPD_DEST
 	wwwauth[]=foobar param1="value1" param2="value2"
@@ -208,24 +269,27 @@ test_expect_success 'access using basic auth with wwwauth header continuations'
 
 	# Basic base64(alice:secret-passwd)
 	cat >"$HTTPD_ROOT_PATH/custom-auth.valid" <<-EOF &&
-	Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==
+	id=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==
 	EOF
 
 	# Note that leading and trailing whitespace is important to correctly
 	# simulate a continuation/folded header.
 	cat >"$HTTPD_ROOT_PATH/custom-auth.challenge" <<-EOF &&
-	WWW-Authenticate: FooBar param1="value1"
+	id=1 status=200
-	 param2="value2"
+	id=default response=WWW-Authenticate: FooBar param1="value1"
-	WWW-Authenticate: Bearer authorize_uri="id.example.com"
+	id=default response= param2="value2"
-	 p=1
+	id=default response=WWW-Authenticate: Bearer authorize_uri="id.example.com"
-	 q=0
+	id=default response= p=1
-	WWW-Authenticate: Basic realm="example.com"
+	id=default response= q=0
+	id=default response=WWW-Authenticate: Basic realm="example.com"
 	EOF
 
 	test_config_global credential.helper test-helper &&
 	git ls-remote "$HTTPD_URL/custom_auth/repo.git" &&
 
 	expect_credential_query get <<-EOF &&
+	capability[]=authtype
+	capability[]=state
 	protocol=http
 	host=$HTTPD_DEST
 	wwwauth[]=FooBar param1="value1" param2="value2"
@@ -251,26 +315,29 @@ test_expect_success 'access using basic auth with wwwauth header empty continuat
 
 	# Basic base64(alice:secret-passwd)
 	cat >"$HTTPD_ROOT_PATH/custom-auth.valid" <<-EOF &&
-	Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==
+	id=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==
 	EOF
 
 	CHALLENGE="$HTTPD_ROOT_PATH/custom-auth.challenge" &&
 
 	# Note that leading and trailing whitespace is important to correctly
 	# simulate a continuation/folded header.
-	printf "WWW-Authenticate: FooBar param1=\"value1\"\r\n" >"$CHALLENGE" &&
+	printf "id=1 status=200\n" >"$CHALLENGE" &&
-	printf " \r\n" >>"$CHALLENGE" &&
+	printf "id=default response=WWW-Authenticate: FooBar param1=\"value1\"\r\n" >>"$CHALLENGE" &&
-	printf " param2=\"value2\"\r\n" >>"$CHALLENGE" &&
+	printf "id=default response= \r\n" >>"$CHALLENGE" &&
-	printf "WWW-Authenticate: Bearer authorize_uri=\"id.example.com\"\r\n" >>"$CHALLENGE" &&
+	printf "id=default response= param2=\"value2\"\r\n" >>"$CHALLENGE" &&
-	printf " p=1\r\n" >>"$CHALLENGE" &&
+	printf "id=default response=WWW-Authenticate: Bearer authorize_uri=\"id.example.com\"\r\n" >>"$CHALLENGE" &&
-	printf " \r\n" >>"$CHALLENGE" &&
+	printf "id=default response= p=1\r\n" >>"$CHALLENGE" &&
-	printf " q=0\r\n" >>"$CHALLENGE" &&
+	printf "id=default response= \r\n" >>"$CHALLENGE" &&
-	printf "WWW-Authenticate: Basic realm=\"example.com\"\r\n" >>"$CHALLENGE" &&
+	printf "id=default response= q=0\r\n" >>"$CHALLENGE" &&
+	printf "id=default response=WWW-Authenticate: Basic realm=\"example.com\"\r\n" >>"$CHALLENGE" &&
 
 	test_config_global credential.helper test-helper &&
 	git ls-remote "$HTTPD_URL/custom_auth/repo.git" &&
 
 	expect_credential_query get <<-EOF &&
+	capability[]=authtype
+	capability[]=state
 	protocol=http
 	host=$HTTPD_DEST
 	wwwauth[]=FooBar param1="value1" param2="value2"
@@ -296,22 +363,25 @@ test_expect_success 'access using basic auth with wwwauth header mixed line-endi
 
 	# Basic base64(alice:secret-passwd)
 	cat >"$HTTPD_ROOT_PATH/custom-auth.valid" <<-EOF &&
-	Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==
+	id=1 creds=Basic YWxpY2U6c2VjcmV0LXBhc3N3ZA==
 	EOF
 
 	CHALLENGE="$HTTPD_ROOT_PATH/custom-auth.challenge" &&
 
 	# Note that leading and trailing whitespace is important to correctly
 	# simulate a continuation/folded header.
-	printf "WWW-Authenticate: FooBar param1=\"value1\"\r\n" >"$CHALLENGE" &&
+	printf "id=1 status=200\n" >"$CHALLENGE" &&
-	printf " \r\n" >>"$CHALLENGE" &&
+	printf "id=default response=WWW-Authenticate: FooBar param1=\"value1\"\r\n" >>"$CHALLENGE" &&
-	printf "\tparam2=\"value2\"\r\n" >>"$CHALLENGE" &&
+	printf "id=default response= \r\n" >>"$CHALLENGE" &&
-	printf "WWW-Authenticate: Basic realm=\"example.com\"" >>"$CHALLENGE" &&
+	printf "id=default response=\tparam2=\"value2\"\r\n" >>"$CHALLENGE" &&
+	printf "id=default response=WWW-Authenticate: Basic realm=\"example.com\"" >>"$CHALLENGE" &&
 
 	test_config_global credential.helper test-helper &&
 	git ls-remote "$HTTPD_URL/custom_auth/repo.git" &&
 
 	expect_credential_query get <<-EOF &&
+	capability[]=authtype
+	capability[]=state
 	protocol=http
 	host=$HTTPD_DEST
 	wwwauth[]=FooBar param1="value1" param2="value2"
@@ -326,4 +396,166 @@ test_expect_success 'access using basic auth with wwwauth header mixed line-endi
 	EOF
 '
 
+test_expect_success 'access using bearer auth' '
+	test_when_finished "per_test_cleanup" &&
+
+	set_credential_reply get <<-EOF &&
+	capability[]=authtype
+	authtype=Bearer
+	credential=YS1naXQtdG9rZW4=
+	EOF
+
+	# Basic base64(a-git-token)
+	cat >"$HTTPD_ROOT_PATH/custom-auth.valid" <<-EOF &&
+	id=1 creds=Bearer YS1naXQtdG9rZW4=
+	EOF
+
+	CHALLENGE="$HTTPD_ROOT_PATH/custom-auth.challenge" &&
+
+	cat >"$HTTPD_ROOT_PATH/custom-auth.challenge" <<-EOF &&
+	id=1 status=200
+	id=default response=WWW-Authenticate: FooBar param1="value1" param2="value2"
+	id=default response=WWW-Authenticate: Bearer authorize_uri="id.example.com" p=1 q=0
+	id=default response=WWW-Authenticate: Basic realm="example.com"
+	EOF
+
+	test_config_global credential.helper test-helper &&
+	git ls-remote "$HTTPD_URL/custom_auth/repo.git" &&
+
+	expect_credential_query get <<-EOF &&
+	capability[]=authtype
+	capability[]=state
+	protocol=http
+	host=$HTTPD_DEST
+	wwwauth[]=FooBar param1="value1" param2="value2"
+	wwwauth[]=Bearer authorize_uri="id.example.com" p=1 q=0
+	wwwauth[]=Basic realm="example.com"
+	EOF
+
+	expect_credential_query store <<-EOF
+	capability[]=authtype
+	authtype=Bearer
+	credential=YS1naXQtdG9rZW4=
+	protocol=http
+	host=$HTTPD_DEST
+	EOF
+'
+
+test_expect_success 'access using bearer auth with invalid credentials' '
+	test_when_finished "per_test_cleanup" &&
+
+	set_credential_reply get <<-EOF &&
+	capability[]=authtype
+	authtype=Bearer
+	credential=incorrect-token
+	EOF
+
+	# Basic base64(a-git-token)
+	cat >"$HTTPD_ROOT_PATH/custom-auth.valid" <<-EOF &&
+	id=1 creds=Bearer YS1naXQtdG9rZW4=
+	EOF
+
+	CHALLENGE="$HTTPD_ROOT_PATH/custom-auth.challenge" &&
+
+	cat >"$HTTPD_ROOT_PATH/custom-auth.challenge" <<-EOF &&
+	id=1 status=200
+	id=default response=WWW-Authenticate: FooBar param1="value1" param2="value2"
+	id=default response=WWW-Authenticate: Bearer authorize_uri="id.example.com" p=1 q=0
+	id=default response=WWW-Authenticate: Basic realm="example.com"
+	EOF
+
+	test_config_global credential.helper test-helper &&
+	test_must_fail git ls-remote "$HTTPD_URL/custom_auth/repo.git" &&
+
+	expect_credential_query get <<-EOF &&
+	capability[]=authtype
+	capability[]=state
+	protocol=http
+	host=$HTTPD_DEST
+	wwwauth[]=FooBar param1="value1" param2="value2"
+	wwwauth[]=Bearer authorize_uri="id.example.com" p=1 q=0
+	wwwauth[]=Basic realm="example.com"
+	EOF
+
+	expect_credential_query erase <<-EOF
+	capability[]=authtype
+	authtype=Bearer
+	credential=incorrect-token
+	protocol=http
+	host=$HTTPD_DEST
+	wwwauth[]=FooBar param1="value1" param2="value2"
+	wwwauth[]=Bearer authorize_uri="id.example.com" p=1 q=0
+	wwwauth[]=Basic realm="example.com"
+	EOF
+'
+
+test_expect_success 'access using three-legged auth' '
+	test_when_finished "per_test_cleanup" &&
+
+	set_credential_reply get <<-EOF &&
+	capability[]=authtype
+	capability[]=state
+	authtype=Multistage
+	credential=YS1naXQtdG9rZW4=
+	state[]=helper:foobar
+	continue=1
+	EOF
+
+	set_credential_reply get foobar <<-EOF &&
+	capability[]=authtype
+	capability[]=state
+	authtype=Multistage
+	credential=YW5vdGhlci10b2tlbg==
+	state[]=helper:bazquux
+	EOF
+
+	cat >"$HTTPD_ROOT_PATH/custom-auth.valid" <<-EOF &&
+	id=1 creds=Multistage YS1naXQtdG9rZW4=
+	id=2 creds=Multistage YW5vdGhlci10b2tlbg==
+	EOF
+
+	CHALLENGE="$HTTPD_ROOT_PATH/custom-auth.challenge" &&
+
+	cat >"$HTTPD_ROOT_PATH/custom-auth.challenge" <<-EOF &&
+	id=1 status=401 response=WWW-Authenticate: Multistage challenge="456"
+	id=1 status=401 response=WWW-Authenticate: Bearer authorize_uri="id.example.com" p=1 q=0
+	id=2 status=200
+	id=default response=WWW-Authenticate: Multistage challenge="123"
+	id=default response=WWW-Authenticate: Bearer authorize_uri="id.example.com" p=1 q=0
+	EOF
+
+	test_config_global credential.helper test-helper &&
+	git ls-remote "$HTTPD_URL/custom_auth/repo.git" &&
+
+	expect_credential_query get <<-EOF &&
+	capability[]=authtype
+	capability[]=state
+	protocol=http
+	host=$HTTPD_DEST
+	wwwauth[]=Multistage challenge="123"
+	wwwauth[]=Bearer authorize_uri="id.example.com" p=1 q=0
+	EOF
+
+	expect_credential_query get foobar <<-EOF &&
+	capability[]=authtype
+	capability[]=state
+	authtype=Multistage
+	protocol=http
+	host=$HTTPD_DEST
+	wwwauth[]=Multistage challenge="456"
+	wwwauth[]=Bearer authorize_uri="id.example.com" p=1 q=0
+	state[]=helper:foobar
+	EOF
+
+	expect_credential_query store bazquux <<-EOF
+	capability[]=authtype
+	capability[]=state
+	authtype=Multistage
+	credential=YW5vdGhlci10b2tlbg==
+	protocol=http
+	host=$HTTPD_DEST
+	state[]=helper:bazquux
+	EOF
+'
+
 test_done