cmd/k8s-operator,k8s-operator,kube/kubetypes: add an option to configure app connector via Connector spec (#13950)

* cmd/k8s-operator,k8s-operator,kube/kubetypes: add an option to configure app connector via Connector spec

Updates tailscale/tailscale#11113

Signed-off-by: Irbe Krumina <irbe@tailscale.com>

k8s-operator/api.md

@@ -21,6 +21,22 @@
 
 
 
+#### AppConnector
+
+
+
+AppConnector defines a Tailscale app connector node configured via Connector.
+
+
+
+_Appears in:_
+- [ConnectorSpec](#connectorspec)
+
+| Field | Description | Default | Validation |
+| --- | --- | --- | --- |
+| `routes` _[Routes](#routes)_ | Routes are optional preconfigured routes for the domains routed via the app connector.<br />If not set, routes for the domains will be discovered dynamically.<br />If set, the app connector will immediately be able to route traffic using the preconfigured routes, but may<br />also dynamically discover other routes.<br />https://tailscale.com/kb/1332/apps-best-practices#preconfiguration |  | Format: cidr <br />MinItems: 1 <br />Type: string <br /> |
+
+
 
 
 #### Connector
@@ -86,8 +102,9 @@ _Appears in:_
 | `tags` _[Tags](#tags)_ | Tags that the Tailscale node will be tagged with.<br />Defaults to [tag:k8s].<br />To autoapprove the subnet routes or exit node defined by a Connector,<br />you can configure Tailscale ACLs to give these tags the necessary<br />permissions.<br />See https://tailscale.com/kb/1337/acl-syntax#autoapprovers.<br />If you specify custom tags here, you must also make the operator an owner of these tags.<br />See  https://tailscale.com/kb/1236/kubernetes-operator/#setting-up-the-kubernetes-operator.<br />Tags cannot be changed once a Connector node has been created.<br />Tag values must be in form ^tag:[a-zA-Z][a-zA-Z0-9-]*$. |  | Pattern: `^tag:[a-zA-Z][a-zA-Z0-9-]*$` <br />Type: string <br /> |
 | `hostname` _[Hostname](#hostname)_ | Hostname is the tailnet hostname that should be assigned to the<br />Connector node. If unset, hostname defaults to <connector<br />name>-connector. Hostname can contain lower case letters, numbers and<br />dashes, it must not start or end with a dash and must be between 2<br />and 63 characters long. |  | Pattern: `^[a-z0-9][a-z0-9-]{0,61}[a-z0-9]$` <br />Type: string <br /> |
 | `proxyClass` _string_ | ProxyClass is the name of the ProxyClass custom resource that<br />contains configuration options that should be applied to the<br />resources created for this Connector. If unset, the operator will<br />create resources with the default configuration. |  |  |
-| `subnetRouter` _[SubnetRouter](#subnetrouter)_ | SubnetRouter defines subnet routes that the Connector node should<br />expose to tailnet. If unset, none are exposed.<br />https://tailscale.com/kb/1019/subnets/ |  |  |
-| `exitNode` _boolean_ | ExitNode defines whether the Connector node should act as a<br />Tailscale exit node. Defaults to false.<br />https://tailscale.com/kb/1103/exit-nodes |  |  |
+| `subnetRouter` _[SubnetRouter](#subnetrouter)_ | SubnetRouter defines subnet routes that the Connector device should<br />expose to tailnet as a Tailscale subnet router.<br />https://tailscale.com/kb/1019/subnets/<br />If this field is unset, the device does not get configured as a Tailscale subnet router.<br />This field is mutually exclusive with the appConnector field. |  |  |
+| `appConnector` _[AppConnector](#appconnector)_ | AppConnector defines whether the Connector device should act as a Tailscale app connector. A Connector that is<br />configured as an app connector cannot be a subnet router or an exit node. If this field is unset, the<br />Connector does not act as an app connector.<br />Note that you will need to manually configure the permissions and the domains for the app connector via the<br />Admin panel.<br />Note also that the main tested and supported use case of this config option is to deploy an app connector on<br />Kubernetes to access SaaS applications available on the public internet. Using the app connector to expose<br />cluster workloads or other internal workloads to tailnet might work, but this is not a use case that we have<br />tested or optimised for.<br />If you are using the app connector to access SaaS applications because you need a predictable egress IP that<br />can be whitelisted, it is also your responsibility to ensure that cluster traffic from the connector flows<br />via that predictable IP, for example by enforcing that cluster egress traffic is routed via an egress NAT<br />device with a static IP address.<br />https://tailscale.com/kb/1281/app-connectors |  |  |
+| `exitNode` _boolean_ | ExitNode defines whether the Connector device should act as a Tailscale exit node. Defaults to false.<br />This field is mutually exclusive with the appConnector field.<br />https://tailscale.com/kb/1103/exit-nodes |  |  |
 
 
 #### ConnectorStatus
@@ -106,6 +123,7 @@ _Appears in:_
 | `conditions` _[Condition](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.3/#condition-v1-meta) array_ | List of status conditions to indicate the status of the Connector.<br />Known condition types are `ConnectorReady`. |  |  |
 | `subnetRoutes` _string_ | SubnetRoutes are the routes currently exposed to tailnet via this<br />Connector instance. |  |  |
 | `isExitNode` _boolean_ | IsExitNode is set to true if the Connector acts as an exit node. |  |  |
+| `isAppConnector` _boolean_ | IsAppConnector is set to true if the Connector acts as an app connector. |  |  |
 | `tailnetIPs` _string array_ | TailnetIPs is the set of tailnet IP addresses (both IPv4 and IPv6)<br />assigned to the Connector node. |  |  |
 | `hostname` _string_ | Hostname is the fully qualified domain name of the Connector node.<br />If MagicDNS is enabled in your tailnet, it is the MagicDNS name of the<br />node. |  |  |
 
@@ -746,6 +764,7 @@ _Validation:_
 - Type: string
 
 _Appears in:_
+- [AppConnector](#appconnector)
 - [SubnetRouter](#subnetrouter)
 
 

k8s-operator/apis/v1alpha1/types_connector.go

@@ -22,6 +22,7 @@ var ConnectorKind = "Connector"
 // +kubebuilder:resource:scope=Cluster,shortName=cn
 // +kubebuilder:printcolumn:name="SubnetRoutes",type="string",JSONPath=`.status.subnetRoutes`,description="CIDR ranges exposed to tailnet by a subnet router defined via this Connector instance."
 // +kubebuilder:printcolumn:name="IsExitNode",type="string",JSONPath=`.status.isExitNode`,description="Whether this Connector instance defines an exit node."
+// +kubebuilder:printcolumn:name="IsAppConnector",type="string",JSONPath=`.status.isAppConnector`,description="Whether this Connector instance is an app connector."
 // +kubebuilder:printcolumn:name="Status",type="string",JSONPath=`.status.conditions[?(@.type == "ConnectorReady")].reason`,description="Status of the deployed Connector resources."
 
 // Connector defines a Tailscale node that will be deployed in the cluster. The
@@ -55,7 +56,8 @@ type ConnectorList struct {
 }
 
 // ConnectorSpec describes a Tailscale node to be deployed in the cluster.
-// +kubebuilder:validation:XValidation:rule="has(self.subnetRouter) || self.exitNode == true",message="A Connector needs to be either an exit node or a subnet router, or both."
+// +kubebuilder:validation:XValidation:rule="has(self.subnetRouter) || (has(self.exitNode) && self.exitNode == true) || has(self.appConnector)",message="A Connector needs to have at least one of exit node, subnet router or app connector configured."
+// +kubebuilder:validation:XValidation:rule="!((has(self.subnetRouter) || (has(self.exitNode)  && self.exitNode == true)) && has(self.appConnector))",message="The appConnector field is mutually exclusive with exitNode and subnetRouter fields."
 type ConnectorSpec struct {
 	// Tags that the Tailscale node will be tagged with.
 	// Defaults to [tag:k8s].
@@ -82,13 +84,31 @@ type ConnectorSpec struct {
 	// create resources with the default configuration.
 	// +optional
 	ProxyClass string `json:"proxyClass,omitempty"`
-	// SubnetRouter defines subnet routes that the Connector node should
-	// expose to tailnet. If unset, none are exposed.
+	// SubnetRouter defines subnet routes that the Connector device should
+	// expose to tailnet as a Tailscale subnet router.
 	// https://tailscale.com/kb/1019/subnets/
+	// If this field is unset, the device does not get configured as a Tailscale subnet router.
+	// This field is mutually exclusive with the appConnector field.
 	// +optional
-	SubnetRouter *SubnetRouter `json:"subnetRouter"`
-	// ExitNode defines whether the Connector node should act as a
-	// Tailscale exit node. Defaults to false.
+	SubnetRouter *SubnetRouter `json:"subnetRouter,omitempty"`
+	// AppConnector defines whether the Connector device should act as a Tailscale app connector. A Connector that is
+	// configured as an app connector cannot be a subnet router or an exit node. If this field is unset, the
+	// Connector does not act as an app connector.
+	// Note that you will need to manually configure the permissions and the domains for the app connector via the
+	// Admin panel.
+	// Note also that the main tested and supported use case of this config option is to deploy an app connector on
+	// Kubernetes to access SaaS applications available on the public internet. Using the app connector to expose
+	// cluster workloads or other internal workloads to tailnet might work, but this is not a use case that we have
+	// tested or optimised for.
+	// If you are using the app connector to access SaaS applications because you need a predictable egress IP that
+	// can be whitelisted, it is also your responsibility to ensure that cluster traffic from the connector flows
+	// via that predictable IP, for example by enforcing that cluster egress traffic is routed via an egress NAT
+	// device with a static IP address.
+	// https://tailscale.com/kb/1281/app-connectors
+	// +optional
+	AppConnector *AppConnector `json:"appConnector,omitempty"`
+	// ExitNode defines whether the Connector device should act as a Tailscale exit node. Defaults to false.
+	// This field is mutually exclusive with the appConnector field.
 	// https://tailscale.com/kb/1103/exit-nodes
 	// +optional
 	ExitNode bool `json:"exitNode"`
@@ -104,6 +124,17 @@ type SubnetRouter struct {
 	AdvertiseRoutes Routes `json:"advertiseRoutes"`
 }
 
+// AppConnector defines a Tailscale app connector node configured via Connector.
+type AppConnector struct {
+	// Routes are optional preconfigured routes for the domains routed via the app connector.
+	// If not set, routes for the domains will be discovered dynamically.
+	// If set, the app connector will immediately be able to route traffic using the preconfigured routes, but may
+	// also dynamically discover other routes.
+	// https://tailscale.com/kb/1332/apps-best-practices#preconfiguration
+	// +optional
+	Routes Routes `json:"routes"`
+}
+
 type Tags []Tag
 
 func (tags Tags) Stringify() []string {
@@ -156,6 +187,9 @@ type ConnectorStatus struct {
 	// IsExitNode is set to true if the Connector acts as an exit node.
 	// +optional
 	IsExitNode bool `json:"isExitNode"`
+	// IsAppConnector is set to true if the Connector acts as an app connector.
+	// +optional
+	IsAppConnector bool `json:"isAppConnector"`
 	// TailnetIPs is the set of tailnet IP addresses (both IPv4 and IPv6)
 	// assigned to the Connector node.
 	// +optional

k8s-operator/apis/v1alpha1/zz_generated.deepcopy.go

@@ -13,6 +13,26 @@ import (
 	"k8s.io/apimachinery/pkg/runtime"
 )
 
+// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
+func (in *AppConnector) DeepCopyInto(out *AppConnector) {
+	*out = *in
+	if in.Routes != nil {
+		in, out := &in.Routes, &out.Routes
+		*out = make(Routes, len(*in))
+		copy(*out, *in)
+	}
+}
+
+// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AppConnector.
+func (in *AppConnector) DeepCopy() *AppConnector {
+	if in == nil {
+		return nil
+	}
+	out := new(AppConnector)
+	in.DeepCopyInto(out)
+	return out
+}
+
 // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
 func (in *Connector) DeepCopyInto(out *Connector) {
 	*out = *in
@@ -85,6 +105,11 @@ func (in *ConnectorSpec) DeepCopyInto(out *ConnectorSpec) {
 		*out = new(SubnetRouter)
 		(*in).DeepCopyInto(*out)
 	}
+	if in.AppConnector != nil {
+		in, out := &in.AppConnector, &out.AppConnector
+		*out = new(AppConnector)
+		(*in).DeepCopyInto(*out)
+	}
 }
 
 // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ConnectorSpec.