zio remote cluster authentication and pod listing

[MohammadForouhesh]

Hi @hnaderi, documentation for listing pods (simplest example) in zio when we have a remote cluster is missing. the following code was the best I came up with.

object Main extends ZIOAppDefault {
  def run = {
    val token = "sha256~xxxx"

    val namespace = "my-namespace"
    val host = "https://cluster_url:443"
    val auth = AuthenticationParams.bearer(token)

    val client = ZClient.default
    val backend = ZIOBackend.make
    val k8sLayer = ZLayer.fromFunction(HttpClient[Task](host, _, auth))
    (for {
      k8s <- ZIO.service[HttpClient[Task]]
      x <- k8s.send(PodAPI(namespace).list)
      _ <- ZIO.logInfo(x.toString)
    } yield ())
      .provide(
        client,
        backend,
        k8sLayer
      )
}

and I encounter the following error:

timestamp=2023-12-05 level=ERROR thread=#zio-fiber-1 message="" cause="Exception in thread "zio-fiber-4" dev.hnaderi.k8s.client.ErrorResponse: Request failed!
status: Other(403)
details: Status(Status,v1,Some(Forbidden),Some(403),Some(Failure),Some(StatusDetails(None,None,None,Some(pods),None,None)),Some(ListMeta(None,None,None,None)),Some(pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" in the namespace "my-namespace"))

        at dev.hnaderi.k8s.client.ZIOBackend.$anonfun$expect$5(ZIOBackend.scala:137)
        at zio.ZIO.$anonfun$map$2(ZIO.scala:960)
        at dev.hnaderi.k8s.client.ZIOBackend.expect(ZIOBackend.scala:137)
        at testK8s.Main.run.client(main.scala:50)
        at testK8s.Main.run(main.scala:65)"

any help will be highly appreciated.

[github-actions]

Thank you for submitting this issue and welcome!
We are glad that you are here and we appreciate your contributions.

[hnaderi]

Hi @MohammadForouhesh , ZIO module uses ZIO http, which lacks the TLS support required for connecting to kubernetes directly and the 403 error is because of this.

There are a few different ways you can resolve this problem:

1. Use other modules to connect directly, as the only module that doesn't support TLS is the ZIO module. I recommend http4s if you want a reliable, mature and multi-platform client. You can also use sttp if all you want is simple scripting.
2. Note that if you want to use ZIO as the effect system, you can use http4s module as well, as it is based on cats-effect and is designed in a tagless final way, so you can use any effect system. For using with zio you can use ZIO interop module for cats effect.
3. You can use kubectl proxy to get a new server locally that handles the underlying TLS connection and authentication, and then use ZIO client on top of it.

Hope this helps! Let me know if there are any problems or questions.

[MohammadForouhesh]

Thanks @hnaderi, for your timely response. Since we will deploy it on the same cluster at the end of the day, and it will work with its service account, can you provide me with some insight on that? and also, can you please further elaborate on your 3rd solution? It would be highly appreciated if you provide me with a working code of your 2nd solution.

[hnaderi]

If you want to access kubernetes API using service account credentials, the easiest approach would be number 1 or number 2, as the http4s client supports automatic detection of credentials from pod or kubeconfig or from environment variables, exactly like how kubectl works.

For third solution, you need to run kubectl proxy and use create the ZIO client like this:

ZIOKubernetesClient.make("http://localhost:8001")

See here for complete example.

For second solution, I will send an example later today, or probably by tomorrow.

[hnaderi]

@MohammadForouhesh

Here's a working example using ZIO, ZIO interop, http4s; that gets a node list.

/*
 * Copyright 2021 Hossein Naderi
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

//> using scala 3.3
//> using dep "io.circe::circe-generic:0.14.6"
//> using dep "dev.hnaderi::scala-k8s-http4s-ember:0.15.2"
//> using dep "dev.hnaderi::scala-k8s-circe:0.15.2"
//> using dep "org.http4s::http4s-circe:0.23.24"
//> using dep "dev.zio::zio:2.0.19"
//> using dep "dev.zio::zio-interop-cats:23.1.0.0"

package example

import cats.effect._
import cats.effect.std.Env
import zio._
import zio.Console._
import zio.interop.catz._
import dev.hnaderi.k8s.circe._
import dev.hnaderi.k8s.client._
import dev.hnaderi.k8s.client.http4s.EmberKubernetesClient
import io.circe.Json
import org.http4s.circe._

object App extends ZIOAppDefault {
  given Env[Task] = new {
    def get(name: String) = ZIO.succeed(sys.env.get(name))
    def entries = ZIO.succeed(sys.env)
  }
  val buildClient = EmberKubernetesClient[Task].defaultConfig[Json]

  def run = buildClient.use(APIs.nodes.list().send).flatMap(nl=> printLine(nl.toString()))
}

You can run it directly using scala-cli

[MohammadForouhesh]

thanks a lot, it really helped