Getting a node's available disk space

[niels-vd-k]

Heya!

For a project I'm working on, I need to get information about how much available disk space each node has. After some research I found that I could get this information for a single node by running kubectl get --raw "/api/v1/nodes/$(hostname)/proxy/stats/summary" | jq .node.fs in bash, but I'm having a hard time figuring out how to call this API from my C# application. I would've expected this to be part of the V1Node objects that I get from client.CoreV1.ListNodeAsync() but either I'm overlooking it, or it's just not in there. I've only been able to find the available ephemeral storage, but that's not what I'm looking for.

Does anyone know how I can retrieve this info? I'd rather not use HttpClient if I can avoid it.

Thanks in advance.

[lasse-tarp-CA]

I run the "df" command inside a pod to get the information. The outside call goes to "CommandDf". In this example I locate diskspace in a timeseries database running in Azure AKS:

var kubectlService = new KubectlService();
var df = await kubectlService.CommandDf("default", "questdb-0", "questdb");

What you get from the "df" command is something like:
Filesystem 1K-blocks Used Available Use% Mounted on
overlay 129886128 25545208 104324536 20% /
tmpfs 65536 0 65536 0% /dev
/dev/root 129886128 25545208 104324536 20% /etc/hosts

/// <summary>
/// Kubectl service
/// </summary>
public class KubectlService
{
    private KubernetesClientConfiguration _config { get; init; }
    private IKubernetes _client { get; init; }

    /// <summary>
    /// Constructor that uses default config from computer
    /// </summary>
    public KubectlService()
    {
        _config = KubernetesClientConfiguration.BuildDefaultConfig();
        _client = new Kubernetes(_config);
    }

    /// <summary>
    /// Constructor that uses config file
    /// </summary>
    /// <param name="configFilePath"></param>
    public KubectlService(string configFilePath)
    {

        _config = KubernetesClientConfiguration.BuildConfigFromConfigFile(configFilePath);
        _client = new Kubernetes(_config);
    }

    /// <summary>
    /// Get pods from kubernetes
    /// </summary>
    /// <param name="namespace">Namespace in kubernetes</param>
    /// <returns>List of pods</returns>
    public async Task<V1PodList> GetPodsAsync(string @namespace = "default")
    {
        return await _client.CoreV1.ListNamespacedPodAsync(@namespace);
    }

    /// <summary>
    /// Get pod from kubernetes based on name and namespace
    /// </summary>
    /// <param name="namespace">Namespace in kubernetes</param>
    /// <param name="podNameStartsWith">Pod name start text</param>
    /// <returns>Will return first pod that starts with podNameStartsWith</remarks>
    public async Task<V1Pod?> GetPodAsync(string @namespace, string podNameStartsWith)
    {
        var pods = await GetPodsAsync(@namespace);
        if (pods.Items == null)
            return null;
        return pods.Items.FirstOrDefault(p => p.Metadata.Name.StartsWith(podNameStartsWith));
    }

    /// <summary>
    /// Send df command to pod and get file info
    /// </summary>
    /// <param name="container">Container name</param>
    /// <param name="namespace">Namespace in kubernetes</param>
    /// <param name="podNameStartsWith">Pod name start text</param>
    public async Task<List<KubeFileInfo>?> CommandDf(string @namespace, string podNameStartsWith, string? container)
    {
        string? raw = await Command(@namespace, podNameStartsWith, "df", container);
        if (raw != null)
        {
            var lines = raw.SplitNewLine();
            List<KubeFileInfo> files = new();
            foreach (var line in lines)
            {
                var parts = line.Split(' ', StringSplitOptions.RemoveEmptyEntries);
                if (parts.Length == 6)
                {
                    files.Add(new KubeFileInfo
                    {
                        FileSystem = parts[0],
                        Size = Int64.Parse(parts[1]),
                        Used = Int64.Parse(parts[2]),
                        Available = Int64.Parse(parts[3]),
                        UsePercentage = parts[4],
                        MountedOn = parts[5]
                    });
                }
            }
            return files;
        }
        return [];
    }

    private async Task<string?> Command(string @namespace, string podNameStartsWith, string command, string? container)
    {
        var pod = await GetPodAsync(@namespace, podNameStartsWith);
        if (pod == null)
            return null;
        var webSocket = await _client.WebSocketNamespacedPodExecAsync(pod!.Metadata.Name, @namespace, command, container);
        var demux = new StreamDemuxer(webSocket);
        demux.Start();
        var stream = demux.GetStream(1, 1);
        var buff = new byte[4096];
        List<byte> chars = [];
        // Read all data from stream
        while (await stream.ReadAsync(buff, 0, buff.Length) > 0)
        {
            chars.AddRange(buff);
        }
        return GlobalSharedStringHelpers.GetString(chars.Where(c => c != '\0').ToArray());
    }
}

Note that some support methods are located in class GlobalSharedStringHelpers

/// <summary>
    /// Convert byte array to string using UTF8
    /// </summary>
    /// <param name="value">Array of bytes to convert.</param>
    /// <returns>UTF8 converted string.</returns>
    public static string GetString(this byte[] value) => Encoding.UTF8.GetString(value);

/// <summary>
    /// Split string from both \r\n and \n (windows and unix new line)
    /// </summary>
    /// <param name="value">Value to split</param>
    /// <returns>Empty or array with values.</returns>
    public static string[] SplitNewLine(this string? value)
    {
        if (value is null)
            return Array.Empty<string>();
        return value.Split(GlobalShared.NewLine, StringSplitOptions.RemoveEmptyEntries);
    }
/// <summary>New line array supporting both Windows and Linux</summary>
    public static string[] NewLine { get; } = new string[] { "\r\n", "\n" };

I return this class.

public class KubeFileInfo
{
    /// <summary>
    /// Gets or sets the file system name.
    /// </summary>
    public string FileSystem { get; set; } = null!;

    /// <summary>
    /// Gets or sets the size of the file.
    /// </summary>
    public long Size { get; set; }

    /// <summary>
    /// Gets or sets the amount of used space in the file system.
    /// </summary>
    public long Used { get; set; }

    /// <summary>
    /// Gets or sets the amount of available space in the file system.
    /// </summary>
    public long Available { get; set; }

    /// <summary>
    /// Gets or sets the percentage of space used in the file system.
    /// </summary>
    public string UsePercentage { get; set; } = null!;

    /// <summary>
    /// Gets or sets the mount point of the file system.
    /// </summary>
    public string MountedOn { get; set; } = null!;
}

I hope it is what you are looking for 🙂

[niels-vd-k]

That seems to work. However it does require me to grant exec permissions to the pod that's running my application, which isn't great for security and I feel like my team isn't going to like it. Really sounds like HttpClient might be my only option after all. Thanks anyway!

[tg123]

seems node spec do not have it today