Pipe Broken with PipeInputStream with kubernetes-client exec()

6/6/2017

I'm using the kubernetes-client to try copy a directory from a pod, but I'm doing something wrong with the input stream from stdout. I get a java.io.IOException: Pipe broken exception when it tries to read(). I'm pretty sure that no data flows at all. I'm half wondering if I need to read the InputStream on a separate thread or something?

The stream is created like this:

public InputStream copyFiles(String containerId,
                             String folderName) {

    ExecWatch exec = client.pods().withName(containerId).redirectingOutput().exec("tar -C " + folderName + " -c");

    // We need to wrap the InputStream so that when the stdout is closed, then the underlying ExecWatch is closed
    // also. This will cleanup any Websockets connections.
    ChainedCloseInputStreamWrapper inputStreamWrapper = new ChainedCloseInputStreamWrapper(exec.getOutput(), exec);

    return inputStreamWrapper;
}

And the InputStream is processed in this function

void copyVideos(final String containerId) {
    TarArchiveInputStream tarStream = new TarArchiveInputStream(containerClient.copyFiles(containerId, "/videos/"));
    TarArchiveEntry entry;
    boolean videoWasCopied = false;
    try {
        while ((entry = tarStream.getNextTarEntry()) != null) {
            if (entry.isDirectory()) {
                continue;
            }
            String fileExtension = entry.getName().substring(entry.getName().lastIndexOf('.'));
            testInformation.setFileExtension(fileExtension);
            File videoFile = new File(testInformation.getVideoFolderPath(), testInformation.getFileName());
            File parent = videoFile.getParentFile();
            if (!parent.exists()) {
                parent.mkdirs();
            }
            OutputStream outputStream = new FileOutputStream(videoFile);
            IOUtils.copy(tarStream, outputStream);
            outputStream.close();
            videoWasCopied = true;
            LOGGER.log(Level.INFO, "{0} Video file copied to: {1}/{2}", new Object[]{getId(),
                    testInformation.getVideoFolderPath(), testInformation.getFileName()});
        }
    } catch (IOException e) {
        LOGGER.log(Level.WARNING, getId() + " Error while copying the video", e);
        ga.trackException(e);
    } finally {
        if (!videoWasCopied) {
            testInformation.setVideoRecorded(false);
        }
    }

}

The InputStream Wrapper class is just there to close the ExecWatch at the end once the InputStream is closed, it looks like this:

private static class ChainedCloseInputStreamWrapper extends InputStream {

    private InputStream delegate;
    private Closeable resourceToClose;

    public ChainedCloseInputStreamWrapper(InputStream delegate, Closeable resourceToClose) {
        this.delegate = delegate;
        this.resourceToClose = resourceToClose;
    }

    @Override
    public int read() throws IOException {
        return delegate.read();
    }

    public int available() throws IOException {
        return delegate.available();
    }

    public void close() throws IOException {
        logger.info("Shutdown called!");
        delegate.close();

        // Close our dependent resource
        resourceToClose.close();
    }

    public boolean equals(Object o) {
        return delegate.equals(o);
    }

    public int hashCode() {
        return delegate.hashCode();
    }

    public int read(byte[] array) throws IOException {
        return delegate.read(array);
    }

    public int read(byte[] array,
                    int n,
                    int n2) throws IOException {
        return delegate.read(array, n, n2);
    }

    public long skip(long n) throws IOException {
        return delegate.skip(n);
    }

    public void mark(int n) {
        delegate.mark(n);
    }

    public void reset() throws IOException {
        delegate.reset();
    }

    public boolean markSupported() {
        return delegate.markSupported();
    }

    public String toString() {
        return delegate.toString();
    }

}
-- Joel Pearson
fabric8
java
kubernetes

1 Answer

6/7/2017

Turns out I had the tar command wrong, so it was causing a failure and the stdout PipeInputStream was dead locking. I managed to find a workaround for the deadlock. But the main reason for the failure was that I forgot to tell tar to actually do something! I at least needed a "." to include the current directory.

-- Joel Pearson
Source: StackOverflow