Sorting files working locally but not on kubernetes

10/22/2020

I have about 1000 files with a date in their name, i would like to sort them by this date in the filename and pick the latest one which has a date same or earlier than an argument.
I have wrote this:

 Pattern PATTERN = Pattern.compile("^\\d{4}-\\d{2}-\\d{2}-file.csv");
    try {
      deviceFiles = Files.list(filesDir.toPath())
          .filter(path -> PATTERN.matcher(path.getFileName().toString()).matches()
          && !getDate(path).isAfter(ARGUMENT_DATE))
          .collect(Collectors.toList());
      Arrays.sort(deviceFiles.toArray(), new FileNamePathDateComparator());
      logger.info("All files found are " + deviceFiles.stream().map(stream -> stream.toAbsolutePath().toString()).collect(Collectors.joining("\n")));
      if (deviceFiles.isEmpty())
        throw new IllegalStateException("There were no device files found");
      else {
        String deviceFilePath = deviceFiles.get(deviceFiles.size() - 1).toAbsolutePath().toString();
        logger.info("Found device file: " + deviceFilePath);
        return deviceFilePath;
      }
    } catch (IOException e) {
      throw new UncheckedIOException(e);
    }

The getDate method:

 private LocalDate getDate(Path path)
  {
    try {
      String[] parts = path.getFileName().toString().split("-");
      return LocalDate.of(Integer.parseInt(parts[0]), Integer.parseInt(parts[1]), Integer.parseInt(parts[2]));
    } catch (NumberFormatException ex) {
      throw new IllegalArgumentException("bye", ex);
    }
  }

The comparator:

 class FileNamePathDateComparator implements Comparator{
    @Override
    public int compare(Object o1, Object o2)
    {
      return getDate((Path)o1).compareTo(getDate((Path)o2));
    }
  }

When i run this locally i see that the logger prints all the files correctly sorted, the comparator works just fine.<br> But on a kubernetes cluster the files are printed randomly, i dont understand this.

-- JBoy
java
kubernetes

1 Answer

10/22/2020

Fixed! I have put the comparator in the stream rather than in the final list, and it works fine.<br> If in the mean while someone can provide an explanation, i would appreciate.<br>

 deviceFiles = Files.list(filesDir.toPath())
      .filter(path -> PATTERN.matcher(path.getFileName().toString()).matches()
      && !getDate(path).isAfter(executionDate))
      .sorted(new FileNamePathDateComparator()::compare)
      .collect(Collectors.toList());
-- JBoy
Source: StackOverflow