5-10% of Uppy/TUS uploads fail

[x] I confirm this is a bug with Supabase, not with my own application.
[x] I confirm I have searched the Docs, GitHub Discussions, and Discord.

I'm trying to use Supabase Storage with uploads via Uppy. Roughly 5-10% of my uploads fail, even though Uppy reports that the upload was completed. At this point, I'm actively looking for a more stable alternative. I've spent a couple of hours working around various issues, I'll try to describe everything I've encountered. I've used https://github.com/supabase/supabase/tree/master/examples/storage/resumable-upload-uppy as a base.
So when I upload a file via Uppy with TUS, Uppy reports that the upload was successful. Yet, when I subsequently try to access the upload file on my backed, the file is not found (resulting in the 500 seen on the screenshot).

API error from the screenshot:
Moving the file uploads/3637052f-8d83-4c to verified/9eaa91fd-da80-4dbc-96b0-83effe600e63
StorageApiError: Object not found
    at eval (webpack-internal:///(rsc)/./node_modules/.pnpm/@supabase+storage-js@2.5.5/node_modules/@supabase/storage-js/dist/module/lib/fetch.js:44:24)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  __isStorageError: true,
  status: 400
}

API code:
export const moveTemplateToVerified = async (
  tmpFileName: string,
  templateId: string
) => {
  console.log(
    `Moving the file uploads/${tmpFileName} to verified/${templateId}`
  );

  const { data, error } = await supabase.storage
    .from("templates")
    .move(`uploads/${tmpFileName}`, `verified/${templateId}.docx`);

  if (error) {
    throw error;
  }
};

This happens roughly 5-10% of the time. This is a big no-go for me and I'm currently looking for a more stable alternative. I believe this is likely due to TUS implementation issues, I'd appreciate a recommendation for a more stable API. At this point I'm wondering whether using pre-signed URL's for uploads + XHR uploads from the server will be more stable. There's also more stuff that's problematic:

every successful TUS upload involves 404 calls - this should not be a part of a successful data flow
It's impossible to reupload files - I have to rename the files during the upload, as otherwise we get a 409, which I didn't manage to get around with custom policies
if the file name/metadata doesn't match between supabase and uppy, the upload just fails with a 404 - pretty much impossible to debug, I've fixed this with trial & error
the UI for storage is super slow

Here's the upload component:
"use client";

import Uppy, { PluginOptions, UploadResult, UppyFile } from "@uppy/core";
import { Dashboard } from "@uppy/react";
import Tus from "@uppy/tus";
import { v4 as uuidv4 } from "uuid";

import<EMAIL_ADDRESS>import<EMAIL_ADDRESS>import { useState } from "react";

interface TemplateUploadProps {
  setTemplateName: (name: string) => void;
  setTmpUploadFileName: (name: string) => void;
}

const SUPABASE_ANON_KEY =
  "..."; // your project's anon key
const SUPABASE_PROJECT_ID = "..."; // your project ref
const STORAGE_BUCKET = "..."; // your storage bucket name

const supabaseStorageURL = `https://${SUPABASE_PROJECT_ID}.supabase.co/storage/v1/upload/resumable`;

const tusConfig = {
  endpoint: supabaseStorageURL,
  headers: {
    authorization: `Bearer ${SUPABASE_ANON_KEY}`,
    apikey: SUPABASE_ANON_KEY,
  },
  uploadDataDuringCreation: true,
  chunkSize: 6 * 1024 * 1024,
  allowedMetaFields: [
    "bucketName",
    "objectName",
    "contentType",
    "cacheControl",
  ],
  onerror: (error: any) => console.log("tus error", error),
};

export default function TemplateUpload({
  setTmpUploadFileName,
  setTemplateName,
}: TemplateUploadProps) {
  const [isUploadComplete, setIsUploadComplete] = useState(false);

  const generateFileName = () => {
    const uuid = uuidv4();
    const name = uuid.substring(0, 16);
    console.log("generated name", name);
    return name;
  };

  const renameFile = (currentFile: UppyFile, files: any) => {
    const modifiedFile = {
      ...currentFile,
      name: generateFileName(),
    };
    setTemplateName(currentFile.name);
    console.log("Uploading file: ", modifiedFile.name);
    return modifiedFile;
  };

  const handleUploadResult = (result: UploadResult) => {
    console.log("Upload result:", result);
    if (result.failed.length > 0) {
      console.log("Upload failed", result.failed);
      return;
    } else if (result.successful.length > 0) {
      setIsUploadComplete(true);
      setTmpUploadFileName(`${result.successful[0].name}`);
    }
  };

  // IMPORTANT: passing an initializer function to prevent Uppy from being reinstantiated on every render.
  const [uppy] = useState(() =>
    new Uppy({
      onBeforeFileAdded: renameFile,
      allowMultipleUploadBatches: false,
      restrictions: {
        maxFileSize: 10000000, // 10mb
        maxNumberOfFiles: 1,
        minNumberOfFiles: 1,
        // allowedFileTypes: ["application/pdf"],
      },
      autoProceed: true,
    }).use(Tus, tusConfig)
  );

  uppy.on("file-added", (file) => {
    const supabaseMetadata = {
      bucketName: STORAGE_BUCKET,
      objectName: `uploads/${file.name}`,
      contentType: file.type,
    };

    file.meta = {
      ...file.meta,
      ...supabaseMetadata,
    };

    console.log("file added", file);
  });

  // uppy.on("upload-success", (file, response) => {
  //   console.log("Upload success", file, response);
  // });

  uppy.on("complete", handleUploadResult);

  return (
    <>
      {isUploadComplete &&
Upload Complete!
}
      {!isUploadComplete && <Dashboard width={600} height={400} uppy={uppy} />}
    </>
  );
}

Hello @lmyslinski thanks a lot for your feedback!
Let's go step by step. Most of the issues are not real issues and how the protocol is supposed to work.

It's impossible to reupload files - I have to rename the files during the upload, as otherwise we get a 409, which I didn't manage to get around with custom policies

This is expected, since uploading a file on the same path (essentially overwriting a file) you are required to provide the x-upsert=true header, if you don't pass this header storage will not allow you to upload to a path where a file already exists.
However, we do strongly recommend using the pattern of creating a unique name for each of the files like you are doing, since overwriting files on the same path will not bypass  the browser cache and the CDN cache takes a little time to propagate the changes
See documentation: https://supabase.com/docs/guides/storage/uploads/resumable-uploads#overwriting-files

if the file name/metadata doesn't match between supabase and uppy, the upload just fails with a 404 - pretty much impossible to debug, I've fixed this with trial & error

Could you explain this further, not sure i understood this correctly

every successful TUS upload involves 404 calls - this should not be a part of a successful data flow

This is how the protocol works, when you start the upload of a file, tus creates a fingerprint and stores it in the local-storage, when you stop the upload and resume it later, TUS will re-use the same upload URL. it first sends a HEAD request to receive back the upload offset so that it can resume where it left-off. In case the upload is successful by default TUS doesn't remove that fingerprint from the local-storage see removeFingerprintOnSuccess, if you delete the file later on, it will try to call the same upload URL but it is now deleted so you'll receive a 404 and it will create a new upload url automatically.

So when I upload a file via Uppy with TUS, Uppy reports that the upload was successful. Yet, when I subsequently try to access the upload file on my backed, the file is not found

This issue is also caused by the flag removeFingerprintOnSuccess if not set to true it uses the default value false.
When the upload succeeds the browser still keeps the fingerprint of the file in the local-storage.
Due to a limitation of tus-node-server it is currently advised to set this value to true , this should solve all the stability problems you are currently having with Resumable upload, since once the browser completes the upload will not be holding that reference in the local-storage for future lookups.

This happens roughly 5-10% of the time. This is a big no-go for me and I'm currently looking for a more stable alternative. I believe this is likely due to TUS implementation issues, I'd appreciate a recommendation for a more stable API

The protocol implementation is a Beta feature in our current platform so there are currently small perks like the above.
However, we have contributed back to the TUS protocol a lot in the past months and we have improved stability dramatically, as a sneak-peak we will announce TUS to be out of beta very soon, entering a stable release.
We are also working on having TUS uploads embedded in the Supabase SDK so that we can set default values more consistently, (this will come a bit later the stable release)
for example:
await supbase.storage.from('bucket').resumableUpload('path', file, {
  onProgress: (uploaded, remaining) => {},
})

the UI for storage is super slow
Could you please send a support ticket i would be happy to look into this personally for you

Again, thanks a lot for the feedback.
I Hope with adding removefingerprintonsuccess: true and the upcoming stable release of TUS you'll have a great experience with Storage
Let me know if you have any more questions

@fenos Thanks for the detailed response. I understand it's in beta - for now I've switched to XHR which works pretty well. I'll soon need to add bulk upload support so I'll give it another shot with removeFingerprintOnSuccess

Awesome!
Closing for now, i'll update this issue when we release TUS stable release.
However, feel free to also update me here for anything else. In case we'll just re-open it.
Thanks @lmyslinski

@lmyslinski resumable upload is now considered stable! 🎉 All the issues above mentioned are now resolved

@lmyslinski resumable upload is now considered stable! 🎉 All the issues above mentioned are now resolved

Hi im using "@uppy/core": "^3.9.2",. Im experiencing what lmyslinski explained. uppy is saying the image upload succeeded but the actual image is not being uploaded to supabase. I've noticed this is only happening to some images

Hi im using "@uppy/core": "^3.9.2",. Im experiencing what lmyslinski explained. uppy is saying the image upload succeeded but the actual image is not being uploaded to supabase. I've noticed this is only happening to some images

I ran in into this issue too, where the upload seems to succeed but nothing gets uploaded to Supabase. Clearing the cache/localstorage seems to fix the issue, for me. But not sure what's the cause exactly.

@lmyslinski resumable upload is now considered stable! 🎉 All the issues above mentioned are now resolved

Hi im using "@uppy/core": "^3.9.2",. Im experiencing what lmyslinski explained. uppy is saying the image upload succeeded but the actual image is not being uploaded to supabase. I've noticed this is only happening to some images

Also happens to me with uppy ^3.11.0

Hi im using "@uppy/core": "^3.9.2",. Im experiencing what lmyslinski explained. uppy is saying the image upload succeeded but the actual image is not being uploaded to supabase. I've noticed this is only happening to some images

I run in into this issue too, where the upload seems to succeed but nothing gets uploaded to Supabase. Clearing the cache/localstorage seems to fix the issue, for me. But not sure what's the cause exactly.

I think I found my issue. I'll document it here in case anyone comes across this.
Totally not an expert, so take it with a bit of salt.
I'm using the TUS client in combination with Supabase and React. It seems that Uppy stores some data about the file to upload in localStorage. In my case, the problem was that uploading the same file twice in separate Uppy instances would not create a new localStorage entry. This, in turn, meant that Uppy assumed this File was already uploaded and just returned successfully. This interfered with my application logic, where I actually want to allow uploading the same file twice and treat them as completely separate entities.
The fix, for me, was to overwrite the file.id for each file added to Uppy to give it a unique ID, which meant that a new localStorage entry would be created every time.
It looks something like this:
new Uppy({
  autoProceed: true,
  id: uppyId,
  debug: false,
  onBeforeFileAdded(currentFile) {
    const newItemId = crypto.randomUUID();
    const modifiedFile = {
      ...currentFile,
      id: `${currentFile.id}-${newItemId}`, // <--- the important part
    };

    return modifiedFile;
  },
})

The entierty of my Upload-Area code can be found below, in case anyone is curious.
Be warned, it's a bit wild (and probably wrong)
The entire example

type CustomMeta = {
  itemId?: UUID;
  bucketName?: string;
  objectName?: string;
  contentType?: string;
};

export function UploadArea({
  dispatch,
  tierlistData,
  pathname,
}: {
  dispatch: (action: tierListActions) => void;
  tierlistData: OptimisticTierlist;
  pathname: string;
}) {
  const supabase = createClient();

  // https://github.com/transloadit/uppy/issues/3185
  const tierlistDataRef = useRef<OptimisticTierlist>(tierlistData);

  // Update ref on every render
  useEffect(() => {
    tierlistDataRef.current = tierlistData;
  });

  const [uppy] = useState(() =>
    new Uppy<CustomMeta>({
      autoProceed: true,
      id: tierlistData.id,
      debug: false,
      onBeforeFileAdded(currentFile) {
        const newItemId = crypto.randomUUID();
        const modifiedFile = {
          ...currentFile,
          id: `${currentFile.id}-${newItemId}`, // Unique ID is important, so that same file can be used in different tierlists
        };

        return modifiedFile;
      },
    })
      .use<UppyCompressorOptions, Compressor>(Compressor, {
        width: 80,
        convertSize: MAX_FILE_SIZE, // If image is larger than 10MB, convert to JPEG
      })
      .use(ThumbnailGenerator, {
        thumbnailWidth: 80,
        thumbnailType: "image/png",
        waitForThumbnailsBeforeUpload: true, // This is important so that we have an up to date React State when uploading
      })
      .use(Tus, {
        endpoint: process.env.NEXT_PUBLIC_SUPABASE_STORAGE_UPLOAD_URL,
        headers: {
          Authorization: `Bearer ${process.env.NEXT_PUBLIC_SUPABASE_ANON_KEY}`,
          // "x-upsert": "true", // optionally set upsert to true to overwrite existing files (not recommended because CDN Cache takes longer to update)
        },
        uploadDataDuringCreation: true,
        onAfterResponse: function (req, res) {
          const url = req.getURL();
          const value = res.getStatus();
          if (res.getStatus() === 409) {
            console.info("⛷ File already exists, skipping image upload");
          } else if (res.getStatus() >= 400) {
            console.info(`🚨 Request for ${url} responded with ${value}`);
          }
        },
        // removeFingerprintOnSuccess: true, // Important to allow re-uploading the same file https://github.com/tus/tus-js-client/blob/main/docs/api.md#removefingerprintonsuccess
        chunkSize: 6 * 1024 * 1024, // NOTE: it must be set to 6MB (for now) do not change it https://supabase.com/docs/guides/storage/uploads/resumable-uploads
        allowedMetaFields: ["bucketName", "objectName", "contentType", "cacheControl", "name"],
      })
      .on("file-added", (file) => {
        const folder = tierlistData.id;

        const supabaseMetadata = {
          bucketName: ITEMS_STORAGE_BUCKET,
          objectName: folder ? `${folder}/${file.name}` : file.name,
          contentType: file.type,
        };

        file.meta = {
          ...file.meta,
          ...supabaseMetadata,
        };

        console.debug("1⃣ file added", file);
      })
      .on("thumbnail:generated", (file, preview) => {
        console.debug("2⃣ thumbnail generated", file, preview);
        const unsortedTier = tierlistDataRef.current.tiers.find(
          ({ unsorted_tier }) => unsorted_tier
        );

        if (file.meta.itemId && unsortedTier?.id) {
          dispatch({
            type: "add_item",
            payload: {
              item: {
                id: file.meta.itemId,
                name: file.meta.itemId,
                previewSrc: preview,
                src: `${ITEMS_STORAGE_BUCKET}/${file.meta.objectName}`,
                tier_id: unsortedTier.id,
              },
            },
          });
        } else {
          toast.error("Something went wrong, please try again.");
          uppy.cancelAll();
        }

        return file;
      })
      .on("upload-error", (file, error) => {
        console.error("👹 Upload error", file, error);
      })
      .on("upload-success", async (file, response) => {
        const unsortedTier = tierlistDataRef.current.tiers.find(
          ({ unsorted_tier }) => unsorted_tier
        );

        console.debug("3⃣ upload success", file, response);

        const { id, name, position } =
          // This items array is only guaranteed to be up to date, if waitForThumbnailsBeforeUpload = true
          unsortedTier?.items.find((item) => item.id === file?.meta.itemId) ?? {};

        try {
          if (!id || !name || position === undefined) {
            throw new Error("👹 Item not found in state");
          }

          if (file?.name && unsortedTier?.id) {
            await upsertTierListAction(tierlistDataRef.current, pathname);
            console.debug("4⃣ item updated in DB");
          }
        } catch (error) {
          console.error("👹 Error saving item", error);
          if (file) {
            toast.error(`Failed to save ${file.name}, please try again.`, { duration: 5000 });
            dispatch({ type: "remove_item", payload: { id: file.meta.itemId } });
            if (file.meta.objectName) {
              await supabase.storage.from(ITEMS_STORAGE_BUCKET).remove([file.meta.objectName]);
            }
          }
        }
      })
      .on("complete", ({ failed, successful }) => {
        console.log("🎉 successful:", successful, "failed:", failed);
        if (failed.length > 0) {
          toast.error(
            `Failed to upload file(s) ${failed.map((f) => f.name).join(", ")}, please try again.`,
            { duration: Math.min(3500 + failed.length * 500, 5000) }
          );

          failed.forEach((f) => {
            dispatch({ type: "remove_item", payload: { id: f.meta.itemId } });
          });
        } else if (successful.length > 0) {
          toast.success("Saved image(s) successfully");
        }
      })
  );

  return (

      <ProgressBar uppy={uppy} />
      <DragDrop
        uppy={uppy}
        className={styles.localOverwrite}
        height="80px"
        locale={{
          strings: {
            dropHereOr: "Drop new images here or %{browse}",
            browse: "browse",
          },
        }}
      />

  );
}

Same issue here for Supabase uploads using Tus. Uppy reports succesful upload for some images and videos files but they do not end up in the buckets.
My packages versions:
    "@uppy/audio": "^2.0.0",
    "@uppy/compressor": "^2.0.0",
    "@uppy/core": "^4.0.1",
    "@uppy/dashboard": "^4.0.1",
    "@uppy/drag-drop": "^4.0.1",
    "@uppy/drop-target": "^3.0.1",
    "@uppy/file-input": "^4.0.0",
    "@uppy/golden-retriever": "^4.0.0",
    "@uppy/image-editor": "^3.0.0",
    "@uppy/progress-bar": "^4.0.0",
    "@uppy/react": "^4.0.1",
    "@uppy/screen-capture": "^4.0.0",
    "@uppy/tus": "^4.0.0",
    "@uppy/webcam": "^4.0.0",

Thankfully the fix mentioned by @1-Felix helped 🙌!
Maybe it's worth reopening the ticket and checking the working solution @fenos.


Split buttons in Menu dropdown

I've put a split button into the user dropdown menü, but when I click to the right side of the button to open the dropdown it does not open, but the user ropdown closes.

Please provide your html code to help you better.


after enabling the rtos part of this project select in lv_conf.h (#define LV_USE_OS   LV_OS_NONE to #define LV_USE_OS  LV_OS_FREERTOS)

ubunu 24.04

I have tried many times but couldn't solve this issue. During this process, I created an Ubuntu virtual machine as well as WSL2 on Windows.

@viys did you followed the instruction?
Could you describe how you setup the project?

Recently, I noticed that you have made updates, but I encountered the following issues while compiling the project.
Here is the process of my setup project,

I won't be able to complete the compilation of the project this time.

Probably it's related to this: https://github.com/lvgl/lv_port_pc_vscode/issues/65#issue-2537471974

@viys Did you checked the CMake path, is it right?

@viys Did you checked the CMake path, is it right?

I directly cloned from the repository, modified it, and followed “To enable the rtos part of this project select in lv_conf.h #define LV_USE_OS LV_OS_NONE to #define LV_USE_OS LV_OS_FREERTOS. Additionally, you have to enable the compilation of all FreeRTOS files by turning on option(USE_FREERTOS "Enable FreeRTOS" OFF) in the CMakeLists.txt file.” to make the changes.😢

@viys Did you checked the CMake path, is it right?

I directly cloned from the repository, modified it, and followed “To enable the rtos part of this project select in lv_conf.h #define LV_USE_OS LV_OS_NONE to #define LV_USE_OS LV_OS_FREERTOS. Additionally, you have to enable the compilation of all FreeRTOS files by turning on option(USE_FREERTOS "Enable FreeRTOS" OFF) in the CMakeLists.txt file.” to make the changes.😢

I have the same error as you, and has it been resolved?

I have the same issue with the segfault. The debugger gives this backtrace
xTaskGenericNotify(TaskHandle_t xTaskToNotify, UBaseType_t uxIndexToNotify, uint32_t ulValue, eNotifyAction eAction, uint32_t * pulPreviousNotificationValue) (.\lv_port_pc_vscode\FreeRTOS\tasks.c:7867)
lv_thread_sync_signal(lv_thread_sync_t * pxCond) (.\lv_port_pc_vscode\lvgl\src\osal\lv_freertos.c:265)
lv_draw_dispatch_request() (.\lv_port_pc_vscode\lvgl\src\draw\lv_draw.c:278)
execute_drawing_unit(lv_draw_sw_unit_t * u) (.\lv_port_pc_vscode\lvgl\src\draw\sw\lv_draw_sw.c:251)
render_thread_cb(void * ptr) (.\lv_port_pc_vscode\lvgl\src\draw\sw\lv_draw_sw.c:352)
prvRunThread(void * pxArg) (.\lv_port_pc_vscode\lvgl\src\osal\lv_freertos.c:330)
prvWaitForStart(void * pvParams) (.\lv_port_pc_vscode\FreeRTOS\portable\ThirdParty\GCC\Posix\port.c:480)
libc.so.6!start_thread(void * arg) (pthread_create.c:442)
libc.so.6!clone3() (clone3.S:81)

The xTaskToNotify variable is null which is causing the segfault

Updating the lvgl and freertos submodules to release/v9.2 and main (974351fe4aec19779f5be774007810c6ab36a8ab) respectively fixes the segfault

@liamhowatt @ulipe could you take a look at this issue?

Looks like the render thread was not created when the sw render executor wants to trigger it.
I will check that


VECTOR data type support

Hi @fanyang01
I’ve implemented a simple version of a Go-duckdb test with the VSS (Vector Similarity Search) plugin, and I’ve been able to confirm that vectorization through the DuckDB VSS plugin is feasible. However, I’ve run into two main issues:

Lack of Support for Vector Type Syntax Parsing: It seems myduck doesn’t yet support syntax parsing related to vector types.

Scan Operation Error with Vector-Type Query Results: When querying for vector-type results, the scan operation throws the following error:
database/sql/driver: unsupported data type: ARRAY: index: 0

My Proposed Approach
To address these, my current approach includes:

Enabling syntax parsing for vector types.
Loading the VSS plugin during initialization.
Supporting the return of vec type results.

I’d really appreciate any suggestions or feedback you might have on this approach, or if you see any potential improvements. Thanks so much in advance for your help!
By the way, I really appreciate the amazing quack you have made!

Hi @NewtonVan,
Nice to see you here! 😄

Enabling syntax parsing for vector types

This should be contributed to the upstream project dolthub/vitess. I'm surprised that in pull request #365, only VECTOR INDEX was supported, but not VECTOR(n).

Supporting the return of vec type results.

Yes, it's a good idea to add support for vector return types. Fortunately, go-duckdb nicely supports nested/structured types.

Loading the VSS plugin during initialization.

Sure. Although the VSS plugin currently comes with several limitations, such as being serialized as a whole for every checkpoint, it's nice to load it and let potential users explore it.

Hi @NewtonVan,
Nice to see you here! 😄

Enabling syntax parsing for vector types

This should be contributed to the upstream project dolthub/vitess. I'm surprised that in pull request #365, only VECTOR INDEX was supported, but not VECTOR(n).

Supporting the return of vec type results.

Yes, it's a good idea to add support for vector return types. Fortunately, go-duckdb nicely supports nested/structured types.

Loading the VSS plugin during initialization.

Sure. Although the VSS plugin currently comes with several limitations, such as being serialized as a whole for every checkpoint, it's nice to load it and let potential users explore it.

Thank you very much for your guidance; it helped me pinpoint the issue more quickly.
Regarding the parsing logic, I’ll look further into how Vitess handles this as I continue exploring possible solutions. I also noted some of the issues highlighted in DuckDB’s blog post on the VSS plugin. Fortunately, adding the plugin only involves minimal changes to the project’s logic, so we can offer basic support for it as a lightweight, experimental feature. I’ll also be exploring this interesting little feature further.
Thanks again for your valuable insights!


Added workflow files for CMSIS compliance and for publishing.

Added workflow files for CMSIS compliance and for publishing.
Renamed title (name: Test Build and Execution) of the hello-ci.yml file. Aligned with the documentation.
Replaced ./FVP/<model_name>.cfg files with ./FVP/<model_name>/fvp_config.txt files.
Updated table i## More CI Examples in README.md file.

The fails for the added workflows ci.yml and verify.yml are expected.


Enable a verbosity level for the GEOPM library through an environment variable

Currently the GEOPM_DEBUG compile time define is used to surpess some warnings (e.g. Failed to load IOGroup).  It would be preferable if some of the warning and error message surpession was managed through a different flag that was configurable at runtime.  The GEOPM_DEBUG flag would then be used to elevate the default verbosity, and would separately enable debug testing in performance critical sections of the code.

Late addition to the WW48 sprint, not considered at planning.  Adding as stretch goal.


Pagination not working

Going to next page doesn't actually change the results listed.

:+1:

@dgreisen-cfpb @imuchnik Who can fix this? This bug makes the tool much less useful than it would be if it were working

I can take a stab at it again.

it is a one-line fix. just inserting the page into the query. pagination was accidentally removed when we added in all the filtering.

:+1:


Listing 2.7 possible error

In Listing 2.7 you say

A file loading the group of modules

and the immediate code that follows is this:
var group = require('./group');

This of course would not work, because group.js does not exist; it should be
var group = require('./index');

since Listing 2.5 demonstrates this.
module.exports = {
    one: require('./one'),
    two: require('./two')
};

Am I correct or have I got it wrong? Well, using ./index in place of ./group makes it work and I get the desired output.
Specs:

Node.js version: 6.3.1
npm: 3.10.3
OS: GNU / Linux Debian testing 64-bit

Yes, I have a package.json with
{ "name": "group", "main": "./index.js" }

in it and still throws errors when I use ./group in place of ./index.

I didn't pay attention to my current path and after re-reading the entire page more 5 times in a row, I have realized I was inside group directory, that's why it didn't work.
I restored the code at its original form (that is, that of the book) and run my testing javascript code from parent directory and boom, it worked.
Apologies for the false alarm authors. Keep up the good work.
Cheers.


[BUG] RuntimeError: No validation backend available! Install one of "flex", "openapi-spec-validator" or "swagger-spec-validator".

Describe the bug
Running into below error when I try to parse openapi data.
RuntimeError: No validation backend available! Install one of "flex", "openapi-spec-validator" or "swagger-spec-validator".
Upon further investigation, found this error which is thrown from openapi-schema-validator which is a dependency of openapi-spec-validator-
Import or syntax error  cannot import name '_legacy_validators' from 'jsonschema'
I am not sure how to proceed further with this because even after installing other back-end "flex" it raises exception the same exception.
Let me know if I am missing something here.
To Reproduce

Install openapi3-parser
Write a simple sample program

from openapi_parser import parse

parse("sample.yaml")

Run the program (throws exception)
Install "flex" backend
Run the sample program again (throws exception)

Expected behavior
A clear and concise description of what you expected to happen.
System details (please complete the following information):

OS: macOS 13.4.1
OpenAPI / Swagger version 3.1.0
Python version 3.11

Additional context
Add any other context about the problem here.

Hey @ajatkj, found this issue in the actual repo of the project I used as a dependency, could you check if installing openapi-schema-validator with 0.1.3 version or higher helps?
Link to the issue - https://github.com/python-openapi/openapi-spec-validator/issues/112

Hi @manchenkoff, upgrading openapi-spec-validator to v0.6.0 solves this issue. This in turn updates openapi-schema-validator to v0.6.1 which handles the latest jsonschema version. Should I raise a PR to bump openapi-spec-validator version to v0.6.0?

@ajatkj that would be nice, thanks!

@ajatkj merged and published as v1.1.14, thanks for the contribution!


Size of marker images (when defined as marker props)

Is this a bug report?
Yes.
Have you read the Installation Instructions?
No. I am using Expo and followed the instructions there.
Environment

"react": "16.3.1",
"react-native": "https://github.com/expo/react-native/archive/sdk-27.0.0.tar.gz"
(documentation [here](https://blog.expo.io/expo-sdk-v27-0-0-is-now-available-898bf1e5b0e4, using react-native-maps 0.21.0)
Target Platform: Android 7.1.0/Galaxy S6/Genymotion

Steps to Reproduce

Because of the problem discussed in #1870, and until it is solved, there is a need to use marker's images as props (and not as children Images within a marker's View)
I am rendering the same image twice:
(1) As a Marker's prop Image
(2) Outside of a Marker
The size of (2) is three times the size of (1). I expected them to be of identical size.

Expected Behavior
Have the images of (1) and (2) displayed with identical size.
Actual Behavior
The size of (2) is three times the size of (1).
Notes:

#1422 talks about the same problem is closed, but I don't see how it explains what I am experiencing
#1870, which is still open, discusses this problem, but talks mainly about the problem of not rendering marker images that are defined as marker's children, in android 7.1 and higher android versions

Reproducible Demo
import React, { Component } from 'react';
import { View, Text, Image, Platform } from 'react-native';
import { MapView } from 'expo';

const centerOffset = { x: 0, y: 0 };
const anchor = { x: 0.125, y: 0.1 };
const cityCoordinates = { latitude: 40, longitude: 40 };
const imageName = require("../images/flag.png");

class CityMarker extends Component {

  constructor(props) {
    super(props);
    this.renderCallout = this.renderCallout.bind(this);
  }

  render() {

    // https://github.com/react-community/react-native-maps/issues/1870
    if (Platform.OS === 'android' && Platform.Version >= 24) {
      return (
        <MapView.Marker
          coordinate={cityCoordinates}
          centerOffset={centerOffset}
          anchor={anchor}
          calloutAnchor={{ x: 0, y: 0 }}
          image={imageName}
          >
          {this.renderCallout()}
        </MapView.Marker>
      );
    } else {
      return (
        <MapView.Marker
          coordinate={cityCoordinates}
          centerOffset={centerOffset}
          anchor={anchor}
          calloutAnchor={{ x: 0, y: 0 }}
          >
          <Image
            source={imageName}
          />
          {this.renderCallout()}
        </MapView.Marker>
      );
    }
  }

  renderCallout() {
    return (
      <MapView.Callout tooltip={true}>
        <View>
          <Text>
            callout text
          </Text>
        </View>
      </MapView.Callout>
    );
  }
}

export default CityMarker;

@rahamin1 Were you able to find a solution to this problem? I'm having the same issue

No, just adjusted to it...


fix: Flush chainstate (and evodb) cache whenever evodb mem usage is getting too high

Issue being fixed or feature implemented
Should fix an issue with high mem usage on IBD/reindex. Alternative to #5003.
What was done?
Give evodb cache its own flush conditions.
How Has This Been Tested?
Works as expected for me both locally (mac x86) and on a remote vps (ubuntu 18.04)
Breaking Changes
Shouldn't be any
Checklist:

[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have added or updated relevant unit/integration/functional/e2e tests
[x] I have made corresponding changes to the documentation

For repository code-owners and collaborators only

[x] I have assigned this pull request to a milestone

Here's my before/after results for 18.0.1 vs this PR (2af8104). Memory usage approximately cut in half.


[BUG] - Container: doesn't accept ref prop

Describe the bug
import { useRef } from "react";
import { Container } from "@nextui-org/react";

export default function App() {
  const containerRef = useRef(null);

  return (
    <Container ref={containerRef} gap={2} justify="center">
      Hi
    </Container>
  );
}

Type '{ children: string; ref: MutableRefObject<any>; gap: number; justify: "center"; }' is not assignable to type 'IntrinsicAttributes & Omit<Props, "gap" | "xs" | "sm" | "md" | "lg" | "xl" | "responsive" | "fluid" | "wrap" | "as" | "display"> & Partial<{ gap: number; ... 9 more ...; display: "flex" | ... 5 more ... | "inline-grid"; }> & NativeAttrs'.
  Property 'ref' does not exist on type 'IntrinsicAttributes & Omit<Props, "gap" | "xs" | "sm" | "md" | "lg" | "xl" | "responsive" | "fluid" | "wrap" | "as" | "display"> & Partial<{ gap: number; ... 9 more ...; display: "flex" | ... 5 more ... | "inline-grid"; }> & NativeAttrs'.ts(2322)

Your Example Website or App
https://codesandbox.io/s/nextui-grid-ref-forked-tigu7s?file=/App.tsx:0-254
Steps to Reproduce the Bug or Issue
https://codesandbox.io/s/nextui-grid-ref-forked-tigu7s?file=/App.tsx:0-254
Expected behavior
Type check successfully passed
Screenshots or Videos

Operating System Version
Windows
Browser
Chrome

@jrgarciadev Hey, I looked to main branch and there is no forward ref there :(


add platform parameter to docker run command. fix #1310

Description
Fixes #1310 add platform parameter to docker run command
TODOs
Read the Gruntwork contribution guidelines.

[x] Update the docs.
[x] Run the relevant tests successfully, including pre-commit checks.
[x] Ensure any 3rd party code adheres with our license policy or delete this line if its not applicable.
[x] Include release notes. If this PR is backward incompatible, include a migration guide.

Release Notes (draft)
Added platform parameter to docker run command
Migration Guide
n/a

I see a test failure. Can this be related to your change?
TestTerraformPackerExample 2024-11-18T23:58:14Z retry.go:91: HTTP GET to URL http://<IP_ADDRESS>:8080
TestTerraformPackerExample 2024-11-18T23:58:14Z http_helper.go:58: Making an HTTP GET call to URL http://<IP_ADDRESS>:8080
TestTerraformPackerExample 2024-11-18T23:58:14Z retry.go:103: HTTP GET to URL http://<IP_ADDRESS>:8080 returned an error: Get "http://<IP_ADDRESS>:8080": dial tcp <IP_ADDRESS>:8080: connect: connection refused. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-18T23:58:19Z retry.go:91: HTTP GET to URL http://<IP_ADDRESS>:8080
TestTerraformPackerExample 2024-11-18T23:58:19Z http_helper.go:58: Making an HTTP GET call to URL http://<IP_ADDRESS>:8080
TestTerraformPackerExample 2024-11-18T23:58:19Z retry.go:103: HTTP GET to URL http://<IP_ADDRESS>:8080 returned an error: Get "http://<IP_ADDRESS>:8080": dial tcp <IP_ADDRESS>:8080: connect: connection refused. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-18T23:58:24Z retry.go:91: HTTP GET to URL http://<IP_ADDRESS>:8080
TestTerraformPackerExample 2024-11-18T23:58:24Z http_helper.go:58: Making an HTTP GET call to URL http://<IP_ADDRESS>:8080
TestTerraformPackerExample 2024-11-18T23:58:25Z retry.go:103: HTTP GET to URL http://<IP_ADDRESS>:8080 returned an error: Get "http://<IP_ADDRESS>:8080": dial tcp <IP_ADDRESS>:8080: connect: connection refused. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-18T23:58:30Z retry.go:91: HTTP GET to URL http://<IP_ADDRESS>:8080
TestTerraformPackerExample 2024-11-18T23:58:30Z http_helper.go:58: Making an HTTP GET call to URL http://<IP_ADDRESS>:8080
TestTerraformPackerExample 2024-11-18T23:58:30Z retry.go:103: HTTP GET to URL http://<IP_ADDRESS>:8080 returned an error: Get "http://<IP_ADDRESS>:8080": dial tcp <IP_ADDRESS>:8080: connect: connection refused. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-18T23:58:35Z retry.go:91: HTTP GET to URL http://<IP_ADDRESS>:8080
TestTerraformPackerExample 2024-11-18T23:58:35Z http_helper.go:58: Making an HTTP GET call to URL http://<IP_ADDRESS>:8080
TestTerraformPackerExample 2024-11-18T23:58:35Z retry.go:103: HTTP GET to URL http://<IP_ADDRESS>:8080 returned an error: Get "http://<IP_ADDRESS>:8080": dial tcp <IP_ADDRESS>:8080: connect: connection refused. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-18T23:58:40Z retry.go:91: HTTP GET to URL http://<IP_ADDRESS>:8080
TestTerraformPackerExample 2024-11-18T23:58:40Z http_helper.go:58: Making an HTTP GET call to URL http://<IP_ADDRESS>:8080
TestTerraformPackerExample 2024-11-18T23:58:40Z retry.go:103: HTTP GET to URL http://<IP_ADDRESS>:8080 returned an error: Get "http://<IP_ADDRESS>:8080": dial tcp <IP_ADDRESS>:8080: connect: connection refused. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-18T23:58:45Z retry.go:91: HTTP GET to URL http://<IP_ADDRESS>:8080
TestTerraformPackerExample 2024-11-18T23:58:45Z http_helper.go:58: Making an HTTP GET call to URL http://<IP_ADDRESS>:8080
TestTerraformPackerExample 2024-11-18T23:58:45Z retry.go:103: HTTP GET to URL http://<IP_ADDRESS>:8080 returned an error: Get "http://<IP_ADDRESS>:8080": dial tcp <IP_ADDRESS>:8080: connect: connection refused. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-18T23:58:50Z retry.go:91: HTTP GET to URL http://<IP_ADDRESS>:8080
TestTerraformPackerExample 2024-11-18T23:58:50Z http_helper.go:58: Making an HTTP GET call to URL http://<IP_ADDRESS>:8080
TestTerraformPackerExample 2024-11-18T23:58:50Z retry.go:103: HTTP GET to URL http://<IP_ADDRESS>:8080 returned an error: Get "http://<IP_ADDRESS>:8080": dial tcp <IP_ADDRESS>:8080: connect: connection refused. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-18T23:58:55Z test_structure.go:29: The 'SKIP_logs' environment variable is not set, so executing stage 'logs'.
TestTerraformPackerExample 2024-11-18T23:58:55Z save_test_data.go:228: Loading test data from ../examples/terraform-packer-example/.test-data/awsRegion.json
TestTerraformPackerExample 2024-11-18T23:58:55Z save_test_data.go:228: Loading test data from ../examples/terraform-packer-example/.test-data/TerraformOptions.json
TestTerraformPackerExample 2024-11-18T23:58:55Z retry.go:91: terraform [output -no-color -json instance_id]
TestTerraformPackerExample 2024-11-18T23:58:55Z logger.go:66: Running command terraform with args [output -no-color -json instance_id]
TestTerraformPackerExample 2024-11-18T23:58:56Z logger.go:66: "i-0f3aeefa3c34dba1f"
TestTerraformPackerExample 2024-11-18T23:58:56Z ec2-syslog.go:36: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-18T23:58:56Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-18T23:58:57Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-18T23:59:02Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-18T23:59:02Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-18T23:59:07Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-18T23:59:07Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-18T23:59:12Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-18T23:59:12Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-18T23:59:17Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-18T23:59:17Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-18T23:59:22Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-18T23:59:23Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-18T23:59:28Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-18T23:59:28Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-18T23:59:33Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-18T23:59:33Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-18T23:59:38Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-18T23:59:38Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-18T23:59:43Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-18T23:59:43Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-18T23:59:48Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-18T23:59:48Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-18T23:59:53Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-18T23:59:54Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-18T23:59:59Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-18T23:59:59Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:00:04Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:00:04Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:00:09Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:00:09Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:00:14Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:00:14Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:00:19Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:00:19Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:00:24Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:00:25Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:00:30Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:00:30Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:00:35Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:00:35Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:00:40Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:00:40Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:00:45Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:00:45Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:00:50Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:00:51Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:00:56Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:00:56Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:01:01Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:01:01Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:01:06Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:01:06Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:01:11Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:01:11Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:01:16Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:01:17Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:01:22Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:01:22Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:01:27Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:01:27Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:01:32Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:01:32Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:01:37Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:01:37Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:01:42Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:01:43Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:01:48Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:01:48Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:01:53Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:01:53Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:01:58Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:01:58Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:02:03Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:02:03Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:02:08Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:02:09Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:02:14Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:02:14Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:02:19Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:02:19Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:02:24Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:02:24Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:02:29Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:02:29Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:02:34Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:02:35Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:02:40Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:02:40Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:02:45Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:02:45Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:02:50Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:02:50Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:02:55Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:02:55Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:03:00Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:03:01Z retry.go:103: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2 returned an error: Syslog is not yet available for instance i-0f3aeefa3c34dba1f in us-east-2. Sleeping for 5s and will try again.
TestTerraformPackerExample 2024-11-19T00:03:06Z retry.go:91: Fetching syslog for Instance i-0f3aeefa3c34dba1f in us-east-2
TestTerraformPackerExample 2024-11-19T00:03:06Z terraform_packer_example_test.go:168: Most recent syslog for Instance i-0f3aeefa3c34dba1f:

Let me check - the weird part is that the same test is failing in the SDKv2 bump branch. I'll take a look today.

It seems the above failing test was related to a missing HTTP server. This has been fixed as part of #1451

Let's rebase the code with the recent change and I'll trigger the test pipeline once again to see if there are any major concern with the change.

rebase done


Standardized name on the Rapid Test certificate is not standard

Describe the bug
When I receive the test-certificate from a rapid test using the CWA workflow the standardized name does not match the standardized name on my immunization certificate.

Expected behaviour
I would expect the standardized name on the rapid test certificate and immunization certificate to be identical, so that they can be correlated for the 2G+ status.

Steps to reproduce the issue

Name: Alan Rick Johnson
Standardized name on Immunization Certificate: JOHNSON<<ALAN<RICK
Standardized name on Rapid Test Certificate: JOHNSON<<ALANRICK

Technical details

CWA is iOS Latest. The name comes from the CWA Rapid Test profile and is not manipulated by the tester.

Possible Fix

Additional context

This might be important apps that validate covid certificates, for example in other countries or apps that validate against the Personal Ausweis.

The test centres are responsible to correctly enter the standardized name.
See also https://github.com/corona-warn-app/cwa-documentation/issues/810

Apologies for the duplicate, I'd not expected this to be in the documentation repository.
I thought the test-center uses 100% the CWA Workflow software. I.e. The scanning app that the tester uses comes from the CWA project, and the name comes from my CWA client. I hadn't realized each test-center uses their own backend software to generate the certificates and I don't understand why this is necessary.
Nevertheless I'm closing the ticket.

@alanrick
Depends on the test center: some use the official portal from TSI, where the standardized names still need to be entered manually.
Other test centers use their own software, where problems with standardized names would need to be detected during the "Abnahmetest" when provider goes into production.
Today, I brought this subject again to attention to all stakeholders: https://github.com/corona-warn-app/cwa-quick-test-frontend/issues/185#issuecomment-1021142950

At this testcenter which I used yesterday she was using a Web form where she does have to key in the standardized name and firstname. Schnelltestportal.de
Is that the TSI one? Has that been improved since I took the photo yesterday? I noticed corona-warn-app/cwa-documentation#810 has now been closed.

Hi @alanrick ,
yes, schnelltestportal.de is the TSI's one. The request to improve the portal to automatically fill out the "standardized name fields" is open since summer 2021.
I'd recommend to follow issue https://github.com/corona-warn-app/cwa-quick-test-frontend/issues/185 on any updates about this. I also suggested to improve/harmonize the quality of standardized name transliteration for all test centers that do not use the Schnelltestportal but which are connected to TSI's backend: https://github.com/corona-warn-app/cwa-quick-test-frontend/issues/185#issuecomment-1021142950
Doesn't look like we'll see improvements in a timely manner, unfortunately.
Please feel free to comment your thoughts in the referenced issue.
Hope that helps,
v.


add support for the Portuguese language of Brazil

Describe the solution you'd like
As a DefectDojo user, I want the system to have support for the Brazilian Portuguese language, so that Brazilian users can use the tool in their native language, improving accessibility and the user experience.
Portuguese is the 5th most spoken language in the world, with around 260 million native speakers. Of these, approximately 215 million are in Brazil, which represents a significant portion of DefectDojo's potential target audience. By offering support for Brazilian Portuguese, the tool will become more accessible and inclusive for this large group of users, improving its adoption and usability.
According to the source: https://github.com/DefectDojo/django-DefectDojo/blob/dev/dojo/locale/en/LC_MESSAGES/django.po

Oh nice idea. Could you pleaes make a PR for this?

Oh nice idea. Could you pleaes make a PR for this?

I haven't done it yet, but I can do it.
I verified that the file for the pt-br locale needs to be created.
Then I will submit the pull request.
I will also mention that issue.

This request has already been completed.

@GraoMelo Obrigado (one of the few Portuguese words I know) for considering a pt-br PR.
Just so you are aware, we've had several members of the community work toward internationalization in DefectDojo though I don't believe any of those efforts were complete.
I just wanted to make sure you're aware that after creating a translation file (.po) pt-br won't be showing everywhere within DefectDojo. I know a while back we had a contributor changing the UI to use translated strings but, to be honest, I don't believe they completed that work before moving on to other things.
We'd be happy to have a pt-br translation especially since I only know maybe 12 Portuguese words (and verbally only) so I can't do that work ~ but ~ some of DefectDojo will reflect those changes but likely not all.
Just didn't want you surprised. :+1:


ISPN-7080 NPE in CacheNotifierImpl by LIRS eviction listener

https://issues.jboss.org/browse/ISPN-7080

Superseded by #4639


Runtime performance

Hello,
In the paper, I haven't seen any performance regarding Runtime/FPS for different sizes.
Do you have any ballpark numbers, how fast is the network for example on 360p and 720p videos?
Would it differ by a large margin if x4 upscale was changed to x2?
Best regards

In my preliminary testing, when running iSeeBetterTest.py using parameters

Namespace(chop_forward=False, data_dir='./Vid4', debug=False, file_list='ggbb_png.txt', future_frame=True, gpu_mode=True, gpus=1, model='weights/netG_epoch_4_1.pth', model_type='RBPN', nFrames=7, other_dataset=True, output='Results/', residual=False, seed=123, testBatchSize=1, threads=8, upscale_factor=4)

on an NVIDIA V100 machine, a grayscale png sequence of 100 frames where each frame measures 96x72(Note: 384x288 original input, but iSeeBetter downscales by the scale factor it for future re-upscaling to original size and testing I think, not 100% sure) and uses around ~20 kb/frame (80kB/frame original, see previous note), takes a (ballpark) 0.2-0.3s per frame(see full output attached). This would extrapolate to around 6-9s of processing time for one second of 96x72 30fps ~0.6 MBps "video". Those are the numbers I have as of now
output.txt

Hey, thanks for such a rapid response, very insightful
the provided parameters show testBatchSize=1, could this be increased to speed the network? What is the memory usage during the inference?

Yeah no worries, I was messing around with it anyway.
 In the attached file, I've listed the output times per batch(so divide the times by the testBatchSize I guess) and normal RAM usage at idle and during the process. Unfortunately I'm using Colab and I can't goddamn figure out how to check vram usage during execution of the loop, which is the what you actually wanted to see I'm guessing. What I can tell you is that the GPU was a V100 with ~16gigs of memory and it fails(CUDA out of memory. somewhere between a testBatchSize of 20 and 24, so I guess it uses around 16gigs somewhere around that batch size. But yeah, seems like it came down to around 0.05s/frame by testBatchSize=20.
testBatchOut.txt

alright, thanks, this was a great help


Frontend : CSS Animations in Document Form

Description
In this, I talk about are key features to make an animation and explain each part of the animation in great detail.
Domain
Frontend Dev HTML/CSS/JS
Type of Contribution
Documentation
Code of Conduct

[X] I follow Contributing Guidelines & Code of conduct of this project.

/assign


Read The Docs build is failing: "Could not import extension sphinx.builders.linkcheck"

Could not import extension sphinx.builders.linkcheck (exception: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with OpenSSL 1.0.2n  7 Dec 2017. See: https://github.com/urllib3/urllib3/issues/2168)
See https://github.com/readthedocs/readthedocs.org/issues/10290

Fixed by adding the following to .readthedocs.yml
build:
  os: ubuntu-22.04


Doesn't appear to work with current version of Webpack

I tried following the dapp tutorial on the Parity wiki, but I get some errors from webpack. It seems that the config format has changed slightly and loaders on line 13 should be changed to rules.
That said, I don't know much about webpack so I thought I should submit an issue rather than a PR.

I had the same error. I just changed loaders to rules and it was working fine. But now the dapp is not rendering and the console shows this error:

Well, I have added mode: 'developement' and changed loaders to rules plus I also added "allowJsEval": true on the manifest.json file
And the voting dapp is working fine now


Package oni2 for homebrew

Oni2 should have a homebrew package for ease of installation - instead of going the dmg -> app route.
It can be in a private tap - it actually makes more sense to be in a tap than being in core.

I maintain an unofficial tap here: https://github.com/marblenix/homebrew-onivim2
Please let me know what you think!


AMBARI-23778 Ambari assigns /home for NameNode, DataNode and NodeMana…

…ger directories
What changes were proposed in this pull request?
The problem is that mount point filtering has changed in the HDP 3.0 version.
Service advisors need a way to define additional banned mount points.
This PR allows that.
This PR  only contains half the fix. The other part of the fix will go into the hdp_ambari_definitions repo.
This change needs to be  merged first, then the other  part.
How was this patch tested?
unit test
Integration test on my local cluster.
Please review Ambari Contributing Guide before opening a pull request.
@swagle please review

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/2169/
Test FAILed.
Test FAILured.


De-lint lib/tests scripts

Motivation for this change
More reliable shell scripts.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Could you lint the shell scripts in
https://github.com/NixOS/nixpkgs/tree/master/nixos/modules/installer/tools
and
https://github.com/NixOS/nixpkgs/blob/master/pkgs/os-specific/linux/nixos-rebuild/nixos-rebuild.sh
?  🙂

Could you lint the shell scripts in https://github.com/NixOS/nixpkgs/tree/master/nixos/modules/installer/tools and https://github.com/NixOS/nixpkgs/blob/master/pkgs/os-specific/linux/nixos-rebuild/nixos-rebuild.sh

For sure! If you have any other specific scripts in mind, please let me know. I was just getting started on pkgs/stdenv/generic/builder.sh, which I figure has a 95% chance of breaking stuff in interesting, possibly bug-revealing ways. :grin:


Add the parent directory to changes set if new files are generated

The "TestChangesWithChanges" case randomlly fails on my development VM with the following errors:
--- FAIL: TestChangesWithChanges (0.00s)
        changes_test.go:201: no change for expected change C /dir1/subfolder != A /dir1/subfolder/newFile

The test details can be found here: http://pastebin.com/bYupfSr4
If I apply the following patch to changes_test.go, the test passes.
diff --git a/pkg/archive/changes_test.go b/pkg/archive/changes_test.go
index 290b2dd..ba1aca0 100644
--- a/pkg/archive/changes_test.go
+++ b/pkg/archive/changes_test.go
@@ -156,6 +156,7 @@ func TestChangesWithChanges(t *testing.T) {
        }
        defer os.RemoveAll(layer)
        createSampleDir(t, layer)
+       time.Sleep(5 * time.Millisecond)
        os.MkdirAll(path.Join(layer, "dir1/subfolder"), 0740)

        // Let's modify modtime for dir1 to be sure it's the same for the two layer (to not having false positive)

It seems that if a file is created immediately after the directory is created, the archive.Changes function could't recognize that the parent directory of the new file is modified.
Perhaps the problem may reproduce on machines with low time precision?
I had successfully reproduced the failure on my development VM as well as a VM on DigitalOcean.
Signed-off-by: Shijiang Wei<EMAIL_ADDRESS>

Hi @mountkin, I did reproduce this when running it with something like goconvey (which runs a lots of tests at the same times). I think it would probably be better to just add the time.Sleep() in the test rather than updating the code ; but we'll wait for other opinions.

My god! While this PR fixed the test on my development VM, it broke the jenkins test.
Perhaps the "/dir1/subfolder" is added to the changes set twice?
@vdemeester At first I also thought adding the sleep is ok. But I'm not sure if it's a potential bug of the Changes function.

:laughing: Tests passed both on my VM and with janky.
ping @vbatts

oh sneaky. let me look.

LGTM

@duglin care to review?

@vbatts okie dokie will do

I'm kind of confused by this change. During the walk() call, why isn't the new dir automatically added? Is it not seen during the walk? The code in the walkFunc seems like it would add new dirs with a change type of "Add".  Or, due to some weird issue, are we actually walking to a file w/o walking to its parent dir?

@duglin After some farther experiments I found that whenever files are created, modified or deleted, the modify time of the direct parent is always changed.
I'll updated the PR again as you had suggested, but only add the direct parent directory to the "changes" slice.
The test case "TestChangesWithChanges" should be modified accordingly.

@mountkin I just want to make sure I fully understand the situation, sorry if these are silly questions...

is all of this due to a bug in the filesystem code where the timestamp of the parent dir isn't modified right away (or perhaps at all)? Previously someone mentioned that adding a sleep to the test seemed to fix it, so that's why I'm wondering if this a timing issue/bug with the FS
the walk() call is traversing the files in the R/W (top) layer, right? During that processing, do you know if it hits the parent dir of the changed file? I'm wondering if the parent dir is present in the R/W layer but just w/o an unmodified timestamp?
if the parent dir is present in the R/W layer, is just the immediate parent there, or are all of the parents there up to / ?

@duglin the "someone" is definitely me, see the patch to changes_test.go in the description of this PR :smile:
I'm also curious that why the time of the directory is not modified if I don't add the sleep in the test code.
For your second question, according the godocs http://godoc.org/path/filepath#Walk,  "files are walked in lexical order", so the parent directory should be hit first.
The third question: I think all the parents should be there.

Thanks to @mountkin for giving me a VM to reproduce this, I was able to get more info:
t1: "2015-06-03 11:51:16.829413949 -0400 EDT"
t2: "2015-06-03 11:51:16.829980647 -0400 EDT"

those are the real time before we create the two subfolders on each layer
Time: "2015-06-03 11:51:16.826944999 -0400 EDT"
Time: "2015-06-03 11:51:16.826944999 -0400 EDT"

those are the stat times for the two subfolders on each layer.  Notice that stat() is returning a less precise/rounded time for each dir.  So, when we check to see if the dirs have been modified, if we didn't cross the threshold for the rounding, then it'll appear as though they didn't change.
I think the chances of this happening in a real environment are pretty slim - you would have to create two layers and two same-named dirs on both layers within that rounding threshold.  Pretty slim.
However, I think the code changes being proposed are still ok to make since it should be a no-op.  The reason we'd need to change the testcase (to add 'dir1' to the list of changes) is because the testcase goes out of its way to set the timestamp on 'dir1' on both layers to be the same value. Not sure why.  @vbatts do you have any insight into that one?
@vbatts if you're ok with the testcase having one more item in the list of changes then I'm ok too.  I think the only other thing we should do is modify the testcase to NOT set 'dir1's mtime since that'll just confuse people.

@duglin well.. I wrote this test :sweat:. I did add this comment "Let's modify modtime for dir1 to be sure it's the same for the two layer (to not having false positive)". But it turns out it's wrong :disappointed:, got false positive even..
If I remember correctly, this was to be sure it would detect the change inside dir1 without detecting a dir1 change. But the more I look at this, the more ugly it seems to me (the way I did it :sob:).

@vdemeester I don't think 'dir1' is the issue. Granted, with this PR I think modifying its timestamp isn't necessary (and could be confusing) but the bigger issue is why stat()'s precision on the mtime is so bad? Probably not a lot we can do about it, but at least this PR seems to at least work around it.

I just removed the os.Chtimes block.
According my experiment results when a file is deleted from or added to a directory, the mtime of the directory is changed. Since /dir1/file1-2 is deleted, the mtime of dir1 should change, so it must be in the changes slice.

If I have /dir1/dir2/dir3/file.txt in a base layer. Then in the next layer I touch file.txt. In a "normal" environment, "dir3" will have its mtime changed and this PR shouldn't be needed - which means hopefully it'll be a no-op, right?.  However, would this change make it so that 'dir2' is in the change list?
I just modified the testcase to try this out. Perhaps I did something wrong but I only see file.txt in the change list w/o this PR. With this PR I see dir3 as well.  I even added a sleep(3) before I touched file.txt to make sure we didn't hit a timing issue.
@vbatts should 'dir3' be in the change list irrespective of this PR?

ok in my testing a dir's mtime is only modified when a new file appears, or disappears, under it, not when a file under it is changed.   I wonder if we need to take into account the type of change of the original file in question before we add its parent.

So, back to my original question... i think this PR does in fact add 'dir2' by mistake.  By modifying the testcase so it adds a new file instead of touching a file (in dir3) I can make 'dir2' appear in the change list but I don't think it should be.
Not sure yet what the right fix is, so for now just FYI...

ok just wrappering the new logic that adds the parent dir with
if change.Kind == ChangeAdd || change.Kind == ChangeDelete {

seems to show the correct results.

clearly need more tests for this too :-)

@duglin When I added if change.Kind == ChangeAdd || change.Kind == ChangeDelete { and modified the test case, it passed outside of the test infrastructure but failed inside the test infrastructure.
The reason that janky failed is that f.Size() and stat.Size() are not equal when testing in docker.
"rwPath: /dir2, roPath: /tmp/docker-changes-test-layer911993879/dir2, f.Size: 60, stat.Size: 80, f.Mode:<PHONE_NUMBER>, stat.Mode:<PHONE_NUMBER>, f.ModTime: 2015-06-08T15:56:22.937557991Z, stat.ModTime: 2015-06-08T15:56:22.937557991Z"
After digging the godocs I found the following comment in the os.FileInfo struct (http://godoc.org/os#FileInfo).
Size() int64        // length in bytes for regular files; system-dependent for others

So my conclusion: the parent directory should be added to the change list irrespective of the kind of the change.

ok. so where are we on this? review needed now or still refining?
Also, there look to be a couple of failures in the CI

@mountkin I think the testcase needs to be changed.
See my branch here: https://github.com/duglin/docker/tree/Changes
and let me know if you think my thinking is wrong.

Thanks @duglin I think your test case is reasonable. I just cherry-picked your test case to this PR.
But after squash it seems that your name isn't showing in the commit logs. Sorry for that.

@mountkin no worries, as long as the correct code gets in, that's all that matters.
@vbatts @vdemeester you guys ok with the changes?  Please check the testcases too since we did modify those a little.  I want to make sure we don't incorrectly add, or remove, from the list of changes people are used to seeing.

LGTM

LGTM

LGTM


vitest 0.12.x incompatible with remix.run / @web-std/file

Describe the bug
vitest 0.12 seems to add an issue when importing import { Blob as NodeBlob, File as NodeFile } from "@web-std/file";, which the remix.run test setup uses. (https://github.com/remix-run/remix/blob/main/packages/remix-node/globals.ts):
TypeError: Class extends value does not have valid prototype property undefined
 ❯ Object.<anonymous> node_modules/@web-std/file/dist/src/file.cjs:59:15
 ❯ Object.<anonymous> node_modules/@web-std/file/dist/src/lib.node.cjs:6:12

This causes the tests to fail. It seems like this is a regression with the 0.12 release, downgrading vitest to 0.11 makes the tests run again.
Reproduction
Install the remix.run indie stack with npx create-remix --template remix-run/indie-stack, edit package.json and set vitest to ^0.12.4, run npm i and npm test.
The remix installGlobals is imported in the vitest testup file located under test/setup-test-env.ts.
System Info
System:
    OS: macOS 12.3.1
    CPU: (10) arm64 Apple M1 Max
    Memory: 1.73 GB / 64.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 16.14.2 - ~/Library/Caches/fnm_multishells/50575_1651571166571/bin/node
    Yarn: 1.22.18 - /opt/homebrew/bin/yarn
    npm: 8.5.0 - ~/Library/Caches/fnm_multishells/50575_1651571166571/bin/npm
  Browsers:
    Firefox: 100.0
    Safari: 15.4
  npmPackages:
    @vitejs/plugin-react: ^1.3.0 => 1.3.2
    vite: ^2.9.1 => 2.9.8
    vitest: ^0.12.4 => 0.12.4

Used Package Manager
npm
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

I have the exact same error message when importing the user-event package from testing-library.
Class extends value does not have valid prototype property undefined

@sheremet-va This problem is still present
https://github.com/remix-run/indie-stack/runs/6536194156?check_suite_focus=true
https://github.com/remix-run/blues-stack/runs/6536184942?check_suite_focus=true
https://github.com/remix-run/grunge-stack/runs/6536186808?check_suite_focus=true

@sheremet-va This problem is still present
https://github.com/remix-run/indie-stack/runs/6536194156?check_suite_focus=true https://github.com/remix-run/blues-stack/runs/6536184942?check_suite_focus=true https://github.com/remix-run/grunge-stack/runs/6536186808?check_suite_focus=true

We haven't released a new version yet.

Oh I'm sorry, I thought you had 🙈

Oh I'm sorry, I thought you had 🙈

Understandable. GitHub closes issues on PR merge, and not on releases.

Any ETA on the release?

Today, in an hour.

@MichaelDeBoey released in 0.12.8

Seems to be fixed indeed, thanks!


display is number 0 when variable return is null

when variable return is null, data have been display is none.
I think, should display is number 0

I guess that makes more sense then returning an empty string. I'll do it tomorrow. Thanks.

Fixed in v2.0.0

Fixed in v2.0.0

Thank @Zmetser


I have a repo on the disk, and it has an existing .gitignore file that I want respected.  Meaning, parse the .gitignore file and then apply the rules in it.  Is there a way to do that?
Might I proceed as the following (pseudo code):
namespace Ignore
{
    public class Ignore
    {
        /*...*/

        public void Ingest(string pathname /*pathname of the .gitignore file*/)
        {
            try
            {
                if (!File.Exists(pathname)) return;
                if (!".gitignore".Equals(Path.GetFileName(pathname)) return;

                var lines = File.ReadAllLines(pathname).ToList();
                if (lines == null) return;
                if (lines.Count == 0) return;

                Add(lines.ToArray());
            }
            catch(Exception ex)
            {
                /* log the exception or handle it in another fashion */
            }
        }

        /* ... */
    }

Listing 1. Proposed methodology for ingesting an existing .gitignore  file that is already on the disk.
Would the code above work?

That was also my thinking but I did want to validate it.  Thanks for that.  At the very least, the code snippet above is offered above for educational purposes.
Question: I request permission to be allowed to contribute to your project's Wiki so I can post the snippet there.  My only concern with it remaining in this issue is, once this issue is closed, I'm worried that the code snippet might no longer be easily accessible.

Sure thank you! There is no wiki right now but we can start one. Do you have permissions to edit/start the wiki?

Figure 1. Tabs displayed to me as a repository "viewer."
Hello @goelhardik I just checked and there does not seem to be a 'Wiki' tab on my display... Correct me if I am wrong, but I am thinking that starting a Wiki is only available to repo owners such as yourself.  And if you can designate me as a "contributor," then it would be my speculation that doing that would then open it up to myself.
I respectfully request to be a "contributor" so I can also contribute ideas / suggestions for improving the code.  Thank you.
For example, I'd like to enhance it to make it more "SOLID" (see, e.g, Wikipedia).  Also, would like to make it possible to "unload" a .gitignore file too...by clearing out the internal rules list and such so you can then load a new .gitignore file without having to throw away the object and start fresh etc., say, when a user of this class wishes to access a new Git repo on their hard drive.

Thanks @astrohart. I have added you as a collaborator. You should be able to access the Wiki now.


training

I was trying to execute the training code on a cpu. With the following hyperparemeters.
python train_ditto.py
--task Structured/Beer
--batch_size 64
--max_len 64
--lr 3e-5
--n_epochs 5
--finetuning
--lm distilbert
--da del
--dk product
--save_model
--summarize
I think some how the dev_f1 score is zero and the accuracy stuck at 0.84 and the epoch is not increased and its going in loops since due to the while loop in mixda is epoch <= hp.n_epochs.
is there something I am missing or is it going in an infinite loop

I see. Will fix the bug of infinite loops. The hyper-parameters are not ideal for this dataset either (I will change the README with an updated set). Meanwhile, you can try this one
CUDA_VISIBLE_DEVICES=0 python train_ditto.py \
  --task Structured/Beer \
  --batch_size 32 \
  --max_len 128 \
  --lr 3e-5 \
  --n_epochs 40 \
  --finetuning \
  --lm roberta \
  --fp16 \
  --da drop_col

which should work better.

I have this same error before as well, forgot to capture this
/data/home/vijaya.chennupati/.conda/envs/txtclass/lib/python3.8/site-packages/sklearn/metrics/_classification.py:1221: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 due to no predicted samples. Use zero_division parameter to control this behavior.
_warn_prf(average, modifier, msg_start, len(result))
accuracy=0.846
precision=0.000
recall=0.000
f1=0.000

I am using CPU ... so removed the parameter --fp16

I see. I tried the CPU version and got the same error. It seems to be related to data augmentation but I am not sure. I tried the baseline version (no DA) and it works fine on CPU:
CUDA_VISIBLE_DEVICES= python train_ditto.py   --task Structured/Beer   --batch_size 32   --max_len 128   --lr 3e-5   --n_epochs 40   --finetuning   --lm distilbert

You might also use this colab notebook to run it on GPUs.

Will give it a try.

Hello, does the latest version fixed this bug?
I also have this problem when I use CPU to train my program...

I have the same bug when training on cpu, and it seems like the summarizing phase leads to too few input data for training. However I train the model based on my own implementation instead of original code, thus it may not be the case for you. Check the summarized data (.su filename extension) and see if it applies.


Multiple memory bank problem

Thank you for sharing your code. I have a question because there is a discrepancy between the results of Table 1 and Table 2 and results of mine.
Regarding the results for 3D and RGB, I am getting results similar to those in the paper. However, for RGB+3D, the AUPRO score is significantly lower than the results in the paper. It seems that multiple memory bank doesn't work well. I suspect there might be an issue with the function add_sample_to_late_fusion_mem_bank calculates s_map, but I haven't been able to resolve it. Do you have any idea what the problem might be?
 I have a question because there is a discrepancy between the results of Table 1 and Table 2 and results of mine.
Regarding the results for 3D and RGB, I am getting results similar to those in the paper. However, for RGB+3D, the AUPRO score is significantly lower than the results in the paper. It seems that multiple memory bank doesn't work well. I suspect there might be an issue with the function add_sample_to_late_fusion_mem_bank calculates s_map, but I haven't been able to resolve it. Do you have any idea what the problem might be?

I only tried the dino method, but the effect is much worse than yours, I would like to ask if you have made any improvements to the RGB-only method, or what is the device environment like?

My results of DINO are similar to those in paper, but Point_MAE method didn't perform well on some classes of the mvtec3d

Hi, I ran the results with only the I-AUROC metric not matching the results in the paper at 0.936 (0.945 in the paper), and saw that your I-AUROC result was 0.94, and would like to ask what is your graphics card configuration? The UFF training weights I used are the best training weights provided by the authors (it's the UFF Module under checkpoints)

@VegetableChicken504
I used NVIDIA's V100 at that time. I think 0.09 points lower is possible due to random numbers and other conditions.


Consolidate cross app scrollbar styles

There's a few places we style the scrollbars individually. We should do a cross app styling and allow for component level overrides.

this is done


meshtastic: fix version and add to all-packages

Description of changes
Patch out pkg_resources (https://setuptools.pypa.io/en/latest/pkg_resources.html)
Add meshtastic to all-packages, as it is mostly used as cli to control meshtastic devices.
Things done

patch out pkg_resources and replace version with substituteInPlace ${version}

Add meshtastic to all-packages

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)

[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage

[x] Tested basic functionality of all binary files (usually in ./result/bin/)

23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Please change the commit messages to python310Packages.meshtastic: .... It's a Python module.


Add new area labels (carm, metrics, scaling)

These labels represent areas that have become more prominent recently,
indicating a growing need to categorize and track issues related to metrics
scalability and CARM
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

/lgtm


Patch 1

https://github.com/MicrosoftDocs/OfficeDocs-Exchange/issues/1294

@nenonix @JohanFreelancer9 Copy editing is needed for this PR. Thanks.

@JoanneHendrickson PR has been copyedited and is ready for final review, could you please check and merge? Thanks!

@JoanneHendrickson PR has been copyedited and is ready for final review, could you please check and merge? Thanks!