ERR. Download aborted

Hi, thanks for your great work.
I get this error and I don't know why it happens. any idea?
sudo figma-backup -e "EMAIL" -p "PASSWORD" -t "TOKEN" --projects-ids "ID" --download-timeout 45
> Starting the backup task...
> Authenticating the bot...
        . Bot successfully logged in.
	. Caching the cookies...
	. Looking for cached cookies...
	. Restoring the cached cookies...
> Backuping the file(Project Name)...
        . Setting the download behaviour...
	. Saving a local copy of the file(Project Name)...
	. Opening up the figma command pallete...
	. Typing down the download command...
	. Execute the download command...
        ERR. Download aborted | Timeout of 2700s exceeded.
Backup task finished! (time elapsed: 2797s)

@mostafaznv

ERR. Download aborted | Timeout of 2700s exceeded.

It seems that the download process has exceeded the  given --download-timeout option.
Try to re-run the process using --debug option and check if that's the case.
If so, try to increase the amount of --download-timeout option. (you are probably facing a huge Figma file)

I have the same issue

@Believemenot
Try to re-run the process using --debug option and please report me the situation.

with --debug option I get this error:
Error: Failed to launch the browser process!
[168853:168853:0225/102648.561888:ERROR:browser_main_loop.cc(1409)] Unable to open X display.
[0225/102648.579050:ERROR:nacl_helper_linux.cc(307)] NaCl helper process running without a sandbox!
Most likely you need to configure your SUID sandbox correctly

TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md

    at onClose (/usr/lib/node_modules/figma-backup/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:203:20)
    at Interface.<anonymous> (/usr/lib/node_modules/figma-backup/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:193:68)

I'm trying to find a workaround for this error but I don't know if it's about your package or my machine.

When i run with --debug option but nothing dowloads
`> Starting the backup task...

Authenticating the bot...
. Bot successfully logged in.
. Caching the cookies...
. Looking for cached cookies...
. Restoring the cached cookies...
Backuping the file(3D иконки)...
. Setting the download behaviour...
. Saving a local copy of the file(3D иконки)...
. Opening up the figma command pallete...
. Typing down the download command...
. Execute the download command...
. File (3D иконки) successfully downloaded.
Backuping the file(App Icon)...
. Setting the download behaviour...
. Saving a local copy of the file(App Icon)...
. Opening up the figma command pallete...
. Typing down the download command...
. Execute the download command...
. File (App Icon) successfully downloaded.
Backuping the file(App Mockups)...
. Setting the download behaviour...
. Saving a local copy of the file(App Mockups)...
. Opening up the figma command pallete...
. Typing down the download command...
. Execute the download command...
. File (App Mockups) successfully downloaded.
Backuping the file(Продавцы App)...
. Setting the download behaviour...
. Saving a local copy of the file(Продавцы App)...
. Opening up the figma command pallete...
. Typing down the download command...
. Execute the download command...
. File (Продавцы App) successfully downloaded.
Backup task finished! (time elapsed: 116s)
`

@mostafaznv
The error "Unable to open X display" is a known error on Linux machines and It's related to the Puppeteer (a package we are using for Bot purposes).
There's a workaround for this:
Stackoverflow
Puppeteer's Github

@Believemenot
It seems that there's no problem with the Bot itself.
Can you also provide the full ActionLog of the process without --debug option?

> Starting the backup task...
> Authenticating the bot...
        . Bot successfully logged in.
	. Caching the cookies...
	. Looking for cached cookies...
	. Restoring the cached cookies...
> Backuping the file(3D иконки)...
        . Setting the download behaviour...
	. Saving a local copy of the file(3D иконки)...
	. Opening up the figma command pallete...
	. Typing down the download command...
	. Execute the download command...
        ERR. Download aborted | Timeout of 60s exceeded.
> Backuping the file(App Icon)...
        . Setting the download behaviour...
	. Saving a local copy of the file(App Icon)...
	. Opening up the figma command pallete...
	. Typing down the download command...
	. Execute the download command...
        ERR. Download aborted | Timeout of 60s exceeded.
> Backuping the file(App Mockups)...
        . Setting the download behaviour...
	. Saving a local copy of the file(App Mockups)...
	. Opening up the figma command pallete...
	. Typing down the download command...
	. Execute the download command...
        ERR. Download aborted | Timeout of 60s exceeded.
> Backuping the file(Продавцы App)...
        . Setting the download behaviour...
	. Saving a local copy of the file(Продавцы App)...
	. Opening up the figma command pallete...
	. Typing down the download command...
	. Execute the download command...
        ERR. Download aborted | Timeout of 60s exceeded.
Backup task finished! (time elapsed: 366s)

@mostafaznv
The error "Unable to open X display" is a known error on Linux machines and It's related to the Puppeteer (a package we are using for Bot purposes).
There's a workaround for this: Stackoverflow Puppeteer's Github

Thanks. I fixed that issue (with this workaround)
then I tried to run script with --debug option. this is the result:
> Starting the backup task...
> Authenticating the bot...
        . Bot successfully logged in.
	. Caching the cookies...
	. Looking for cached cookies...
	. Restoring the cached cookies...
> Backuping the file(Project Name)...
        . Setting the download behaviour...
	. Saving a local copy of the file(Project Name)...
	. Opening up the figma command pallete...
	. Typing down the download command...
	. Execute the download command...
        ERR. Download aborted | Timeout of 300s exceeded.
Backup task finished! (time elapsed: 384s)

it doesn't provide any extra information
more information:

my project is a simple project (file size is 300kb)
I tried to run script with two servers. Iran and Netherlands but result was the same.

@mostafaznv
--debug runs the bot in headful mode which will open up a chromium and proceed each steps in the browser.
Couldn't you see the actions in a headful way?

@mostafaznv
Unfortunately I can't reproduce the problem due to differences in the machines we are using.
All I can tell (based on the given error on the downloading process) is that your download process is taking too long. However, the problem might be something else.
Since your machine is preventing the bot to run a headful process (via --debug option) we can't investigate further, so I encourage you to do run the process with --debug option on:
1- Your machine (Not a VM) and monitor the process.
2- Another machine (preferably MacOS) and monitor the process.
You have to somehow get the bot to work headfully on your machine so you can monitor the process on the browser itself.
Please read the following docs for more information:
Puppeteer's Github
Puppeteer's Troubleshooting

> Starting the backup task...
> Authenticating the bot...
        . Bot successfully logged in.
	. Caching the cookies...
	. Looking for cached cookies...
	. Restoring the cached cookies...
> Backuping the file(3D иконки)...
        . Setting the download behaviour...
	. Saving a local copy of the file(3D иконки)...
	. Opening up the figma command pallete...
	. Typing down the download command...
	. Execute the download command...
        ERR. Download aborted | Timeout of 60s exceeded.
> Backuping the file(App Icon)...
        . Setting the download behaviour...
	. Saving a local copy of the file(App Icon)...
	. Opening up the figma command pallete...
	. Typing down the download command...
	. Execute the download command...
        ERR. Download aborted | Timeout of 60s exceeded.
> Backuping the file(App Mockups)...
        . Setting the download behaviour...
	. Saving a local copy of the file(App Mockups)...
	. Opening up the figma command pallete...
	. Typing down the download command...
	. Execute the download command...
        ERR. Download aborted | Timeout of 60s exceeded.
> Backuping the file(Продавцы App)...
        . Setting the download behaviour...
	. Saving a local copy of the file(Продавцы App)...
	. Opening up the figma command pallete...
	. Typing down the download command...
	. Execute the download command...
        ERR. Download aborted | Timeout of 60s exceeded.
Backup task finished! (time elapsed: 366s)

@Believemenot
Dude it's working :)
You have to look for figma-backup-root directory.
Figma Backup Official Docs

ERR. Download aborted | Timeout of 60s exceeded.

@Believemenot
Ok let me explain the process of the downloading your files and the meaning of this error, so that you might figure out what you are doing wrong.
The bot fetches the given projects' files using Figma API and then navigates to each of them one by one.
When your Figma file loaded successfully the bot is going to open up the command palette and write down the command to download a copy of your file.
The download process doesn't give us any callback or promise to check whether it is done or not. To fix that problem we are checking the existence of an specific DOM Node through a listener. However, this listener has a timeout and when it exceeds the timeout its going to stop the downloading process of that specific file with the error  ERR. Download aborted | Timeout of Xs exceeded.
So a workaround for this error is to give a bigger downloadTimeout value.

Unfortunately, I know a little about Typescript, I try to explain what I did, I hope it helps we can find the root cause:

Build the image with the Dockerfile

FROM node:18.9-slim

RUN apt update && \
    apt install -y fonts-liberation libappindicator3-1 \
    libasound2 libatk-bridge2.0-0 libatk1.0-0 libc6 libcairo2 \
    libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgbm1 libgcc1 \
    libglib2.0-0 libgtk-3-0 libnspr4 libnss3 libpango-1.0-0 libpangocairo-1.0-0 \
    libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 \
    libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 lsb-release \
    gconf-service libgconf-2-4 libgdk-pixbuf2.0-0 \
    wget xdg-utils xvfb ca-certificates libappindicator1 && \
    rm -rf /var/lib/apt/lists/*

RUN npm install -g figma-backup
CMD ["figma-backup"]

with this command:
docker build -t figma -f Dockerfile .

Clone the repository:

git clone https://github.com/mimshins/figma-backup.git

Run the docker container with the volume:

docker run --name figma-backup -it  -v "${PWD}/figma-backup:/app" figma bash

Run the figma-backup command in the debug mode:

figma-backup -e "<YOUR_EMAIL>" -p "<YOUR_PASSWORD>" -t "<YOUR_ACCESS_TOKEN>" --projects-ids "ID1" "ID2" ... "IDx" --debug

give the following error:
Error: Failed to launch the browser process!
[19:19:0914/132053.471715:ERROR:browser_main_loop.cc(1409)] Unable to open X display.

TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md

    at onClose (/usr/local/lib/node_modules/figma-backup/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:203:20)
    at ChildProcess.<anonymous> (/usr/local/lib/node_modules/figma-backup/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:194:79)
    at ChildProcess.emit (node:events:525:35)
    at ChildProcess._handle.onexit (node:internal/child_process:291:12)

I found a workaround (this):

Xvfb -ac :99 -screen 0 1280x1024x16 &
export DISPLAY=:99

Now it does not complain, it aborts with timeout though.
> Starting the backup task...
> Authenticating the bot...
        . Bot successfully logged in.
        . Caching the cookies...
        . Looking for cached cookies...
        . Restoring the cached cookies...
> Backuping the file(sample_project)...
        . Setting the download behaviour...
        . Saving a local copy of the file(sample_project)...
        . Opening up the figma command pallete...
        . Typing down the download command...
        . Execute the download command...
        ERR. Download aborted | Timeout of 300s exceeded.

To find if this is the case for all the docker containers, I wrote a simple script, download.js and copy it to the /app directory:

const puppeteer = require('puppeteer');
const path = require('path');
const downloadPath = path.resolve('/app');
async function simplefileDownload() {
    const browser = await puppeteer.launch({
        headless: false,
        args: ['--no-sandbox']
    });

    const page = await browser.newPage();
    await page.goto(
        'https://unsplash.com/photos/tn57JI3CewI',
        { waitUntil: 'networkidle2' }
    );

    await page._client.send('Page.setDownloadBehavior', {
        behavior: 'allow',
        downloadPath: downloadPath
    });
    await page.click('.mef9R ')
}
simplefileDownload();

Run node download.js would download but it stuck and did not exit. I think it is why we got timeout and the project might be downloaded and simply we do not know where it is placed.

Do you have any idea about that? and could we have an option to specify the downloadPath?

@monegim

Do you have any idea about that? and could we have an option to specify the downloadPath?

The reason it gets stuck is that you can't exactly tell when the download process ends so you could close the page and terminate the puppeteer.
The same thing is going for Figma with a small difference. When we actually downloading files there will be a Snackbar (Toast) component showing the download progress.
await page.waitForFunction(
  () => !document.querySelector('[class*="visual_bell--shown"]'),
  { timeout: downloadTimeout }
);

Here I wait for that component to hide from the DOM Tree.
Note: I'll wait for the specified download-timeout option (defaults to 5m or 300s).
There might be an issue with Figma not showing this component to your machine. However, even if the download aborted due to timeout excess, it would've completely downloaded the file in the relative download path (if the file is small enough to be downloaded in that time).
Unfortunately puppeteer doesn't support specifying absolute download path.
It is relative to the directory which you ran the command.
In figma-backup I'm creating a directory called figma-backup-root relative to the working directory.
It's working fine both for MacOS and Windows, but unfortunately it has known issues for Linux.
This issue definitely needs further investigations on Linux Machines and Docker Containers.

@monegim

Now it does not complain, it aborts with timeout though.

Can you lookup for figma-backup-root directory after the error occurs? (or any .fig files)

They are tracking this bug for 3 years and yet It's not fiexd, read more here

Can you lookup for figma-backup-root directory after the error occurs? (or any .fig files)

Sure @mimshins jan,
I ran in both --debug mode enabled and disabled, unfortunately, nothing is there:
figma-backup-root/
├── backups
└── cookies.json

1 directory, 1 file

Could you see the bot opening the chromium and does the task?

@mimshins jan, I execute the command in a Linux server, so I cannot see opening the browser.

@me-dira
Thanks Mehdi,
I will consider the solutions you've provided if I couldn't resolve the issue.

Thanks @monegim,

figma-backup-root/
├── backups
└── cookies.json

1 directory, 1 file

So based on the results you gave me, it seems that you could find the figma-backup-root. Now based on the codebase:
downloadPath: path.join(
  BACKUP_DIR,
  SESSION_DATA.date!.toISOString(),
  projectName
)

The backups should be presented in the backups directory. The path to the files should be something like figma-backup-root/backups/2022-09-15T08:09:08.817Z/PROJECT_NAME/file1.fig.
Can you confirm?

The backups should be presented in the backups directory.

Yes, but it is empty.

I'm trying to find other ways to download files that don't depend on the browser and operating system.
I'll inform you the further information and progression.

Update:
@mimshins jan,
I executed the figma-backup in windows too. It ran successfully, however, figma-backup-root is also empty.
 Directory of C:\Users\Mostafa\figma-backup-root\backups

09/15/2022  06:32 PM    <DIR>          .
09/15/2022  06:33 PM    <DIR>          ..
               0 File(s)              0 bytes
               2 Dir(s)  85,476,380,672 bytes free

The .fig files have been downloaded in the Downloads dir. Should it be so?

Update:
@mimshins jan, I executed the figma-backup in windows too. It ran successfully, however, figma-backup-root is also empty.
 Directory of C:\Users\Mostafa\figma-backup-root\backups

09/15/2022  06:32 PM    <DIR>          .
09/15/2022  06:33 PM    <DIR>          ..
               0 File(s)              0 bytes
               2 Dir(s)  85,476,380,672 bytes free

The .fig files have been downloaded in the Downloads dir. Should it be so?
I also searched for any file with the .fig extension, but I could not find anything in the container:
root@db2fcca94266:~# find / -regex '.*\.fig$'
root@db2fcca94266:~#

Well this is an issue with Windows OS. It doesn't support default download paths.

@monegim
I have patched a few things, there's a good chance that the problem with Linux Machines has been fixed.
Please try out v1.0.4 and give me feedback.
Thanks.

@mimshins jan,
It is resolved, now I can see the file in this directory:
/figma-backup-root/backups/2022-09-16T18:19:44.662Z/Team project

@mimshins jan,
It is resolved, now I can see the file in this directory:
/figma-backup-root/backups/2022-09-16T18:19:44.662Z/Team project

Just a minor issue, when I ran the figma-backup --version, I still got 1.0.3:
root@664265bc2f08:~# figma-backup --version
1.0.3

Just a minor issue, when I ran the figma-backup --version, I still got 1.0.3:
root@664265bc2f08:~# figma-backup --version
1.0.3

Oh I see!
I'm going to upgrade the puppeteer to its latest version (v17) and release<EMAIL_ADDRESS>in few days and I'll fix that version issue as well.
Thanks for your contribution ❤🤝


Rendering attachment images causes performance issues

When using virtual list libraries such as react-virtuoso, rendering attachment images causes performance issues. This can probably be solved by using some sort of lazy loading technique for images.

Video of issue.
https://github.com/widgetbot-io/message-renderer/assets/94828874/f8589e74-4005-4600-944b-d2c322377e4e


Using "parameter: List[str] = Form(...)" produces unusable Swagger

First check

[x] I added a very descriptive title to this issue.
[x] I used the GitHub search to find a similar issue and didn't find it.
[x] I searched the FastAPI documentation, with the integrated search.
[x] I already searched in Google "How to X in FastAPI" and didn't find any information.
[x] I already read and followed all the tutorial in the docs and didn't find an answer.
[x] I already checked if it is not related to FastAPI but to Pydantic.
[x] I already checked if it is not related to FastAPI but to Swagger UI.
[x] I already checked if it is not related to FastAPI but to ReDoc.
[x] After submitting this, I commit to one of:

Read open issues with questions until I find 2 issues where I can help someone and add a comment to help there.
I already hit the "watch" button in this repository to receive notifications and I commit to help at least 2 people that ask questions in the future.
Implement a Pull Request for a confirmed bug.

Example
Here's a self-contained, minimal, reproducible, example with my use case:

from fastapi import FastAPI, Form, Response
from typing import List

app = FastAPI()

@app.put("/test")
async def test(mylist: List[str] = Form(...)):
  return Response(content=str(mylist))

Don't forget to install python-multipart to support Form parsing.
Description

Open the browser at /docs,  add two entries a and b in the Swagger UI and call the endpoint /test.
It returns the string representation of the parameter mylist: ['a,b']
But I expected it to return ['a', 'b'].

The Swagger UI generates the API call like this (I put the relevant -d parameter first for legibility):
curl -d "mylist=a,b" -X PUT "http://<IP_ADDRESS>:8081/test" -H "accept: application/json" -H "Content-Type: application/x-www-form-urlencoded"

It should generate it like this in order to be parsed correctly by FastAPI:
curl -d "mylist=a&mylist=b" -X PUT "http://<IP_ADDRESS>:8081/test" -H "accept: application/json" -H "Content-Type: application/x-www-form-urlencoded"

Environment

OS: Linux

FastAPI Version: 0.58.1

Python version: 3.6.9

Additional context
This does work correctly for Query parameters. After replacing Form  with Query in the example above, Swagger generates the following API call:
curl -X PUT "http://<IP_ADDRESS>:8081/test?mylist=a&mylist=b" -H "accept: application/json"

in which case the route correctly outputs ['a', 'b'].
This probably has something to do with the OpenAPI serialization option, but I am not versed enough on this topic to be sure.

Any idea how to solve this issue? I am facing the same thing.

Same issue here!
I don't see an option to provide explode=True to the form param!

Having the same problem! Was having a hard time trying to figure out what I did wrong :( I eventually traced it back to this
Couldn't find much on this issue. Perhaps it's not a common way Form() is used

This is not a FastAPI issue but a SwaggerUI issue, no?
have you tried in Redoc?

have you tried in Redoc?

Redoc does not have "Try it" feature.

The Try it feature is not available in Redoc. You will need access to Redocly API docs or Developer portal products to use the Try it feature.

And yes, its a SwaggerUI issue IMO.


Enhance public API documentation related to parts exposed to model modifiers and listeners

Robot Framework's programmatic API is documented at http://robot-framework.readthedocs.org. The documentation is not that comprehensive and could be enhanced in many ways. This issue covers enhancing the parts that are exposed to pre-run and pre-Rebot modifiers (added in RF 2.9, #1976) and listener v3 (added in RF 3.0, #1208). In practice this issue covers enhancing documentation of running and result model objects and the visitor interface.

Tasks related to this issue:

[ ] Review and enhance visitor documentation. On a quick look it seems to be in pretty good shape already.
[ ] Add links to related execution and result objects into User Guide's pre-run and pre-Rebot modifier sections.
[ ] Make sure all methods and attributes of running objects are documented. This includes also classes used by these objects.
[ ] Same as the above but with result objects.

See #2275 for another issue making API docs better.

Enhanced API docs are now ready for review. You can see the commits containing changes above and http://robot-framework.readthedocs.org/en/master/ contains the latest generated version. Direct links to the most important docs related to the listener v3 and model modifiers are these:

Execution related model objects: http://robot-framework.readthedocs.org/en/master/autodoc/robot.running.html#module-robot.running.model
Result related model objects: http://robot-framework.readthedocs.org/en/master/autodoc/robot.result.html#module-robot.result.model
Visitor interface: http://robot-framework.readthedocs.org/en/master/autodoc/robot.model.html#module-robot.model.visitor

Comments, enhancement ideas, fixes, etc. related the docs are highly appreciated!

I did read the doc and couldn't think anything to improve. Mainly because I haven't used those functionalities that much.

Ok, thanks for the review @aaltat . You know where to submit issues/PRs if you later find ways to enhance the docs.

Awesome stuff, a great improvement for the docs. It is a bummer that we the inherited attributes dont inherit the docs, but It is still a lot better than before...
I couldnt find anythig to improve, but to be honest it is difficult for me to read this api like I was a newcomer to this...


Update CsvRowDataSerializer.cs to handle multiline values

Basically checks if an value contains an \n cause that should denote a new line, and that should fix problems with multiline strings

Hi @GreatNovaDragon, please add some unit tests.


Update broken links to docs in changelog.md

@ttmc: I changed the links in the release notes already.
Thx to @sadysnaat for telling us about this.

Codecov Report

Merging #1179 into master will not change coverage.

@@           Coverage Diff           @@
##           master    #1179   +/-   ##
=======================================
  Coverage   96.92%   96.92%
=======================================
  Files          49       49
  Lines        2279     2279
=======================================
  Hits         2209     2209
  Misses         70       70

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6aac745...67194e3. Read the comment docs.

Maybe I could just make more of the recent versions visible, e.g. v0.9.0 and v0.9.1 --- and (in the future) v0.9.2?

If you mean by "visible", that links to http://docs.bigchaindb.com/.../v0.9.0/... work again, then yes. That would help. Up to you.

Okay, the URL https://docs.bigchaindb.com/projects/server/en/v0.9.0/ was made "active" and "Public" on ReadTheDocs: it works.
I also made v0.5.1 inactive and private, so the list of versions stayed the same length.


Get metadata about the cache entries

Hi,
It's very useful already how we can call cache.size(). What would you think of also exposing something like cache.entries() that gives you the list of keys, when they were last updated, and when they expire?

Hmm, could be useful. I worry people will use it in their normal control flows instead of just for analytics. How about only when debug mode is on?

That's a good point, it should only be for analytics (e.g. logging cache stats every minutes, etc...).
I wouldn't want to turn on debug: true on production though because console.log has performance impacts. Would you be OK with a new stats: true then, that gathers stats when keys are set, and can be retrieved with cache.stats()? This shouldn't be noticeable on execution speed, and still be obvious it's meant for reporting purposes.

That, or instead of returning the data, console.log() it so you clearly have to jump through hoops to use it programmatically.

The way we'd like to use it though is for live reporting / monitoring. No runtime logic depends on it, but it still runs on production to gather and report data. For example, we can expose the age of key XYZ, and another process can monitor it & raise an alert if it wasn't updated in the last 5 minutes.
console.log could work (the other process just trawls the logs), but sadly it's a blocking IO call, so should never be turned on in production  :frowning:
Some more flexible options would be logging the stats using our logging framework (Bunyan or any other), exposing the info from a /status route, or pushing the key's age (lastUpdate - Date.now()) to NewRelic. All 3 options Require the data to be available in JavaSript though. Thoughts?

Wow, I really dropped the ball on this. Did you solve it yourself? Thinking a bit more, adding something that makes you turn on debugging mode before you use it seems like the best option.

Hi, no problem. We ended up maintaining another structure with the last updated date of each key.
Still interested in something like cache.stats(), which could return an empty object by default, unless you use the {stats: true} option in which case you can get key ages, cache size, etc...

It would be the same overhead for either. Checking a boolean or checking if there is an event callback. The boolean is simpler and I always like simplicity over extensibility, so I'd recommend that one if you coded it up I'd accept a PR.

Sounds great thanks! Do you see { memSize, hitCount, missCount } moving into the stats structure?

Hmm. So debug is about spewing notices on events and stats is about introspecting the system while it is running? Sounds good.

:+1: that's how I saw it, where you would not hopefully never turn on debug in production for too long (given console.log is blocking), but it's OK to turn on stats if you're interested in that.


Test Failing - fflib_QueryFactoryTest::addChildQueries_invalidChildRelationship

While I am debugging the cause of this, I would like to know if someone has faced the same issue?
I have latest commit '748caa4c091edc9cebf320bf5ee4a5c8aa29d2b9'
if package.xml is modified to have version as '36.0' instead of '31.0', then above test fails.
the fix in test is as below diff snippate.
I am not sending this as pull request as I am not sure if library wants to upgrade on version 36.0.
diff --git a/fflib/src/classes/fflib_QueryFactoryTest.cls b/fflib/src/classes/fflib_QueryFactoryTest.cls
index 4f21576..8daf17b 100644
--- a/fflib/src/classes/fflib_QueryFactoryTest.cls
+++ b/fflib/src/classes/fflib_QueryFactoryTest.cls
@@ -503,7 +503,7 @@ private class fflib_QueryFactoryTest {
Schema.DescribeSObjectResult descResult = Account.SObjectType.getDescribe();
Exception e;
try {

                  fflib_QueryFactory childQf = qf.subselectQuery(Contact.SObjectType);

                  fflib_QueryFactory childQf = qf.subselectQuery('Contact');
                childQf.selectField('Id');
        } catch (fflib_QueryFactory.InvalidSubqueryRelationshipException ex) {
                e = ex;

closing as this was just for information

FYI @frup42

As of API v34, the global describe map keys used the fully qualified names of objects, rather than the local names (e.g. MyNameSpace__MyObject__c rather than MyObject__c). So mixing and matching API versions across classes isn't advisable.
I've just run all unit tests with Apexlib classes all on v31 and all on v36. The full suite passed in both cases (apart from an ApexMocks test which expected different instanceof null behaviour, but that's another matter).
Closing as I don't think there's anything we need to do here.

I still experience this without having actively changed anything. The package.xml might have been updated by an upload or my IDE. I suggest you just adjust the versions in the library.

@up2go-rsoesemann agree, about time we increased the API version, changing this to an enhancement. THanks.

Any quickfix here so I don't have to comment out this test?
@frup42 @afawcett

@up2go-rsoesemann firstly, happy new year! 👍
Secondly, i see that the Travis build for this repo is passing, so it looks like the latest code is ok. So i'm confused. Are you getting this on the latest code?

I think @up2go-rsoesemann 's pull request #151 fixes this - looking for null objectType rather than 'Contact,' so no namespace worries. Please reopen if you're still getting this on the latest commit!


spermatogenesis should be a cell development or differentiation

Currently oogenesis is a cell development, but spermatogenesis is not. Investigate why this might have been the case and add the appropriate parent if it should be added.

Interestingly, this process includes all of the steps involved in generating sperm, which as parts include all of the cell types that give rise to the mature gamete. So the suggestion in the title of this ticket will not work. However, this should still be classified as a developmental process.

https://github.com/obophenotype/cell-ontology/issues/574

spermatogenesis- The developmental process by which male germ line stem cells self renew or give rise to successive cell types resulting in the development of a spermatocyte (mature sperm?).
Still needs a bit of work, but it's a start. @vanaukenk or @hattrill any suggestions? We want to include the C. elegans biology here as well as the germ-line cyst biology of the fly.

https://github.com/obophenotype/cell-ontology/issues/575

Hi David,
Yes for sure bring it under developmental process. Interesting that stem cell renewal included, but is consistent with structure and oogenesis.
So, for us, the "mature sperm" is the end point.
Mushing some decriptions together, particularly from PMID:24798812:
Germline stem cells divide asymmetrically to produce two cells: one cell maintains stem cell characteristics, while the other daughter cell (a gonialblast) initiates differentiation. Gonialblasts undergo 4 rounds of mitosis with incomplete cytokinesis to generate a cyst of 16 spermatogonia. Spermatogonia initiate terminal differentiation as spermatocytes, which undergo 2 meiotic divisions with incomplete cytokinesis to generate 64 haploid spermatids. Within the syncytial cyst, elongation and maturation of the spermatids, spermiogenesis, is the final stage of spermatogenesis, after which the sperm is transported to the seminal vesicle.

@ukemi  - Yes, as we discussed, 'spermatogenesis' should definitely be a developmental process.
The new definition above should work for C. elegans, but do we want spermatogenesis to end in formation of mature spermatozoa (mature sperm) rather than spermatocytes?
Here's a nice summary statement from a recent C. elegans paper:
"Spermatogenesis is a conserved multistep process that involves mitotic proliferation of spermatogonia, meiotic division of spermatocytes, and differentiation of haploid spermatids into
mature spermatozoa."
https://www.ncbi.nlm.nih.gov/pubmed/30990821
Interestingly, we have terms for 'spermatid development' and 'spermatid differentiation' but not 'spermatozoa development' or 'spermatozoa differentiation'.
I should review the C. elegans annotations to the spermatid terms to make sure they're still okay.
Note: also updating the definition of 'pseudocleavage' since it happens in spermatids as well as embryonic cells.

This paper defines spermatogenesis as ending with the mature spermatozoa as well:
https://www.ncbi.nlm.nih.gov/pubmed/28073824

It should certainly end with the mature sperm. My mistake above.


Add drawing dependencies

Fixes #1.
A package for gtk3 was recently added to conda-forge, so we can now support drawing.  (Due to a minor snag with librsvg, interactive windows won't work until conda-forge/librsvg-feedstock#43 is merged, but that should be imminent.)

Checklist

[x] Used a fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)

Hi! This is the friendly automated conda-forge-linting service.
I wanted to let you know that I linted all conda-recipes in your PR (recipe) and found some lint.
Here's what I've got...
For recipe:

Failed to even lint the recipe, probably because of a conda-smithy bug :cry:. This likely indicates a problem in your meta.yaml, though. To get a traceback to help figure out what's going on, install conda-smithy and run conda smithy recipe-lint . from the recipe directory.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.


feat: add logos to homepage

Closes #10

Apparently, the website has to be checked before being published:
https://inlg2024organizers.slack.com/archives/C06QUJ54A56/p1711516089295079


serial wifi setup failures w/Photon

$ p setup wifi

? Should I scan for nearby Wi-Fi networks? Yes
? Select the Wi-Fi network with which you wish to connect your device: SSID_NAME
? Should I try to auto-detect the wireless security type? Yes
Assertion failed: (timeout != -1), function uv__io_poll, file ../deps/uv/src/unix/kqueue.c, line 160.
Abort trap: 6

$ p setup wifi

? Should I scan for nearby Wi-Fi networks? Yes
? Select the Wi-Fi network with which you wish to connect your device: SSID_NAME
? Should I try to auto-detect the wireless security type? No
! Something went wrong: Serial port closed early

What version of node is that? Thats the first time I've seen a stack trace inside of libuv.

I'm on v0.10.36

Hit this again today using latest CLI source
 p setup wifi
? Should I scan for nearby Wi-Fi networks? Yes
? Select the Wi-Fi network with which you wish to connect your device: CoCo 1 Gb Powered by Comcast
? Should I try to auto-detect the wireless security type? No
Assertion failed: (timeout != -1), function uv__io_poll, file ../deps/uv/src/unix/kqueue.c, line 160.
Abort trap: 6

also just hit this:
p setup wifi
? Should I scan for nearby Wi-Fi networks? No
! Something went wrong: [Error: Cannot open /dev/cu.usbmodem14511]

Knowing the firmware version would be useful. I was not able to repro on 0.5.3


jest: sporadic "failed to cache transform results" error

I'm observing sporadic errorneous behaviour when running tests.
janosimas reported the same issue here: #175

Observation: For the last 4 occurrences, the error message is about file random.alea.js.
Not true - sporadic on any test file.

The issue is already reported: https://github.com/facebook/jest/issues/4444
There is a sort-of fix to use a patched version of graceful-fs.

I've applied patch-1 version of graceful-fs and it looks like it's working fine.
Would you mind a PR for this? We could leave this issue open to be reminded about jest behaviour, and eventually undo it should there ever be an official fix. Spoiler: It does not look like being fixed anytime soon, if ever.
Maybe I can apply that line at my machine locally only, so no source change is necessary. Is there a way to perform that, other than myself being really careful about package.json modifications?

I witness this error on a regular basis, in case of PR #252 , I had a streak of 4-5 times where that error occurred.

Hmm, I just ran npm installon a clean version of the repo and I see:
+        "graceful-fs": {
+         "version": "4.1.11",
+          "resolved": "https://registry.npmjs.org/graceful-fs/-/graceful-fs-4.1.11.tgz",
+          "integrity": "sha1-Dovf5NHduIVNZOBOp8AOKgJuVlg=",
+          "dev": true
+        },

in the diff of package-lock.json (under the dependencies section of jest).
I'm wondering if merely adding the patch to graceful-fs at the top level actually fixes it as a dependency for Jest, i.e. does Jest now use the patched version or does it continue to use the unpatched version?

There is a separate question of should I commit this package-lock.json diff.  Ideally running an npm install on a clean repo should result in no diffs.

I don't know. Searching for graceful-fs yields that all sorts of different versions are requested... (4.1.2, 4.1.4, 4.1.6, 4.1.11).
Looks like npm writes dependency information inside the package.json of the module itself, in case of graceful-js it looks OK though:
 "_resolved": "git+https://github.com/mekwall/node-graceful-js.git#a10aa576f771d7cf3dfaee523f2e02d6eb11a89f",
, which is exactly HEAD of the patch-1 branch.

Unfortunately I continue to observe the error. Because of above I think npm resolved to the patch-1 version; if so, I'll just have to accept jest behaviour.

So I performed a fresch clone, run npm i, and there are a lot of diffs inside file package.lock. I'm not sure but looks like this is dependend on the npm version? I'm on npm version 6.4.0, node version 10.5.0.
Unfortunately the package.lock now looks a bit different, there is no mention of the patch-1 branch anymore, but only to the commit hash (albeit the right one).

Is this still needed? Having this as dependency breaks things with pnpm.

Perhaps it is indeed time to evaluate if this can be removed.  Thanks for looking into this @ksjogo!


Breaking Change - Remove overridden createJSModules

Required for React Native 0.47+
Reference commit: facebook/react-native@ce6fb33

Thank you for the PR. It's good to know that we need to do this for RN 0.47. A couple of thoughts:

The Android unit tests failed in CI. I haven't looked closely to see where and why, but it might be related to this change. It's probably because the tests (devDependencies) use an earlier version of RN. You can see the failure for yourself using the runAllTests script in the repo or using fastlane test_android.
A related issue: 2.0 is a fine place for a breaking change, but this means that we'll have to increase the peerDependency for RN to >= 0.47. This would mean that anyone using 1.x would be stuck until/unless they upgrade to 0.47. We still get reports from devs using RN 0.39. Version 2.0 is nearing release and is intended to replace 1.x entirely. I'm reluctant to increase the required version of RN in 2.0. But of course, without this change, you can't use RN 0.47.

We can either release 2.0 with RN >= 0.40 and then quickly release, say, 3.0 for 0.47 (major version increment required by semver rules for a breaking change). I'm not sure this is the best solution. We can probably find a trick either using automation (RN postlink hook) to modify the source depending on the RN version or maybe we can have two different versions of the module in the package for different versions of RN.
I've been working on another problem this week, but I hope to get the final beta release of 2.0.0 out soon, followed by a prod release. I'll look into the best way to support RN 0.47. Any suggestions (e.g. other SDKs solving this problem) are welcome.

A possible approach to this issue:

Rename RNBranchModule (to RNBranchModuleBase, e.g.).
Make RNBranchModuleBase an abstract class.
Add two subclasses of RNBranchModuleBase: RNBranchModule (for >= 0.47) and RNBranchLegacyModule (for < 0.47).
In the docs, indicate that you should use RNBranchLegacyModule for < 0.47 and RNBranchModule otherwise.
Address the choice of class with automation when possible.

The base will have to be abstract in order to compile with 0.40, since it will be missing a method. I think something like this should work.

This was addressed by #231. Removing the @Override is not a breaking change and works well with RN >= 0.47 and RN < 0.47.


Small chance to compile wrong page with ruby 2.7

Under Ruby 2.7 there is a very small chance that nanoc compile will use the wrong content for a page when a large number of pages change at the same time
Steps to reproduce

use ruby-2.7
Make a small change that affects all pages, like adding a single character to the layout
nanoc compile
Clear the cache to force a recompile rm -rf tmp
nanoc compile --diff

Expected behavior
The second compile should show no changes.
Actual behavior
About 2 to 4 of every 1000 pages changes, to the correct version. Subsequent nanoc compile commands show no changes.
Details
It seems to work OK on ruby 2.6 and it also works fine on smaller repos and smaller changes. Only during a compile that affects 1000 pages does the compilation get confused.

This bug is disturbing, and even more disturbing is that I am unable to reproduce it :(
@Fjan Can you check something for me? Drop the code snippet below into your site’s lib/ directory (location does not matter), and then try to reproduce the issue again. Can you reproduce it with this code snippet in place?
module Nanoc
  module Core
    module CompilationPhases
      class Write < Abstract
        include Nanoc::Core::ContractsSupport

        def initialize(compiled_content_store:, wrapped:)
          super(wrapped: wrapped)

          @compiled_content_store = compiled_content_store
        end

        def start
          super
        end

        def stop
          super
        end

        contract Nanoc::Core::ItemRep, C::KeywordArgs[is_outdated: C::Bool], C::Func[C::None => C::Any] => C::Any
        def run(rep, is_outdated:) # rubocop:disable Lint/UnusedMethodArgument
          yield

          # Caution: Notification must be posted before enqueueing the rep,
          # or we risk a race condition where the :rep_write_ended
          # notification happens before the :rep_write_enqueued one.
          Nanoc::Core::NotificationCenter.post(:rep_write_enqueued, rep)

          writer = Nanoc::Core::ItemRepWriter.new
          writer.write_all(rep, @compiled_content_store)
        end
      end
    end
  end
end

Yes, I believe that fixes it. I did about ten compiles with the patch, each time changing a single character and then doing two compiles, and it appears to work flawlessly.

#1507 should fix this!

Disappointingly, I still managed to get an incorrect compile once when using nanoc 4.11.18 on ruby 2.7. It's hard to reproduce though, it takes a lot of recompiles to get one that goes wrong (on an 8-core iMac Pro).

@Fjan Does it happen with the code snippet mentioned above, too?

I wrote a little script to automate the problem of changing a character in the layout and compiling twice to help detect the issue. I first tried it without the patch. I let that run for half an hour and... wasn't able to reproduce it at all anymore. I would swear I've seen it happen once when I did it manually but also several manual attempts no longer revealed the issue. So I suggest closing this for now until I find a reliable way to reproduce it again.

I've re-opened this issue. Over the last month it has happened to me twice that items got compiled to the wrong name. A recompile fixed it, but it's still rather serious if you don't happen to notice the change. So the issue is very rare, but there is still a race condition in nanoc somewhere that gets exposed by Ruby 2.7.

@ddfreyne @Fjan While I can't reproduce this consistently, we just recently updated to Ruby 2.7 and now we are starting to see random failures in our CI pipeline, which compiles ~1300 files each run. For example: https://gitlab.com/gitlab-org/gitlab/-/jobs/791090474#L2750. The pipeline job runs many times a day, so we have a lot more potential for running into this. Rerunning the job seems to clear it every time.
In the error I listed above, the file causing the error only has two lines, and most certainly does not have the 4 links reported in the error: https://gitlab.com/gitlab-org/gitlab/-/tree/master/doc/user/project/status_page. Nanoc seems to have compiled it with the content from a different file. I didn't dig into it to see which file it took the contents from, but I can try to figure it out if that helps.

Oh, I had naïvely assumed that the issue would lie with the caching, but now I see that it also happens with a fresh compilation.
Now, if only I could reproduce it still. I have managed to get the error twice myself locally, but never again since.
@Ravlen If you apply the patch suggested in https://github.com/nanoc/nanoc/issues/1501#issuecomment-643439217, does the problem go away?

@ddfreyne The chance seems higher with a fresh compilation, I don't think I've seen it happen when only a few pages change. But that could be just because the chance is dependent on the number of pages that change.

Yep — I can reproduce it now, but not with the patch in the comment. Can anyone confirm? (I’m running a long-term test now.)
If that patch actually fixes the problem, then my task is to figure out why… which is still a mystery to me.

Ok, I've added the patch to our repo (minus the include Nanoc::Core::ContractsSupport line since that resulted in warnings). But it's going to take a while to build confidence that it fixes it, because the issue is intermittent.

I believe I have found the culprit: a non-threadsafe object used across threads. A fix is in #1510.
More proof that concurrency is not for the faint of heart.

Will release a fix for this soon.

This is now fixed in the just-released 4.11.19. (It might take a few minutes before that release becomes available for installing.)


Implement I/O commands

Commands identical to CBM BASIC's OPEN, CLOSE, CMD, PRINT#, INPUT#, GET#, STATUS should be implemented in version 3.

https://github.com/Thraka/xcb-ext-io will solve this issue.


Required performace..

I plan to make a BMS using Atmega32u4 (16Mhz) and SN65HVD230 fro ArduPolit.
would you can the Atmage32u4 is enough, or should I go straight for STM32F103 (or some other, small-footprint STM?)

atmega32u4 is a not the best for CAN. low on memory etc. Our earlier versions where on this chip, but newest v4 is the Atsam series
https://rotoye.com/batmon/BatMon-v4-02-with-SMBUS-CAN-FD-LIN-2-packs-p273850043. The entire firmware is available with the purchase of the board.

atmega32u4 is a not the best for CAN. low on memory etc. Our earlier versions where on this chip, but newest v4 is the Atsam series
https://rotoye.com/batmon/BatMon-v4-02-with-SMBUS-CAN-FD-LIN-2-packs-p273850043. The entire firmware is available with the purchase of the board.

I don't know which CAN controller you are going to use. If you plan to use MCP2515 with the 107-Arduino-MCP2515 library please note that this only works with samd,mbed,esp32.
So I suggest to stay with these architectures.
At the moment the 107-Arduino-UAVCAN library is also only tested with samd and mbed.

I don't know which CAN controller you are going to use. If you plan to use MCP2515 with the 107-Arduino-MCP2515 library please note that this only works with samd,mbed,esp32.
So I suggest to stay with these architectures.
At the moment the 107-Arduino-UAVCAN library is also only tested with samd and mbed.

Hi @AndKe :wave: Both @echoGee and @generationmake are correct, atmega32u4 might be problematic due too low RAM and is definitely problematic because ArduinoCore-AVR does not support the C++ standard used within this library.

Hi @AndKe :wave: Both @echoGee and @generationmake are correct, atmega32u4 might be problematic due too low RAM and is definitely problematic because ArduinoCore-AVR does not support the C++ standard used within this library.


Make DiscordIpc trait object safe

Decided to make this in order to have an ability to create Box<dyn DiscordIpc>, Arc<dyn DiscordIpc> etc. To achieve this I have removed generic type parameters from the functions of DiscordIpc.

Done.


Update the go version for all the CI

Right now we are using 1.11.6 as go version for all the CI and our internal build system going to be 1.12.8 to compile the 4.3 installer side so we need to update the go version to CI also.

Fixed by #800


Can't run cluster_cells function more than once

Please refer to our FAQ and look at our known issues before opening a bug report.
This is once again Candace and Hadeesha's account @cliu72
Describe the bug
When you try to run the cluster_cells function more than once, there is an error. Reason seems to be because of these lines:
https://github.com/angelolab/ark-analysis/blob/7aadf27cb64e16fd0c293e1d1a36fb644c9a5425/ark/phenotyping/cell_cluster_utils.py#L543-554
Once you run cluster_cells one time, it adds the 'cell_som_cluster' column onto the cluster_counts_size_norm table, and those lines linked above checks to make sure that cluster_counts_size_norm and weights have the same column names. That is no longer true after you've ran cluster_cells once.
Expected behavior
Allow users to run cluster_cells more than once (this is probably not going to be common at all, but I still think it'll be better to support multiple runs).
To Reproduce
Please either copy/paste or screenshot all of the code you ran which produced the error. In addition, copy the full error message.

@cliu72 (or @HPiyadasa lol don't which account's used) I think the easiest way to do this would be to specify an overwrite flag for both train_pixel_som/train_cell_som and cluster_pixels/cluster_cells. In this way we can allow the user to a force SOM training and assignment to happen again.

@cliu72 alternatively we could also set the flag in the notebook instead, although I think it would be better to just leave as a hidden param due to the rarity of this use case.

Sure, I like this solution. Adding the overwrite flag in the function call sounds good to me.

@cliu72 after looking into this issue more, because cluster_cells and cell_consensus_cluster always run through the whole pipeline regardless of state, it's better not to add an overwrite flag for these two functions. Instead, we'll just check if the column 'cell_som_cluster' already exists in cluster_cells.
I've added overwrite functionality to the entire Pixie pixel clustering process, as well as for cell SOM cluster training. I won't be adding an explicit variable in the notebooks to set overwriting, as this won't be common and can easily be found in the documentation.

So if the overwrite flag defaults to false, and someone runs the pipeline a second time, does that mean they will still get this error? Is the behavior different for different steps in the process?

@ngreenwald not this specific error because cluster_cells won't actually use the overwrite flag, I've instead added custom logic to ensure cell_som_cluster gets dropped if it already exists.
The PR I've linked to this issue encompasses the broader problem of what happens if a user needs to rerun a second time, since it's currently impossible to "break out" of the check that skips any processing if certain files already exist. It's not always as simple as ensuring a column gets dropped.


Add config as code support for credentials configuration

Pending incremental from jcasc for the export test to pass (https://github.com/jenkinsci/configuration-as-code-plugin/pull/1752)
Importing works fine with this PR alone
some minor jcasc testing cleanup done as well
Fixes https://github.com/jenkinsci/azure-keyvault-plugin/issues/92

Waiting on the companion PR to merge, but I will try to merge other PRs and releasing before to not have incrementals in release.

Propogating through atm, will merge and release bom and update bom, if you're in no rush I'd hold off

if you're in no rush I'd hold off

Doing some reviews/merge today, if not done today, will come back on it soon-ish after ;)

Possibly caused: https://issues.jenkins.io/browse/JENKINS-67170


Backport PR #1413 on branch 0.12.x (Fix codespell)

Backport PR #1413: Fix codespell

This fixes what it aims at fixing. Let's see about the ui-tests separately.


Update scams with privatix.pro latest scam

Hello,
This is a phishing site against Privatix.
Phishing site: http://privatix.pro
Original site: https://privatix.io
Technical details:
Domain creation date: Yesterday (2017-10-20)
Domain holder: No details - Privacy  Protected, and under CloudFlare
Malicious content: The attacker change the wallet address to his own wallet address - trying to steal money. (https://etherscan.io/address/0x071cf579507DD0661845694F709E68CF7Ea506a4)
More Details:
http://privatix.pro is an impersonating site, aiming to phish customers of Privatix, which is now in the middle of an ICO (selling digital currency).
Their original website is under https://privatix.io.
Privatix confirmed this is not their site.
http://privatix.pro had been purchased yesterday (Oct 20), and behind CloudFlare.
This is the 3rd impersonation attack on them in the last couple of days (previous ones were with privatix[.]live and privatex[.]io
The wallet address of the attacker is 0x071cf579507DD0661845694F709E68CF7Ea506a4
and it is different than the wallet address of the ICO (which started on Oct 19), and thus stealing investors money!!!


Foreign keys

Would it be possible to create a feature to create foreign keys to other tables?

@bbrakenhoff could you elaborate more on this?

Yes :). I found out that there is ColumnConstraint for this, though I cannot find out how to refer to the column of the other table I'm referring to. Maybe I should make an other issue for this, but I also could nog figure out to use a join and an order by clause... It seems impossible to chain these two.

@bbrakenhoff ah, yes. I will be sure to include REFERENCES as part of the FOREIGN KEY constraint in the next release.

@alexfu I would like to work on this new feature if you don't mind. Thanks.

Resolved in #47


chore: Create placeholder pull-request

I've created this pull-request using the sic-jss branch as a source branch to see if I can reproduce the following error that we've encountering in production (Cloud Console):

Error: Provider produced inconsistent final plan.
When expanding the plan for google_cloudfunctions2_function.webhook to include
new values learned so far during apply, provider \"registry.terraform.io/hashicorp/google\"
produced an invalid new value for .service_config[0].environment_variables: was null, but
now cty.MapVal(map[string]c ... (truncated error message, see full error in Revision.logs)

NVM, I was able to reproduce the error in https://github.com/GoogleCloudPlatform/terraform-genai-knowledge-base/pull/143/checks?check_run_id=27521991321. See build logs.

Thanks for the quick review, Drew! :)
I have updated the sic-jss tag and confirmed that the deployment works.


added sections to sidebar

create the side bar ui according to the ui decribed in the issue #1

something is breaking in the bigger screen, maybe the width. Try to debug and fix it and then I will merge your code.


Bad practice to exchange access token for access token

I think its bad practice to exchange an access token to an access token. If you want to do that you should properly use the refresh function. Even though it just the upgrade guide, I still think we should teach user the correct way of using the library

@Hanspagh agreed, thank you 👍


Preliminary support of pos 'line display' printing through Epson printer

This pulls in the changes by @Ousret.
This is useful for setups that have a display in the chain before the printer. It selects the display, resets it, sends text and then again selects the printer.

closing in favor of #224


Postgres fallback variables

Background
This pull request addresses an issue where environment variables might not be available, particularly when working with forks. To improve the contributor experience, we need to provide default values for these environment variables.
Changes 🏗

Updated the workflow to include default values for the DB_USER and DB_PASS environment variables.

The new configuration ensures that if the secrets are not available, the username and password will default to specific values that will allow tests to pass.

PR Quality Scorecard ✨

[x] Have you used the PR description template?   +2 pts
[ ] Is your pull request atomic, focusing on a single change?   +5 pts
[ ] Have you linked the GitHub issue(s) that this PR addresses?   +5 pts
[ ] Have you documented your changes clearly and comprehensively?   +5 pts
[ ] Have you changed or added a feature?   -4 pts

[ ] Have you added/updated corresponding documentation?   +4 pts
[ ] Have you added/updated corresponding integration tests?   +5 pts

[ ] Have you changed the behavior of AutoGPT?   -5 pts

[ ] Have you also run agbenchmark to verify that these changes do not regress performance?   +10 pts

do we even need to set passwords?

@ntindleI don't believe so. I removed the password variable fallback and set it to an empty password.

What happens if you just remove the whole username and password lines? What do the docs for the action say?

Also may want to check to make sure this is actually running your workflow file

@ntindle I'm waiting to see the documentation for removing the lines, but I did notice that my workflow file is running.

@ntindle After removing the lines for the password and username, the same issue is still occurring.

Seems like something is overwriting it. Maybe check the bottom of the file for any global env that may need to be swapped instead?

Looks like DATABaSE_URL is invalid. Maybe we set them back up  and try the original fix in both places top and bottom?

Oh ‘ ‘ isn’t a valid password lol, use postgres

@ntindle oops lol.. I was a bit tired last night to notice what I did

No worries, push an empty commit here to test it works when you do it :)


RequiredValidator with typed properties

Q
A

Yii version
2.0.49

PHP version
7.4

Is bugfix
Yes

Breaks BC?
No (not sure)

What steps will reproduce the problem?
$model = SomeModelWithTypedProperties();
$model->setAttributes([]);
$model->validate();

What is the expected result?
Validation error, Prop1 cannot be blank.
What do you get instead?
Typed property prop1 must not be accessed before initialization
Solution
Change RequiredValidator. See @DplusG/yii2-required-validator

I'm not sure if we should get back to this old thing already mentioned in https://github.com/yiisoft/yii2/issues/17954
What exactly is your use-case? I mean, how does your SomeModelWithTypedProperties look like?

`
class SomeModelWithTypedProperties extends Model
{
public string $prop1;
public ?string $optionalProp2 = null;
public function rules(): array
{
    return [
        [['prop1'], 'required'],
    ];
}

}
// I don't know current data like in https://github.com/yiisoft/yii2/issues/17954
// and I want to see which properties are really required at the language level (not like: public ?string $prop1 = null)
$model->setAttributes($post);
`
Thank you, I see in https://github.com/yiisoft/yii2/issues/17954 'prefer not handing it' from samdark
In this issue I suggest to change the programm error to the validation error.
I can suggest an internal validator initialized, But it's no so clear.
Validation error is what we received in php <=7.3. I show it in my repo
return [     [['prop1'], 'initialized'],     [['prop1'], 'required'], ];

Fixing typed properties in validation seems like a lost cause to me, due to how Yii handles validation in models: it first assigns value to property, and then validates it. So if you type property as string and someone pass array as value for it, it will throw fatal error before even validation begins.

Yes, likely that isn't fixable in Yii2.

Ok, thanks!
In my case, I did not try to solve typecasting. Only initialization.
Can I add packages from Yii3 to solve this problem?
Validator, hydrator, form-model?

True, but this is another case here. Yii is checking the attribute first before assigning it hence the error for not initializing. DplusG has a model with invalid state for $prop1 here. The thing with incompatible types and this problem both are developer issues.

DplusG - for your case this is simple, you must initialize the property.


Update loader.js

It seems like cystoscape.min.js has been relocated and is no longer accessible at: http://cytoscape.github.io/cytoscape.js/api/cytoscape.js-latest/cytoscape.min, however even from a valid address (e.g. https://raw.githubusercontent.com/cytoscape/cytoscape.js/master/dist/cytoscape.min.js) it appears to be uncompatible (the reason is beyond my understanding) from version 3.0.0 cytoscape.min.js does not work.
Using a legacy version available at cloudflare worked for me.

Ping @idekerlab - the library doesn't work without this change. Please merge and release a new version as soon as possible. 😄

This is needed to use the package. Any ideas when it will be approved?

Given the latest commit https://github.com/cytoscape/cytoscape.js/commit/c8aa6c4b2502aa9c5d6c2def122d1b1df8fc3c49 from @maxkfranz I've committed 4704cc2
do feedback - thanks!

@jorgeboucas Thanks for the update, but a good, robust, longterm solution would be to use npm+webpack or npm+browserify to build your loader.js file and manage its dependencies properly.  py2cytoscape should really only be using exact versions of the cytoscape package that have been tested and vetted -- on a per-release basis of py2cytoscape.
npm init && npm i -D browserify && npm i -S cytoscape would be the fastest/easiest way to set this up if you're unfamiliar with js tooling.  Then use an npm script like "build": "browserify loader.src.js -o loader.js".
Until this change is made, py2cytoscape is at risk of breaking just as it did before -- at any time.  An added benefit of this change would be that this software could be used offline.
A quick, temporary workaround would be to use unpkg or cdnjs, e.g<EMAIL_ADDRESS> Github is not a cdn, and pointing to a branch rather than a version number of a release is risky.


Support CPFP transactions

The CPFP will have multiple inputs, one of those must (obviously) be
the tx that needs CPFP. The builder will try to meet a certain package
feerate.

I went with a trait called CpfpableTransaction that encapsulates a transaction that can have CPFP, and is implemented on spends and unvaults (this way in revaultd I don't have to treat them separately, I can just throw them in the same map). The trait is the only way of creating a cpfp, and returns a PSBT as the signing is done externally

Heh, that's a great idea!

I pushed some fixes on top of your commits :)


RichTextRun add vertAlign attribute

PR Details

Description

Related Issue

Motivation and Context

How Has This Been Tested

Types of changes

[ ] Docs change / refactoring / dependency upgrade
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

Thanks for your contribution. I have added a check for vertAlign enumeration, update doc for set cell-rich text docs, and unit test based on your branch.


Remove JSON pretty print from $schema and $model topic payloads.

Thanks for the commit, but it isn’t changed everywhere; e.g. fanuc/{machine}/stat_info/1 is without indentation, but fanuc/{machine}/stat_info/$schema not. It looks like all topics with a dollar sign still use indentation.

Leaving output as is.


made outputOptions.environment.arrowFunction to false to generate fun…

Summary
When libraryTarget is set to module, the externals configuration appends hardcoded arrow functions that cannot be transformed
Fixes (#17861)
Details

Inorder to rectify this issue outputOptions.environment.arrowFunction is made to false in the basicFunction

Fixed here https://github.com/webpack/webpack/pull/17919


Semantic segmentation + running Hydra on custom data

First of all, thank you for releasing such an interesting work in the field of Semantic SLAM.
Considering that the Semantic Segmentation Network used by Hydra is not yet ready for publication, I'd like to ask if there are alternatives that can be considered in the meantime, including the use of other networks, or the supply of custom "ground truth" data (in that case I'd like to know which format Hydra expects).
Additionally, I was wondering if there are plans to add support for custom bag files/sequences, or alternatively adding instructions to do so, reutilizing the existing launch file infrastructure. We are very interested in testing Hydra with data recorded from our own robot, which is fitted with an RGBD camera, and it could also provide odometry information directly (nav_msgs/Odometry) instead of requiring a visual-inertial odometry component. It is also not presently clear to us which topics are needed as inputs to the system.

Thank you for your interest in our work!
On the choice of a 2D semantic segmentation network: We actually use an off-the-shelf network for 2D semantic segmentation as mentioned in our paper (we use HRNet from the MIT Scene-Parsing benchmark). The pre-trained pytorch models can be found here. Using other networks or sourced of ground-truth semantics is also possible as long as you post-process the semantic labels and set up your configs correctly downstream.
Actually setting up 2D semantic segmentation for use with Hydra (and Kimera-Semantics): The code we haven't released yet just loads in a ONNX model (which we exported from pytorch) into TensorRT, subscribes to a RGB image topic, runs inference on received RBG images, and then condenses and colors the resulting semantic label image. You could also do the same with pytorch directly (we just wanted to avoid python2 to python3 issues in 18.04). There are typically two steps when setting up a new segmentation network:

Kimera-Semantics is limited to doing inference on 20 categories, so it's likely you will have to condense the full set of predicted labels (from the 2D segmentation network) down into 20 groups. You can view our mapping here for the pre-trained model that I pointed to earlier.
We tend to convert each of the 20 condensed labels to an unique color to help with visualization (i.e. mapping label 0 to red, label 1 to green, etc.). Kimera-Semantics requires a CSV mapping colors to labels. An example CSV for the pre-trained network that we used is available here. Note you could also just use the label as one of the channels, but that makes it hard to distinguish the different semantic classes in the mesh

Once you have that, you just need to combine the depth and semantic image using this nodelet in depth_image_proc to obtain the semantic pointcloud that Hydra expects as input.
Running with a different pose source
Our plan is to add instructions for configuring Hydra to work for new datasets and sensors, but it may be a little bit before I have a chance to write them up, and they will be specific to Kimera-VIO when I do write them up.  That said, it's certainly possible to set up a launch file for a different sensor and odometry source. The base launch files that we share between our different datasets are here and here. You can look at how we specialize our launch files here and  here respectively.
The two inputs that are required for Hydra to function are a semantic pointcloud (by default, Hydra subscribes to /semantic_pointcloud) and a TF frame with the current pose of the sensor producing the pointcloud, which is specified by argument here.  Enabling our full loop closure detection pipeline is a little trickier. You'll want to make sure the arg use_gt_frame is set to true so that this node is actually enabled (which will save you having to publish the robot pose graph yourself).  Then you have to publish BoW vectors with this message to /kimera_vio_ros/bow_vectors. To run with LCD enabled, you just have to run with enable_dsg_lcd:=true. If you don't have a source for visual descriptors, the scene graph LCD should still work, though we've found that the performance can be a little hit-or-miss when just using the scene graph for LCD instead of also using the vision-based descriptors.
Finally, if you end up working on standing up your own pipeline, I'd kindly request that if possible, you take notes on the process (that we can add as documentation) to help others also use Hydra (your experience may be more informative to other users than the instructions that I write-up). Please also don't hesitate to reach out with other questions! I'll leave this issue open for now as temporary documentation as I'm sure others have similar questions.

Hi @feramozur, have you been successful in using Hydra on data from your robot?

Do you think ZED 2i camera is a suitable sensor setup to try Hydra for real-time use?

Hi, yes the ZED 2i is likely suitable for use as an input to Hydra (and it's on our list of cameras to try out). We've used similar cameras in the past for Hydra (the Kinect Azure for the RSS paper and the realsense D455 in some of our testing) though the underlying stereo technologies will probably affect the reconstruction quality in different environments (i.e. the time-of-flight sensor in the kinect vs. the active/passive D455 combo vs. the passive zed with a larger baseline).
If you're trying to run with Kimera-VIO, then the zed or the d455 are reasonable options: the zed reconstruction quality won't suffer but isn't rolling shutter, and the d455 is global shutter on the infrared cameras (I believe) but requires turning off the IR emitter which can reduce the depth quality.

Thanks for your reply! Im trying to run ZED2i camera with Kimera-VIO, but it doesnt work as expected. The mesh coming out of it appears to be chopped and broken, very far from the mesh I`m getting from the EuRoC dataset. I generated the config files for the camera using cam_info_yamlized.launch. And below is my kimera_vio_ros_zed2i.launch file itself.
<launch>
  <arg name="dataset_name" value="ZED2i"/>
  <arg name="online" default="True" />

  <arg name="log_output" default="false"/>
  <arg name="log_output_path"
       default="$(find kimera_vio_ros)/output_logs/"
       if="$(arg log_output)"/>
  <arg name="log_gt_data" default="false"/>
  <arg name="gt_topic" default=""/>
  <!-- Set use_sim_time to true if you use rosbag with clock argument -->
  <arg name="should_use_sim_time" default="false"/>

  <!-- Only used when parsing a rosbag -->
  <arg name="rosbag_path" default="default.bag"
       unless="$(arg online)"/>"/>

  <arg name="visualize" default="true"/>

  <!-- Frame IDs. These DO NOT match frame id's on the video streams, as the
	RealSense and Kimera publish conflicting Tf's -->

  <arg name="world_frame_id"     value="world"/>
  <arg name="base_link_frame_id" value="base_link1"/>
  <arg name="map_frame_id"       value="map1"/>
  <arg name="left_cam_frame_id"  value="left_cam"/>
  <arg name="right_cam_frame_id" value="right_cam"/>

  <arg name="run_stereo_dense"     default="false"/>

  <!-- Subscribed Topics -->
  <arg name="left_cam_topic"  value="/zed2i/zed_node/left/image_rect_gray"/>
  <arg name="right_cam_topic" value="/zed2i/zed_node/right/image_rect_gray"/>
  <arg name="imu_topic"       value="/zed2i/zed_node/imu/data"/>

  <!-- Launch actual pipeline -->
  <include file="$(find kimera_vio_ros)/launch/kimera_vio_ros.launch"
           pass_all_args="true"/>

 <!-- Launch static TF node from base_link to velo_link -->
 <node pkg="tf" type="static_transform_publisher" name="velo_link_broadcaster"
   args="0 0 0 0 0 0 1 world map 100"/>

</launch>

The mesh coming in RVIZ looks like this:

Can you please point me in the right direction? I`ve also had issues with ORB-SLAM3 and one opinion there is that ZED camera is a rolling shutter camera, while the algorithm requires a global shutter. Could this be an issue? Or is it just poor calibration of the camera? Many thanks in advance, any hint would be highly appreciated!

I'm unfortunately not a maintainer for Kimera VIO so I probably won't be able to help too much, but will say that the mesher output that you're seeing isn't really related to the quality of the mesh that Kimera Semantics produces, which is what Hydra actually uses for reconstruction (i.e. getting the mesher in Kimera VIO to work is irrelevant for Hydra). As long as you get a reasonable pointcloud out of the zed camera and a reasonable pose estimate from Kimera VIO, you should be set for running with Hydra (after setting up a new config and launch files for it).
w.r.t. rollling shutter / calibration: Rolling shutter effects can affect pose estimation quality, but are most severe under high velocity or fast rotations and I doubt it would be causing catastrophic issues. There could be issues with the intrinsics calibration for the zed if you did it yourself, as the left/right images might already be undistorted to do passive stereo (and calibrating an already undistorted image can lead to problems). However, having never directly worked with the camera myself, I'm not sure what the factory calibration looks like / whether the left/right images are already undistorted/rectified.

Thanks! Do you know how can I configure Kimera Semantics to take the pose and point cloud directly from ZED ros wrapper and avoid using Kimera VIO at all? An example of the launch file would be super useful!

I put together an untested sample launch file that tries to configure both the hydra_topology and hydra_dsg_builder base launch files with as few arguments as possible here. If I get time this week I'll try to replace the arguments with the corresponding uhumans2 information as a sanity check that I didn't miss anything. You may notice that I pointed Hydra to use the uhumans2 configuration parameters in the launch file: these are also a good set of defaults for a new dataset.
You will still need to set up a 2D semantic segmentation network external to the launch file, which is discussed in the first two sections in my response earlier in the issue here .  It might also be worth looking also at the pointers in that response to how the files for hydra_topology and hydra_dsg_builder get specialized for uHumans2 (which is maybe now a little bit more useful since I fixed the link to the uhumans2 hydra_topology launch file :grimacing:).

Hi thanks a lot for your reply! It helped a lot and I managed to launch semantic mesh building and it works pretty well. Still have to figure out loop closure. So I have a couple of questions:

Will the pipeline for enabling lcd still work as you described here? In case I`m using ZED SDK which runs its own loop closure algorithm and I just specify the frame published by it in hydra ros launch file.

I've noticed that there is no mask for humans in my mesh - humans are meshed. They are correctly colored as humans and their label is added to dynamic_semantic_labels. Should I enbale use_dynamic_mask also?

Below I attach my launch file and csv file with colors remapped (is this the way it`s supposed to be?)
<launch>
    <arg name="sim_time_required" default="false"/>
    <param name="use_sim_time" value="$(arg sim_time_required)"/>

    <!-- pointcloud for hydra: no need to change -->
    <arg name="pointcloud_topic" value="/semantic_pointcloud"/>

    <!-- Replace these with your actual sensor information -->
    <arg name="sensor_frame" default="zed2i_left_camera_optical_frame"/>
    <arg name="depth_topic" default="/zed2i/zed_node/depth/depth_registered"/>
    <!-- These should point to a 2D semantic segmentation image -->
    <arg name="semantic_info_topic" default="semantics/camera_info"/>
    <arg name="semantic_topic" default="semantics/rectified"/>

    <!-- semantic configuration -->
    <arg name="semantic_map_path" default="$(find kimera_semantics_ros)/cfg/ade150_colour_mapping.csv"/>
    <arg name="typology_dir" default="$(find hydra_dsg_builder)/config/d455"/>
    <arg name="typology_config" default="d455_typology.yaml"/>

    <!-- see uhumans2 launch file for how these are used -->
    <arg name="dsg_output_dir" default="$(find hydra_dsg_builder)/output/uhumans2"/>
    <arg name="dsg_output_prefix" default="office"/>

    <!-- good starter rviz file, though topics could be edited for images -->
    <arg name="rviz_dir" default="$(find hydra_dsg_builder)/rviz"/>
    <arg name="rviz_file" default="uhumans2.rviz"/>

    <!-- turns rgb (or 2D semantics) + depth into pointcloud -->
    <include file="$(find hydra_utils)/launch/includes/rgbd_to_pointcloud.xml">
        <arg name="rgb_info_topic" value="$(arg semantic_info_topic)"/>
        <arg name="rgb_topic" value="$(arg semantic_topic)"/>
        <arg name="depth_topic" value="$(arg depth_topic)"/>
        <arg name="pointcloud_topic" value="$(arg pointcloud_topic)"/>
    </include>

    <!-- performs reconstruction and extracts places -->
    <include file="$(find hydra_topology)/launch/hydra_topology.launch" pass_all_args="true">
        <arg name="semantic_color_path" value="$(arg semantic_map_path)"/>
        <arg name="config" value="d455_topology_config.yaml"/>
        <arg name="semantic_config" default="ade150_semantic_config.yaml"/>
        <arg name="config_dir" value="$(find hydra_topology)/config"/>
        <arg name="debug" value="false"/>
    </include>

    <!-- constructs rest of scene graph from reconstruction and places -->
    <include file="$(find hydra_dsg_builder)/launch/dsg_builder.launch" pass_all_args="true">
        <arg name="robot_id" value="0"/>
        <arg name="use_gt_frame" value="true"/>
        <arg name="use_oriented_bounding_boxes" value="true"/>
        <arg name="config_dir" value="$(find hydra_dsg_builder)/config/d455"/>
    </include>

</launch>

Will the pipeline for enabling lcd still work as you described here? In case I`m using ZED SDK which runs its own loop closure algorithm and I just specify the frame published by it in hydra ros launch file.

No, sorry (when I wrote response I forgot that Hydra depends on some unreleased kimera-vio code to actually compute the relative pose of the two keyframes for the vision-based loop closures).  However, if you have information from the ZED as to when a loop-closure was detected (and what the relative pose is), then you could publish that as a pose_graph_tools::PoseGraph message (along with the odometry coming from the zed) and avoid using the pose_graph_publisher_node.  Hydra should respect the external loop-closures and optimize the resulting pose graph / scene graph / mesh together.

I've noticed that there is no mask for humans in my mesh - humans are meshed. They are correctly colored as humans and their label is added to dynamic_semantic_labels. Should I enbale use_dynamic_mask also?

No, you shouldn't have to do this (that's only an argument for the kimera_semantics_rosbag.launch file and wouldn't have any effect on whether or not the dynamic semantic labels are used).  I've confirmed that the dynamic masking should be working for uhumans2 (see the end of #2), so I'd suspect something isn't agreeing between the semantic colors and the actual labels. You can set this param to true in your frontend config to have Hydra publish the pointclouds and you'll also want to set this to all possible semantic labels (i.e. 0-20). You can then look to see what integer label each "semantic color" is being mapped to by subscribing to /incremental_dsg_builder_node/object_mesh_vertices/X where X is the label you want to look at.

Many thanks for your reply, the problem with humans got solved! I`m still unsure about what exactly you meant by:

However, if you have information from the ZED as to when a loop-closure was detected (and what the relative pose is), then you could publish that as a pose_graph_tools::PoseGraph message (along with the odometry coming from the zed) and avoid using the pose_graph_publisher_node.

So would I have to rewrite the pose_graph_publisher_node to construct the pose_graph based on the odometry coming from ZED? Id greatly appreciate it if you could expand on this further. Also, would it be easier to do LCD in Kimera-PGMO instead of ZED SDK? In this case, Id only have odometry and depth images coming out of ZED. How can I enable LCD in this case if I provide the BoW? Or is it something that kimera-vio (unreleased)  is supposed to do?

I'm glad you were able to resolve the issue with dynamic labels! If you get a chance, adding a brief statement about what was wrong may help other people who are having similar issues (editing your comment above is probably the best way to do this)

Also, would it be easier to do LCD in Kimera-PGMO instead of ZED SDK?

No, Kimera-PGMO doesn't generate loop closure proposals. Hydra and Kimera-PGMO share the same machinery for processing visual loop closures (DsgBackend in Hydra inherits from KimeraPgmoInterface) from an external VIO / SLAM pipeline (like Kimera-VIO).

So would I have to rewrite the pose_graph_publisher_node to construct the pose_graph based on the odometry coming from ZED? I'd greatly appreciate it if you could expand on this further.

My apologies, I made a bad assumption on the part of the zed (that the zed gave you access to information about loop closures that it detected), which definitely made my previous instructions confusing.
More documentation as to how Hydra handles loop closures: The backend (and the frontend) subscribe to /incremental_dsg_builder_node/pose_graph_incremental, which is either published by pose_graph_publisher_node or kimera vio (though the changes for this are not released). Each message on this topic contains two node pose estimates (the most recent and second most recent keyframe pose) and at least one edge (an odometry edge with the relative pose between the most recent and second mot recent keyframe pose). pose_graph_publisher_node is designed to simulate the incremental messages that you would get from kimera by periodically grabbing the current pose of a tf frame, computing the relative pose between that and the last pose to be cached, and then publishing a pose graph message (with the two nodes and an odometric edge from the relative pose). This "odometric" pose graph (i.e. where all the edges are odometric between consecutive keyframes) is the minimum necessary information to get any loop closure detection working with Hydra as the scene graph-based loop closure detection constructs descriptors for the objects and places around each keyframe if enabled (more on that later). The "odometric" pose graph also is also added to the backend factor graph. If the pose message contains additional edges with type LOOPCLOSE instead of ODOM between two non-consecutive keyframes, then Hydra adds a factor to the factor graph representing the loop closure (and then optimizes and corrects the scene graph).  However, pose_graph_publisher_node will never publish loop closures (it doesn't have any information to detect them / compute the relative pose between the keyframes).  Any loop closure received this way is an "external" loop closure.
Hydra can also detect "internal" loop closures. An odometric pose graph is enough for the scene graph-based loop closures, but the object-based registration (to solve for the relative pose between two keyframes) can be poorly conditioned if you don't have enough objects in the scene or the semantic segmentation is noisy.  Hydra's loop closure detector will proceed down the descriptor hierarchy (places, then objects, then BoW, see the paper for exact details). If BoW descriptors are not present, Hydra will skip doing BoW detection and visual registration. To enable BoW based detection, you'd have to publish the BoW descriptor for each keyframe via the pose_graph_tools::BowQuery message to the topic /kimera_vio_ros/bow_query (there's a hardcoded remap in the base hydra launch file).  To enable visual registration, you would have to write a service server that handles the service request inside this function. This service takes the id of two previous keyframes and returns the relative pose between them (if it was possible to compute). The code that handles this in kimera-vio is also not released yet.
Note that Hydra will always respect external loop closures, even if internal loop closure detection is enabled. We have switches in kimera to disable detecting loop closures so we don't get duplicate visual loop closures when running with internal loop closures.
All that said, there are four ways to proceed here:

Enable scene graph loop closures: This will require no development on your part, but is the least tested of the options (we haven't had much of a reason to run the scene graph loop closure pipeline without BoW descriptors). As long as you have the pose_graph_publisher_node operating (which use_gt_frame:=true will enable), then all you have to do is set enable_dsg_lcd:=true and internal loop closure detection and registration will be enabled. You may also want to set min_glog_level:=0 and verbosity:=2 to see the current status for the internal loop closures.

Internal/External vision-based loop closures: This requires a fair amount of development. You'd have to augment the pose graph publisher node to subscribe to rgb images (and optionally depth to help with registration), selecting some subset of them as keyframes (generally we enforce that keyframes occur 0.2 seconds apart). For each keyframe, you'd compute the BoW descriptor and get the current pose at the keyframe's timestamp (instead of periodically looking up the current pose like the pose graph publisher node does now).  If you want to provide external loop closures, you'd look through all the previous keyframes to see if any of the BoW vectors match. If they do, you'd solve for the relative pose using feature matching / PnP (or another registration method) and publish it as an loop closure edge in the current pose graph message. If you're trying to provide internal loop closures, you'd publish the BoW vectors, and then have a service server that tries to compute the relative pose between any two previous keyframes via feature matching / PnP (or another registration method).

Use another VIO/SLAM pose source:  If there's another open-source VIO solution that works with the zed, it's probably pretty feasible to publish a pose graph message with the necessary information for external loop closures. Obviously, I can see why you'd prefer using the pose estimate from the zed directly (and I'm not aware if anyone has a out-of-the-box solution for the zed, but I haven't looked for one myself).

Wait for us to release our changes to kimera-vio: Still not sure what the timeline is on the next release for kimera, and this would also require a working kimera-vio for the zed.

Hi @araujokth , sorry for the delay in responding. It's looking like we'll have the version of Kimera-VIO that we used pushed as a pre-release in the next couple weeks (we're trying to have it out before the IROS submission deadline).

Thanks so much for the reply @nathanhhughes ! Very looking forward for that update and good luck with the IROS submission!

@araujokth (and others) Thank you for your patience! We've added our changes to a pre-release version of Kimera and updated the documentation on the Hydra side here

Fantastic, thanks for the update! I will give it a try!

Regarding setting up a 2D semantic segmentation for use with Hydra (and Kimera-Semantics) that you wrote about here, have you released any code related to this? Or do you have any other pointers for setting it up? Thank you!

Hi, @nathanhhughes @MikeDmytrenko ,I also tried running Hydra on the zed2i camera. I created my own dataset and did not use semantic images (only RGB images) for testing. My launch file is the same as @MikeDmytrenko, but I reported an error:
Input pointcloud queue getting too long! Dropping some pointclouds Either unable to look up transformer timestamps or the processing is taking too long.
Do you know where the problem is? Is it a problem with the dataset or do I need to change some parameters? I am using version 1.0 of Hydra

Hi, @nathanhhughes @MikeDmytrenko ,I also tried running Hydra on the zed2i camera. I created my own dataset and did not use semantic images (only RGB images) for testing. My launch file is the same as @MikeDmytrenko, but I reported an error:
Input pointcloud queue getting too long! Dropping some pointclouds Either unable to look up transformer timestamps or the processing is taking too long.
Do you know where the problem is? Is it a problem with the dataset or do I need to change some parameters? I am using version 1.0 of Hydra


Fix mismatch return type error span for chained expression UDFs

In chained expressions, the error for the bad return type would only highlight the semicolon instead of the last child node of the variadic op node.

✅ No public API change.

✅ No public API change.


Setup custom domain with web servers that redirect to Bintray

The plan for epic 2 (providing an APT/YUM repo) states that we're going to use Bintray for hosting.
In order to not be dependent on Bintray (i.e. so that users' URLs don't break if we ever move away from Bintray), we should use the domains {apt,yum}.fullstaqruby.org. These domains should be backed by a bunch of redundant, stateless web servers, whose only jobs are to HTTP redirect to Bintray.

Work is here: https://github.com/fullstaq-labs/fullstaq-ruby-infra

This is done. Verified to work.


logind integration

With the latest dbus on Arch Linux, the xrdp-sesman session cannot access the user bus. Adding the following lines to /etc/pam.d/xrdp-sesman fixed it:
auth      include   system-remote-login
account   include   system-remote-login
password  include   system-remote-login
session   include   system-remote-login

Hmm, duplicate of #290, I guess.


@deck.gl/jupyter-widget: Change version variable for JupyterLab installation

The version variable needs to be defined in a way that it can be used by JupyterLab (which installs from the source code).

For #3640

Background

Change List

Define version from the distribution bundle

Coverage decreased (-0.04%) to 80.791% when pulling 8b4fad41cd53a0edac0ffac7b7fd3d5b30f1a93f on ajd/publish-new into 8eebb82e213a910155290e826f2839c465e0ea8d on master.


Added both cs and js example in Textlabel API doc

thanks for your contribution :)
Maybe you could improve it by using tabs for the js/c# part?

thanks for your contribution :)
Maybe you could improve it by using tabs for the js/c# part?

sure!

done


KRM updater is not working

Our latest release PR https://github.com/GoogleCloudPlatform/blueprints/pull/130/files does not seem to be running updaters for bumping versions. Last PR released a month ago, the updaters worked https://github.com/GoogleCloudPlatform/blueprints/pull/119
/cc @chingor13

fixed by https://github.com/googleapis/release-please/pull/1269


template + innerHTML VS DOMParser for fragment

Initial checklist

[X] I read the support docs
[X] I read the contributing guide
[X] I agree to follow the code of conduct
[X] I searched issues and couldn’t find anything (or linked relevant results below)

Problem
Which solution should be preferred?
Solution
https://github.com/rehypejs/rehype-dom/pull/19
Alternatives
N/A

Hey, long time no see! :wave:
I think the method used here in hast-util-from-html-isomorphic is relatively elegant. As far as I can tell it works fine. I would love to hear about cases where the other method is preferred though.

I believe the linked issue explains more. Using a template in the current document sends HTTP requests: https://github.com/rehypejs/rehype-dom/issues/18.

I am seeing a network request when using the code from https://stackoverflow.com/questions/12747350/using-documentfragment-to-parse-html-without-sending-http-requests (linked from https://github.com/rehypejs/rehype-dom/issues/18), but not when using a <template> tag as is used in the current approach.

Hmm, funky. Might be the template tag?

Yes, I think so.
This HTML document triggers a request:
<!DOCTYPE html>

    <meta charset="UTF-8" />

This one doesn’t
<!DOCTYPE html>

    <meta charset="UTF-8" />

    <template>

    </template>

Sure, I missed this issue for a long time.
I'm not for sure to ensure will template + innerHTML be XSSed? But I think DOMParser will never be?


Bug in the Quantum State Learning notebook

Issue description
When trying to run this gallery notebook, at first execution of the line:
_, cost_val, fid_val, ket_val = session.run([min_cost, cost, fidelity, ket])

the kernel dies with the error:
F ./tensorflow/core/kernels/transpose_functor.h:169] Check failed: in.dims() >= 2 (1 vs. 2)

System information

Operating system:

Ubuntu Linux 16.04

Anaconda environment

python=3.6.5=1
numpy=1.14.3=py36h28100ab_2
pip=9.0.3=py36_0
StrawberryFields==0.9.0
tensorboard==1.6.0
tensorflow==1.6.0

Thanks @artix41. We've had this reported through other channels the last week or so, and it looks like a bug in TensorFlow v1.6; specifically, this bug is related to taking the tf.conj() function. While we're still tracking it down, a solution is to temporarily downgrade to TensorFlow v1.3

Ok, thanks for you quick reply, I will use TF 1.3 in the meantime.


Remove Display Version from Azul.Zulu.11.JDK

Newer versions have the proper display version. Since the ISV has had the issue fixed for some time, it seems a better course of action to remove the DisplayVersion from the relatively few manifests that have it rather than updating the manifests without it.

microsoft/winget-cli#4459

Microsoft Reviewers: Open in CodeFlow

Intentionally didn't bump version

URL: https://cdn.azul.com/zulu/bin/zulu11.2.3-jdk11.0.1-win_x64.msi
OK
(Automated message - build 882)


docs: Improve readme with installation link and fix extension ID

While working on #2, I noticed that the extension ID in the readme is wrong and also figured you would appreciate a simpler way of installation.
I also did some formatting changes - feel free to improve or remove them, I added you as a collaborator on ffflorian/sort-everything 🙂

The extension ID wasn't wrong. I was actually just mentioning the Command ID which is "bennycode.sort-everything.sortFile":

I will merge your updates though to make use of the nice kbd styles.

The extension ID wasn't wrong. I was actually just mentioning the Command ID which is "bennycode.sort-everything.sortFile":

Yes, but searching for the command ID in the extension search doesn't work:

I fixed it in https://github.com/bennycode/sort-everything/pull/6 🙂


RTL Problem

Incorrect features location in vector layers in RTL pages.
To Reproduce
1- go to https://openlayers.org/en/latest/examples/cluster.html?q=vector
2- Set dir="rtl" for HTML tag.
3- Zoom in and out to see the problem.

Thanks for reporting the issue.
I can not see a problem there, though. It does not behave differently than without dir=rtl for me. What browser are you using?

See https://codesandbox.io/s/cluster-11u5p

I have the same problem. My  tag has dir="rtl" property. And when I load the map I see no tiles (They get loaded. I see them in Network panel of my browser). And when I remove the dir property or change it to ltr, they will show up. This problem occurred after I migrated from version 5 to 6.

Hello @sky93, thanks for reporting. Can you provide a minimal example where this occurs? Then we can investigate the problem further. And can you tell which browser you are using?

I fugured out that this is because dir='rtl' and also text-align of the map element is left
This is my html:
<!DOCTYPE html>

    <title>XYZ</title>

This is my index.js:
import 'ol/ol.css';
import Map from 'ol/Map';
import View from 'ol/View';
import TileLayer from 'ol/layer/Tile';
import XYZ from 'ol/source/XYZ';

new Map({
  target: 'map',
  layers: [
    new TileLayer({
      source: new XYZ({
        url: 'https://{a-c}.tile.openstreetmap.org/{z}/{x}/{y}.png'
      })
    })
  ],
  view: new View({
    center: [-472202, 7530279],
    zoom: 12
  })
});

And this is package.json:
{
  "name": "cluster",
  "dependencies": {
    "ol": "6.0.1"
  },
  "devDependencies": {
    "parcel": "1.11.0"
  },
  "scripts": {
    "start": "parcel index.html",
    "build": "parcel build --experimental-scope-hoisting --public-url . index.html"
  }
}

Sandbox: https://codesandbox.io/s/bug-with-rtl-064bi

With dir='rtl'and text-align: left, the layer canvas is shifted to the left and is not visible.
Same if text-align: right; is set to the map (but without the dir): the layer canvas is shifted to the right.

With dir='rtl'and text-align: left, the layer canvas is shifted to the left and is not visible.
Same if text-align: right; is set to the map (but without the dir): the layer canvas is shifted to the right.

Yeah and this doesn't happen in version 5.3.3.

I'm using Google Chrome Version 77.0.3865.90 (Official Build) (64-bit)
The problem I believe still occurs in 6.0.1
Goto https://openlayers.org/en/latest/examples/cluster.html
Write the following in the console
document.body.style.direction = "rtl"