<Space>fc (FzCommands) display some commands with completion incorrectly

This can be triggerd when a command is defined with -complete=custom[list],XXX (XXX is a long function name).
MRE:
vim9script
set pp=~/vimfiles  # modify if necessary
syntax on  # without this, it is not initialized correctly
packadd vim-fuzzy-origin  # modify if necessary

command! -nargs=* -complete=custom,LongFuncName MRE echo
def LongFuncName(..._: list<any>): string
    return ''
enddef

expect:
search "MRE", something like this shows:
MRE                 echo        mre.vim:6

actually:
MREFuncName         echo        mre.vim:6

why:
:verbose command MRE gives the following result (command-complete info may exceeds 43 columns):
    Name              Args Address Complete    Definition
    MRE               *            custom,LongFuncName echo
        Last set from /tmp/mre.vim line 6

So maybe the regex here https://github.com/lacygoill/vim-fuzzy/blob/51a3b3109d9822493addacfcb885f1875148fda1/autoload/fuzzy.vim#L458
should be modified to ^\S*\zs.*\%43c\S* (with "\S*" after it)?

Sorry for the late response; I was not notified of the issue because I didn't watch the repo.
I included the suggested fix in this commit.

Thanks!


How to grab images from https?

Hello. Sorry for posting an issue here, as I consider the following not an issue but rather a general question:
How to set up the plugin so that instagram screenshots are loaded over https instead over the http protocol? Any chances I can get it working this way. I'm running a server with SSL enabled.
Here's a test page: https://www.pizzariameurancho.com.br/pizzagram/
Thanks in advance.

hmm.. by default, instagram loads all images via https, I suggest you run a test page that isn't on SSL and then see whether instagram content still loads via http, if it doesn't then I'm guessing its an issue on your setup.
Check out this page regarding mix content, you might also want to check whether bootstrap.js is loaded.


[BUG] Latest Main branch commit breaks AsyncStreams (CurrentValue, Passthrough etc)

Describe the bug
I noticed a code behavior change since adding the actor code.
For example, I use Passthrough as a way to pass events to an onReceive method. That stopped as it supposed the moment I updated to the updated main. The best I can describe the behavior is that fires once and then stops working after that.
To Reproduce
Steps to reproduce the behavior:
var events = AsyncStreams.Passthrough<Event>()

// I used
events.nonBlockingSend(.myEvent)

then

// I have a modifier that wraps this in SwiftUI (onReceive)
task {
 do {
         for try await event in viewModel.events {
               switch event {
                   ...
               }
         }
     } catch {}
}

Expected behavior
AsyncStreams works like the commit e4aeb0f and before.
Environment:

Main

** Additional Info **
I'm not sure how to really explain what is going on here. This is the best I can do.

Hi @Prince2k3
I'm currently rollbacking from using Actors for the Passthrough/CurrentValue/Replay streams due to potential losing some values while not being yet registered as a client of these streams.
Anyway, it could be nice if you could provide une repo that I could take a look at and make sure the next version will restore the expected behaviour.
thanks.

@Prince2k3
could you check with that branch please ? -> https://github.com/AsyncCommunity/AsyncExtensions/tree/feature/multicast
for the record nonBlockingSend has been remove. you can use send directly.

There was an issue using an Actor to handle the registrations of new continuations in the streams (Passthrough, CurrentValue, Replay). The registration had to be Async since performed on an Actor. Unfortunately the creation of an AsyncStream offers a callback that does not supports Async. So the registrations had to be performed in a Task. In some occasions, some values could be pushed to the streams while the task was not yet executed. It resulted on loosing these values.

I reviewed the PR on the branch and used that branch to run through the usage in my current code. Seems to be working. I'll definitely be running more tests today.
@twittemb

@Prince2k3 hey
did you had the chance to perform further tests ? should we close the issue ?
thanks.

Thanks


Spurious "unable to parse coalesced Google QUIC" errors in trace_quic_rfc_quant_basic

This seems wrong, as no Google QUIC is used in the PCAP, it is all IETF/RFC version QUIC.
Output says:
packet not long enough for QUIC hdr:           0
packet not long enough for QUIC token:         0
packet not long enough for QUIC length:        0
unable to parse coalesced Google QUIC:         2
unrecognised QUIC version:                     0
unsupported QUIC message type:                 0
unrecognised QUIC message type:                0

Solved in the latest commit. There was never any real issue, but the parsing of packets needs to stop at the first short form frame inside one packet


Terraform support

What does this implement/fix? Explain your changes.
This adds support for terraform fmt.
Does this close any currently open issues?
Yes - https://github.com/Glavin001/atom-beautify/issues/645
Any other comments?
Hope all fits.
npm run postinstall

Did not work.
As I am no fulltime dev it would be great if you can guide me what else I have to deliver.
Checklist
Check all those that are applicable and complete.

[x] Merged with latest master branch
[x] Regenerate documentation with npm run docs
[x] Add change details to CHANGELOG.md under "Next" section
[ ] Added examples for testing to examples/ directory
[ ] Travis CI passes (Mac support)
[ ] AppVeyor passes (Windows support)

Travis failed because:
/usr/local/Homebrew/Library/Homebrew/brew.rb:12:in `<main>': Homebrew must be run under Ruby 2.3! (RuntimeError)

Have I to fix something or is this a pipeline thing?

npm run postinstall
Did not work.

I don't think there is a postinstall command: https://github.com/Glavin001/atom-beautify/blob/master/package.json#L412-L419
I think you meant npm run docs. What was the error?

For the question regarding postinstall I used this: https://github.com/Glavin001/atom-beautify/blob/master/docs/add-languages-and-beautifiers.md

I am not sure what I have to change to fix the issues. Can you support here ?

Hi @derBroBro you need to have Terraform installed on both the AppVeyor (WIndows) and Travis CI (Linux and MacOS) build files for the tests to pass.  .travis.yml for Travis and appveyor.yml for AppVeyor.
For Travis, you can use uncrustify as an example, as it looks like there is a Homebrew formula for Terraform:  http://formulae.brew.sh/formula/terraform.  Basically if the OS is OSX, brew install terraform otherwise docker pull <docker image here.

Thank you, now appveyor looks ok, travis throws /usr/local/Homebrew/Library/Homebrew/brew.rb:12:in `<main>': Homebrew must be run under Ruby 2.3! (RuntimeError)

There's an issue with Homebrew on Travis, uncomment line 95: https://github.com/Glavin001/atom-beautify/blob/master/.travis.yml#L95 and push it again.

Worked, I think with this the I changed everything.
The changes requested should be also fulfilled (not sure why not detected)

@Glavin001 anything I can to finish the PR?

Hello, any updates here?

Looks good to me, I requested another review from Glavin.

Reviewing now. I apologize, I have been unavailable to review Pull Requests lately. Let's see about getting this merged 😄 .

Looks like merge conflicts now, too. Once resolved, should be good to go!

Awesome! Thank you for contributing!

Published to v0.30.6


Change List.IterAll to read-ahead 8MB chunks over 6 threads

This copies the approach that Blob.Copy uses, with some extra logic to
adjust the number of values to read as more values are read.

This isn't to submit yet, because it hardly makes a difference in csv-export. @rafael-atticlabs could you check it's sensible?
Here is what the csv exported used to do (400MB of csv data):
> go install ./samples/go/csv/... && (time bin/csv-export db::csv) >/dev/null

real	0m14.539s
user	0m22.550s
sys	0m4.139s

and with my changes:
> go install ./samples/go/csv/... && (time bin/csv-export db::csv) >/dev/null
read 1000 chunks (345 kB)
adjusted chunk size from 1000 to 24301 (scaled by 24.301)
read 1000 chunks (355 kB)
adjusted chunk size from 1000 to 23600 (scaled by 23.600)
read 1000 chunks (371 kB)
adjusted chunk size from 1000 to 22620 (scaled by 22.621)
read 1000 chunks (351 kB)
adjusted chunk size from 1000 to 23899 (scaled by 23.899)
read 1000 chunks (364 kB)
adjusted chunk size from 1000 to 23054 (scaled by 23.054)
read 1000 chunks (318 kB)
adjusted chunk size from 1000 to 26354 (scaled by 26.354)
read 1000 chunks (338 kB)
adjusted chunk size from 1000 to 24821 (scaled by 24.821)
read 1000 chunks (346 kB)
adjusted chunk size from 1000 to 24280 (scaled by 24.280)
read 26354 chunks (8.4 MB)
read 24280 chunks (7.7 MB)
adjusted chunk size from 24280 to 26305 (scaled by 1.083)
read 24280 chunks (7.7 MB)
adjusted chunk size from 24280 to 26392 (scaled by 1.087)
read 24280 chunks (7.7 MB)
adjusted chunk size from 24280 to 26331 (scaled by 1.084)
read 24280 chunks (7.7 MB)
adjusted chunk size from 24280 to 26286 (scaled by 1.083)
read 24280 chunks (7.8 MB)
adjusted chunk size from 24280 to 26246 (scaled by 1.081)
read 24280 chunks (7.9 MB)
adjusted chunk size from 24280 to 25825 (scaled by 1.064)
read 24280 chunks (7.8 MB)
adjusted chunk size from 24280 to 26189 (scaled by 1.079)
read 25825 chunks (8.3 MB)
adjusted chunk size from 25825 to 26240 (scaled by 1.016)
read 26189 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.3 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.3 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.3 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.3 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.4 MB)
read 26240 chunks (8.3 MB)
read 23945 chunks (7.6 MB)

real	0m14.637s
user	0m22.263s
sys	0m4.393s

final observations before I did into this again:

the codec benchmark is also identical
maybe this needs to run on S3 to see the change?
marshal test fails. I think it's something to do with IterAll and (un)marshalling using the same internal state (like values()?) and getting confused.

latest patch is simpler/better but marshal test still fails, and csv export is unaffected (in fact it might be slower).

There we go.

OK, @rafael-atticlabs (and @arv) PTAL.

The marshal tests are now passing - I guess the values changes that @arv suggested fixed that (though honestly it does still worry me a little bit why it was failing in the first place).
I tried with a much larger dataset (~2GB), with a hacked-up csv-export to not actually construct the CSV, and the old code took 35s and the new code 30s, so that's quite encouraging.


[HACKATHON] init vitessce heatmap

Background
Link to issue
https://biomage.atlassian.net/browse/BIOMAGE-1544
Link to staging deployment URL
https://ui-martinfosco-ui524.scp-staging.biomage.net/
Links to any Pull Requests related to this
Anything else the reviewers should know about the changes here
Changes
Code changes
Definition of DONE
Your changes will be ready for merging after each of the steps below have been completed:
Code updates
Have best practices and ongoing refactors being observed in this PR

[x] Migrated any selector / reducer used to the new format
[x] Validated that current unit tests work as expected and are enough

Testing

[ ] Unit tests written
[x] Tested locally with Inframock (with latest production data downloaded with biomage experiment pull)
[ ] Deployed to staging
[ ] Passed integration tests

To set up easy local testing with inframock, follow the instructions here: https://github.com/biomage-ltd/inframock
To deploy to the staging environment, follow the instructions here: https://github.com/biomage-ltd/biomage-utils
Documentation updates
Is all relevant documentation updated to reflect the proposed changes in this PR?

[ ] Relevant Github READMEs updated
[ ] Relevant wiki pages created/updated

Approvers

[ ] Approved by a member of the core engineering team
[ ] (UX changes) Approved by vickymorrison (this is her username, tag her if you need approval)

Just before merging:

[ ] After the PR is approved, the unstage script in here: https://github.com/biomage-ltd/biomage-utils is executed. This script cleans up your deployment to staging

Optional

[ ] Photo of a cute animal attached to this PR

In the default view, I suggest removing "Cells" and "Genes" because they are unnecessary and take up a lot of space.

From the UI, it is not clear that I can zoom on the heatmap. To be fair, this is also true for the embedding. Having thought about it more, I actually think that we should add zoom in/out controls to both the embedding and the heatmap to improve UX. This should be a separate ticket. You can ignore this comment for now.
I thought you said you would remove the settings->expression values menu and options because they are no longer relevant?
Similarly, there is no legend so the settings->legend option in the menu can also be removed.
I added metadata tracks and then changed their order - this broke the heatmap. I could reproduce it easily:

I added metadata tracks and then changed their order - this broke the heatmap. I could reproduce it easily:

This issue is independent from vitessce, it can reproduced in ui-default right now too. I want to fix it but first wanted to finish this ticket.

I thought you said you would remove the settings->expression values menu and options because they are no longer relevant?

Yes, but I was waiting to begin work on this until you and alex had coordinated on whether we also wanted to remove this from plots & tables, but I can just do it here for now.

In the default view, I suggest removing "Cells" and "Genes" because they are unnecessary and take up a lot of space.

They are unnecessary initially, but when zooming in these legends are replaced by the gene/cell names labels.
Also, if the heatmap can be resized, is this really an issue?

I added metadata tracks and then changed their order - this broke the heatmap. I could reproduce it easily:

This issue is independent from vitessce, it can reproduced in ui-default right now too. I want to fix it but first wanted to finish this ticket.

No problem! As I said, just made a ticket for the bug and don't worry about it for now.

I thought you said you would remove the settings->expression values menu and options because they are no longer relevant?

Yes, but I was waiting to begin work on this until you and alex had coordinated on whether we also wanted to remove this from plots & tables, but I can just do it here for now.
There's also a couple of questions more or less related to this I have for alex that I need to wait until monday for (like whether we should use the truncated or the raw expressions to display).

The plots and tables heatmap is a separate question. Don't wait on an answer about the P&T heatmap to complete this one. You and Alex's conclusion is that this menu is not relevant here in Data Exploration so just remove it and don't make any changes in P&T for now.

In the default view, I suggest removing "Cells" and "Genes" because they are unnecessary and take up a lot of space.

The spaces are unnecessary initially, but when zooming in these legends are replaced by the gene/cell names labels, in fact I believe the space allocated to the margin depends on the length of the gene's/cell's names. You can see the difference here:
They are both the same code, the "Cells" and "Genes" texts are just added but don't define the margin sizes
Apart from this, if the heatmap can be resized, is this likely to be an issue?

It's clear from the onboarding callt that many users have smaller screens than my macbook. So I think we need to consider the best use of the available space as high priority, especially in the Data Exploration module. Can the Genes / Cells labels be removed in the default view and then appear when the plot resizes on zoom? As an alternative compromise, I suppose I could live with the Gene label being there on the default view, but the Cells label is completely useless and really does take up a lot of space.

Looking really good! Only 1 comment: the bottom gene in the heatmap is cut off. This happens even though my browser window is the full height of my screen. Considering that most users have smaller screens than me, it would be good to fix this.


⚠ Bofrost.at has degraded performance

In b874336, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 3034 ms

Resolved: Bofrost.at performance has improved in a87bc7d after 6 minutes.


CMake Error: CMake was unable to find a build program corresponding to "Ninja".  CMAKE_MAKE_PROGRAM is not set.

Steps to Reproduce

I used the master channel of flutter
Then I want to build linux binary for upload to release. The releated github action file is below.

name: Linux-CI

on:
  push:
    branches:
      - master
    tags:
      - '*'

jobs:
  build:
    runs-on: ubuntu-latest
    if: contains(github.ref, 'tags')
    steps:
      - uses: actions/checkout@v2
      - name: Branch name
        id: branch_name
        run: |
          echo ::set-output name=SOURCE_NAME::${GITHUB_REF#refs/*/}
          echo ::set-output name=SOURCE_BRANCH::${GITHUB_REF#refs/heads/}
          echo ::set-output name=SOURCE_TAG::${GITHUB_REF#refs/tags/}

      - uses: actions/setup-java@v1
        with:
          java-version: '8.x'
      - uses: subosito/flutter-action@v1
        with:
          channel: 'dev'

      - name: Change flutter channel to master, enable linux support
        run: |
          flutter channel master
          flutter upgrade
          flutter config --enable-linux-desktop
          flutter pub get

      - name: Build linux
        run: |
          flutter build linux

      - name: Debug
        run: |
          sudo npm install tree-node-cli -g
          treee build

      - name: Prepare release files
        run: |
          cp -r build/linux/release python_exercise_builder-linux-${{ steps.branch_name.outputs.SOURCE_NAME }}
          zip python_exercise_builder-linux-${{ steps.branch_name.outputs.SOURCE_NAME }}.zip -r python_exercise_builder-linux-${{ steps.branch_name.outputs.SOURCE_NAME }}

      - name: Upload binaries to release
        uses: svenstaro/upload-release-action@v1-release
        with:
          repo_token: ${{ secrets.GITHUB_TOKEN }}
          file: python_exercise_builder-*
          tag: ${{ github.ref }}
          overwrite: true
          file_glob: true

When the ci run, it failed with the following error.

Expected results:
Linux build successfully.
Actual results:
Linux build failed.

Logs

Run flutter build linux
Downloading linux-x64/linux-x64-flutter-gtk tools...                1.5s
Downloading linux-x64-profile/linux-x64-flutter-gtk tools...         0.9s
Downloading linux-x64-release/linux-x64-flutter-gtk tools...         0.8s
Building Linux application...
CMake Error: CMake was unable to find a build program corresponding to "Ninja".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.

CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
Unable to generate build files
##[error]Process completed with exit code 1.

Building for Linux requires ninja, as indicated by flutter doctor.

Hi @stuartmorgan, I tried to install ninja via add - uses: seanmiddleditch/gha-setup-ninja@master in my github workflow. Now it have ninja, but it failed with another error.
Run flutter build linux
Downloading linux-x64/linux-x64-flutter-gtk tools...                1.5s
Downloading linux-x64-profile/linux-x64-flutter-gtk tools...         0.9s
Downloading linux-x64-release/linux-x64-flutter-gtk tools...         0.5s
Building Linux application...
CMake Error at /usr/local/share/cmake-3.17/Modules/FindPkgConfig.cmake:467 (message):

  A required package was not found

Call Stack (most recent call first):

  /usr/local/share/cmake-3.17/Modules/FindPkgConfig.cmake:647 (_pkg_check_modules_internal)

  flutter/CMakeLists.txt:24 (pkg_check_modules)

Unable to generate build files
##[error]Process completed with exit code 1.

Any ideas about this problem?

This is why flutter doctor exists; you need to run it, and install everything it tells you is necessary.

Finally, I can add a step like the following to make it work for linux package.

      - name: Patch for linux build
        run: |
          flutter doctor
          sudo apt-get update -y
          sudo apt-get install -y ninja-build libgtk-3-dev
          flutter doctor

Yea I ran into same issue when trying to set up flutter for Linux desktop. Thanks to flutter doctor. The result after running the flutter doctor command shows that some of the building programs are missing

clang++ required sudo apt install clang fixed that
ninja is required sudo apt install ninja-build fixed that
GTK 3.0 required sudo apt install libgtk-3-dev fixed that also

You'd get other similar errors and sudo apt install <package> command should fix the thing 👍👍👍👍👍

Always remember apt install xxx -y or it will be stuck in Confirmation.


New command: slot

I want to add some fun commands as well to the Bot. There are a lot of useful commands In Bot.
I would like to add a slot machine command.
Preview Example:

What will change in code?
I will add a new file in cogs/hacksquad > fun.py

This is bot is for a competition, Not for a public bot

Hello @codePerfectPlus 👋
As mentioned by @Artemis6969 this bot is only for a competition and not for such fun commands.
Thank you for your interest in contributing though 🙏


Create ipywidgets UI to manipulate Focus objects

For this round of TAC funding, we need to write a function that returns an ipywidgets UI that helps users manipulate the focus objects associated with grids.
In #81, I wrote a function that returns two sliders that manipulate the shape (as in resolution) of the grid. We can use that as a starting point.
ipywidgets docs are here: https://ipywidgets.readthedocs.io/en/stable/
This is a difficult problem to solve from a user-experience perspective. The focus parameter is that is passed to a the Gridgen constructor is an arbitrarily-sized collection of _FocusPoint objects, which take 4 parameter: (location, axis, factor (strength), extent).
So it'd be nice to have a way to dynamically add a new _FocusPoint to the main focus parameter, but the first step is to build a UI that manipulates a single _FocusPoint.
Need a brush up on grid focus? Check out:

the existing docs
this demo notebook

@kqasem I'm on this one. I've got a working demo in a notebook and I'm cleaning it up for a PR.


[5.9🍒] require consume x for noncopyable pattern bindings since they're consuming

@swift-ci please test


Use the ECS RunTask API to run attached processes

Not ready to merge.
This is a prototype for switching emp run to use the ECS RunTask API, instead of calling out to Docker directly. The primary motivation behind this is to allow containers started with emp run to use AWS Roles for ECS tasks. This is described in https://github.com/remind101/empire/wiki/RFC:-Use-RunTask-API-for-attached-runs.
There's some serious caveats with this implementation at the moment:

Because the ECS API, and the amazon-ecs-agent don't provide a way to pass down the -i (open stdin) and -t (allocate TTY) flags of docker run when running tasks, we have to run a "dummy" command (tail -f /dev/null), then use docker exec to start the command and attach to the container. The major downside of this is that emp ps currently won't reflect what was provided at emp run (it would show tail -f /dev/null).
I haven't yet addressed backwards compatibility. It may be a good idea to provide a flag to use the old method for running attached runs entirely through Docker.

I think the first issue above could be addressed by forking the ecs-agent, and adding a method to provide Tty: true and OpenStdin: true on the CreateContainer call, but I'd hate to have Empire rely on a fork of the ECS agent.
On the plus side, this sets up some infrastructure for adding an emp exec/attach command for starting up a process in an existing container.

To expand a bit on the idea of adding a method to provide the Tty and OpenStdin flags to CreateContainer in the ECS agent, now that Go 1.8 is released and has support for plugins, we could add a feature in the ECS agent for "task plugins", to manipulate the task before starting. I'll open an issue on the amazon-ecs-agent repo to see if there would be any interest in supporting something like that.

Oh that would be awesome if they were down for it.

Ok, @phobologic. If you wanna give this another look, I think this is probably good to go. We've been running this on staging, without any hitches so far.
Primary changes since the last review:

Added docs/caveats about the new method, and requirements for patching the ECS agent.
Fixed some bugs with attaching to tasks that enter RUNNING for a brief period of time.
Stripped out support for placement constraints. This requires a more thoughtful implementation, and isn't necessary for most use cases. I'll follow up on this later.
Added an Attaching to task/<task id>... message, since we no longer see the Docker pull output (yay!).

The first commit can just be ignored, since it's just an update to the AWS SDK.

Noticed some zombie procs lying around. This should probably try a StopTask after detaching.

This has been working out pretty well for us in our staging environment without any major issues/bugs. Gonna go ahead and merge.


Multiple Pikabus should make multiple shades

Right now, because shade is instantiated on the Pikabu container and not on the individual Pikabus, you can't have more than one shade.  Should we allow for multiple shades for each Pikabu on the page? My thought is: yes.

What’s the use case? The only thing I can think of is to add different styling or animation. That could be enabled with a single shade if that’s easier.

@ry5n Currently, the way shade is built, its properties only get passed through on init. So if you want one Pikabu to have a different style of shade than the previous one, you're SOL. If you want it to cover something different or be somewhere else, you're SOL. I think our best bet is to have one per.

@kpeatt Yup, makes sense.

We should really fix this.


How to get the metadata of a video?

Please guide on how to get the metadata of a video. Ideally, it should show on onLoad but it's not there. Also, it would be helpful if you can write some more documentation in the readme.

There is no metadata event listener, there is few props for playback handling.

onLoad
onProgress
onSeek
onEnd
onError

The props and events and playback handling are good but what I am also looking for is to get audio tracks and subtitle tracks and props to set audio & subtitles track. It's there in https://github.com/razorRun/react-native-vlc-media-player
where the audio & subtitle tracks are retrieved with the onLoad event. Can you please add that?


docker-compose taking forever to boot up containers

Hi,
so following your step-by-step instructions in the readme, when I run docker-compose for a second time in the project it can take a minute or more to boot up!
Example of it taking over a minute when i gave up:

Was not the case when i first pulled it and ran the command the first time.

NVM, seems to be my system that's just slow :)


Battle ship

2 nd release

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

400 of 457 (87.53%)  changed or added relevant lines in 16 files are covered.
5 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-8.7%) to 90.031%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

repository/IngSoft2-Model/Game.class.st
181
184
98.37%

repository/IngSoft2-Model/Board.class.st
70
124
56.45%

Files with Coverage Reduction
New Missed Lines
%

repository/IngSoft2-Model/Game.class.st
1
97.15%

repository/IngSoft2-Model/Board.class.st
4
68.13%

Totals

Change from base Build<PHONE_NUMBER>:
-8.7%

Covered Lines:
587

Relevant Lines:
652

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

400 of 457 (87.53%)  changed or added relevant lines in 16 files are covered.
5 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-8.7%) to 90.031%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

repository/IngSoft2-Model/Game.class.st
181
184
98.37%

repository/IngSoft2-Model/Board.class.st
70
124
56.45%

Files with Coverage Reduction
New Missed Lines
%

repository/IngSoft2-Model/Game.class.st
1
97.15%

repository/IngSoft2-Model/Board.class.st
4
68.13%

Totals

Change from base Build<PHONE_NUMBER>:
-8.7%

Covered Lines:
587

Relevant Lines:
652

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

400 of 457 (87.53%)  changed or added relevant lines in 16 files are covered.
5 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-8.7%) to 90.031%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

repository/IngSoft2-Model/Game.class.st
181
184
98.37%

repository/IngSoft2-Model/Board.class.st
70
124
56.45%

Files with Coverage Reduction
New Missed Lines
%

repository/IngSoft2-Model/Game.class.st
1
97.15%

repository/IngSoft2-Model/Board.class.st
4
68.13%

Totals

Change from base Build<PHONE_NUMBER>:
-8.7%

Covered Lines:
587

Relevant Lines:
652

💛  - Coveralls


Merge map pdf and legend pdf into one pdf

Fix map resolution issue

This change is

Reviewed 5 of 5 files at r1, 2 of 2 files at r2.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


Consider increasing accuracy of pull request merging trigger

We currently use pushes to the main branch as the trigger for checking that a pull request was merged:
https://github.com/githublearn/introduction-to-github/blob/60017ba97de7a27dcd93d192d238c00cc280dfb0/.github/workflows/4-merge-your-pull-request.yml#L7-L13
While it's true that merging a PR into the main branch will trigger in just such an event, it is also true that many other activities (like editing in the UI, other pushes, merging other PRs, etc.) could also result in that trigger. 😅
As such, we may want to consider increasing the accuracy of this event trigger if any folks start reporting such issues. 🤷🏻♂
e.g. suggested:
on:
  # Note that supporting `workflow_dispatch` will require some extra logic checks if
  # if we switch to using `pull_request` instead of `push`!
  workflow_dispatch:
  pull_request:
    branches:
      - main
    types:
      - closed

Plus updating the job-level if from:
https://github.com/githublearn/introduction-to-github/blob/60017ba97de7a27dcd93d192d238c00cc280dfb0/.github/workflows/4-merge-your-pull-request.yml#L24-L29
to:
    if: ${{ github.repository_owner != 'githublearn' && github.event.pull_request.merged == true }}

We could also get specific about the name of the PR head branch, if that makes sense:
    if: >-
      ${{
        github.repository_owner != 'githublearn' &&
        github.event.pull_request.merged == true &&
        github.head_ref == 'pr_branch_name'
      }}

:warning: To still support workflow_dispatch, we would need to tweak the if condition further as well, e.g. something like:
    if: >-
      ${{
        github.repository_owner != 'githublearn' &&
        (
          github.event_name == 'workflow_dispatch' ||
          (
            github.event.pull_request.merged == true &&
            github.head_ref == 'pr_branch_name'
          )
        )
      }}

The code you have provided is an if statement that checks if the following conditions are met:
The repository owner is not githublearn.
The event name is either workflow_dispatch or the pull request has been merged and the head ref is pr_branch_name.
If all of these conditions are met, the if statement will execute the code that follows it.
This code could be used to run a workflow only if the repository owner is not githublearn and the event is either a workflow dispatch or a pull request that has been merged and the head ref is pr_branch_name.
For example, you could use this code to run a workflow that deploys a new version of your application to production only if the repository owner is not githublearn and the pull request has been merged and the head ref is pr_branch_name.
Here is a breakdown of the code:
if: >-
${{
github.repository_owner != 'githublearn' &&
(
github.event_name == 'workflow_dispatch' ||
(
github.event.pull_request.merged == true &&
github.head_ref == 'pr_branch_name'
)
)
}}
The first line of the code is the if keyword. This keyword tells GitHub that the following code should only be executed if the conditions inside the curly braces are met.
The next line of code is a nested conditional statement. This statement checks if the following conditions are met:
The repository owner is not githublearn.
The event name is either workflow_dispatch or the pull request has been merged and the head ref is pr_branch_name.
The && operator is used to combine the two conditions. This means that both conditions must be met for the nested conditional statement to be true.
If the nested conditional statement is true, the if statement will execute the code that follows it.

es muy interesante y ayuda mucho el curso

It unlikely we will make this update and have capacity to throughly test it as well, so we will close for now.


Create Friends page. Related #65.

create friends page
create route and route tests
add emoji-sprite.svg
add slider-button.svg

Codecov Report

Merging #103 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master   #103   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           2      2
  Lines          32     34    +2
  Branches        2      2
=====================================
+ Hits           32     34    +2

Impacted Files
Coverage Δ

src/routes.js
100% <100%> (ø)
:white_check_mark:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 77a43aa...96b179b. Read the comment docs.


no dest gas fee shown, Mulitple errors in console

after a while i got it but see these errors

logs
bridge.connext.network-1678714851439.log

We have reduced the number of unnecessary requests. It looks good on my end now. Please test it.


tags

Would it be possible to create tags for release > 0.2.1?
The reason being that we use bower to manage our front-end dependencies and bower in turn uses git tags to enumerate available versions.
PS. Adding a bower.json file might not be a bad idea either...

Totaly +1 - it will make using your library so much more convinient.


QA: html comments

Testing comments incentives per html elements

A test comment

another test comment

another comment

/help

/help

/help

/help

/help

/help


Adding whitelist filter to ActionUtil

Currently ActionUtil only has support to remove bad values based on a blacklist. While effective in most cases, I feel the addition of a whitelist can be beneficial.
This was spurred from a need in my current personal project where I was unable to directly edit the param list used by Sails (I'm mixing in google data in my controller and need the query params).
I am opening this pull request to see what the community thinks of it.
I'd like to add tests but I am unsure where they belong for ActionUtil.

So, it looks like _.omit removes everything if it's an empty array. I should have realized that last night. Will be fixing tonight when I get home from work.

Hi @krishnaglick, just getting caught up -- sorry for the slow reply.  I think the same principle from https://github.com/balderdashy/sails/pull/3102#issuecomment-172287546 applies here.  If this is still a relevant use case for you, would you mind providing some more information?

I coded around the issue by using lodash in my controller. No biggie. I was receiving some params that weren't used my by system, and were actually intended for google's api, and wanted an easy way to define what I would take.

Haven't seen any more chatter about this, so I'm going to close.  It's not a bad proposal by any means, but without seeing a lot more requests for the feature, and seeing as it can be worked around in userland code, it doesn't justify the maintenance burden to put it in core right now.


Doc-strings need a stable ordering

Assert.AreEqual failed. Expected:<help 1

help 2>. Actual:<help 2

help 1>.
   at AnalysisTests.AnalysisSaveTest.OverloadDocString() in c:\RollBuild\Python\Tests\Analysis\AnalysisSaveTest.cs:line 178

Alternatively, the test needs updating to not care about ordering.

This is resolved (overloads are sorted by number of parameters)


Fix interface/fragment support

Currently fragment references on interfaces generates bad code. Instead of a struct with union on and the variants having fields, I get fields for each type as well as an on field. (This is when using #105.)

It should be possible to create a minimal query to reproduce the problem in the integrations test folder. I'll prioritize this above the rest when I have time to work on graphql-client (this weekend probably).

I opened a PR with a test to reproduce the problem: https://github.com/tomhoule/graphql-client/pull/153
Although apparently it's not even compiling, instead of generating bad code. Anyhow it needs to be fixed, I'll do this in priority.

Fixed by #153


Protect against session fixation attacks

Sometimes, a malicious user will tamper the session cookie to inject a controlled session id.
Without this protection, the session handler tries to read a session value with this session id and, if the id is not found, a new session entry is created using the malicious id.
To avoid that the PHP team introduced the use_strict_mode option in the native sessions module.
This commit introduces the use of the use_strict_mode option in the SessionHandler to prevent session fixation by issuing a new session id for the current session when an unknown session
id is used.
For the sake of facility, we rely only on the opened sessions to conclude whether or not the session id is known.
We could add opened session ids history as in https://github.com/1ma/RedisSessionHandler
but I can't see any useful use case for now.
PR review appreciated

I just tested these changes: They do not work. A new session id is generated and used for storage, but PHP still sends the old session ID in headers. So in the end the user gets a fresh session without any data with each request.
I guess the only way to prevent session fixation is to implement SessionUpdateTimestampHandlerInterface.

I think we have to reissue a session cookie with the new value. I thought the session module of PHP was handling that but it looks like it's not the case.
The problem with the SessionUpdateHandlerInterface is that it's a PHP 7 feature and will introduce BC break

PHP is responsible for sending the session cookie. I don't think the application should fiddle with it.
PHP 7 is out for more than 2 years now, support for earlier version will end in less 11 months. I think having a BC break is a good thing. People need a reason to upgrade.
And lets not forget: This is a security issue.

I think the correct way to handle this would be to call SessionStorageInterface::regenerate or SessionInterface::migrate in case the session key does not exist when we load it. Unfortunately there is no way to do this cleanly from within a SessionHandlerInterface implementor in Symfony. Something to be improved on the Symfony side perhaps with a new interface?

Started from 3.4, Symfony team provides new session handlers for this issue :

https://github.com/symfony/symfony/blob/3.4/src/Symfony/Component/HttpFoundation/Session/Storage/Handler/AbstractSessionHandler.php
https://github.com/symfony/symfony/blob/3.4/src/Symfony/Component/HttpFoundation/Session/Storage/Handler/StrictSessionHandler.php

Thanks for the pointer @BboyKeen - I implemented that in https://github.com/snc/SncRedisBundle/pull/404 - closing this as it's not the right approach IMO.

I perfectly agree. I wi use your PR in production as well.


Create xxxx-scaling-service-provider-hex-boosts.md

This Helium Improvement Proposal (HIP) proposes changing the minimum size hex that can be boosted from res12 hex to res10 hex, scaling the price to boost, and limiting the maximum boost to 50X.

What's the status on this HIP @heatedlime ?

What's the status on this HIP @heatedlime ?

Gonna close it


Unable to publish

Conclusion: Refactor location of resources and .resx
Launch python script from resource manager in code

Sophos treats application as a virus

Solution: utilize IronPython and execute python script

Strange solution:

name executable resource as .tmp then rename it to .exe
must have at least two publishes for it to work properly


much more battery hungry than other apps with open connections

ntfy seems to use much more battery than other apps which keep open network connections.
Phone: Fairphone 3+
OS: eOS-1.4 (Android-10)

see screenshot

For example Conversations. It has two XMPP accounts logged in. And I'm opening Conversations in foreground quite often. Nevertheless, Conversations needs less battery than ntfy. Same when comparing to microg.
Please check if you can optimize battery usage!

Although this might be counter intuive:
What about adding a mode to receive a notification via PlayServices (microg) to wake ntfy? Instead of keeping an open connection. This still has better privacy than pure PlayServices, because PlayServices won't know which notifications ntfy will then receive from it's own server.

The battery usage is already pretty optimized and uses <1% per day for pretty much everyone out there. This must be a problem with your phone or configuration. In my limited personal experience, the Fairphone is not the greatest phone in various aspects, so maybe it's that. Feel free to drop in the Discord/Matrix to discuss, but I won't do any more battery work, because it's been solid for over a year.


Doublecheck that the turing warning "link" doesn't affect results or performance

See https://github.com/TuringLang/DynamicPPL.jl/issues/422
The question is whether the
┌ Warning: [DynamicPPL] attempt to link a linked vi
└ @ DynamicPPL C:\Users\jesse\.julia\packages\DynamicPPL\1qg3U\src\varinfo.jl:821

warnings actually matter.  We should:

Make sure we can replicate that issue happing with rbc_joint_1 or something like that.  If not, we can revisit the strategy.
Witth the current main branch do an estimate of rbc_joint_1 and rbc_kalman_1 with the burnin initial conditions we saved, for 1000 draws or something like that
With the latest ssgu branch, do an ]instantiat and then redo the experiments.  If the estimates and the ESS/s are roughly comparable then that is good enough.  Confirm that there is at least one of those warnings popping up.

I'm having the same problem when using NUTS and HMC. It works with IS and Prior. My model is
@model function OneCompFit(prob, data, Sample, num_subj)

    # Define prior indormation

    Cl ~ Normal(log(8), 1)
    V  ~ Normal(log(25), 1)
    Ka ~ Normal(log(1.5), 1)

    Σ  ~ InverseWishart(num_subj, diagm([0.2]))
    Ω  ~ InverseWishart(6, diagm(fill(0.09, 3)))
    η  ~ filldist(MvNormal(Ω), num_subj)

    #ΩCL  ~ truncated(Cauchy(0.01, 0.2), 0.0, 2.0)
    #ΩV   ~ truncated(Cauchy(0.01, 0.2), 0.0, 2.0)
    #ΩKa  ~ truncated(Cauchy(0.01, 0.2), 0.0, 2.0)

    # CLᵢ ~ filldist(LogNormal(log(Cl), Ω[1,1]), num_subj)
    # Vᵢ  ~ filldist(LogNormal(log(V), Ω[2,2]), num_subj)
    # Kaᵢ ~ filldist(LogNormal(log(Ka), Ω[3,3]), num_subj)

    # println(η)

    CLᵢ = exp.(Cl .+ η[1,:])
    Vᵢ  = exp.(V  .+ η[2,:])
    Kaᵢ = exp.(Ka .+ η[3,:])

    function ens_fn(prob, i, repeat)
        par = [CLᵢ[i], Vᵢ[i], Kaᵢ[i]]
        remake(prob, p=par, saveat=Sample, save_idxs=[2])
    end

    ens_prob = EnsembleProblem(prob, prob_func=ens_fn)
    ens_sol = solve(ens_prob,AutoTsit5(Rosenbrock23()),
                    trajectories=num_subj,
                    dense=false,
                   save_everystep=false,
                   abstol=1e-10,
                   reltol=1e-7,
                   maxiters=1e5)
    if !(ens_sol.converged)
        pred = repeat([-Inf], num_subj)
    else
        pred = reduce(vcat, [Array(ens_sol[i]) for i in 1:num_subj])
    end
   for i in eachindex(pred)
        data[i] ~ LogNormal(log(max(pred[i],1e-10)), Σ[1])
    end

    #return Cl, V, Ka, σ, Ω

end

Should it be something to worry about? Thanks!!


Format

#3 GJF added as github Action.
Are you C++ programmer by any chance?:) You might consider changing variable names so they explicitly describe their purpose.

Thanks for your contribution! I will review the code soon! ^^

Hey, thank you for accepting!
Just an advice -I refactored part of the code as an example, but i cant do all of it because i don't know details of your implementation and algorithms. Every number and string/letter value in code should be extracted to a constant. Every repeating part of code even when its one line should be extracted as a separate method.
I hope i helps!:)


Platform dev atm sychronizing with Platform dev

Synchronizing changes in Platform dev (2021-04-13) to Platform dev atm.

@ziu1986 Are you making a pull request to "platform_dev" or "main"?


Make TabContainer use TabBar internally

Kept you waiting, huh?
It's finally here, after a long time and a lot of PRs to make TabBar work correctly, TabContainer can now use it instead of its own implementation! This will considerably improve maintainability and facilitate improvements.
Methods Modified:

get_tab_idx_at_point() exposed for TabBar. Just so TabContainer and everyone else can use it with ease.
get_tab_disabled() -> is_tab_disabled() for TabContainer (all methods below are also for that). The naming makes more sense and matches the one in TabBar.
get_tab_hidden() -> is_tab_hidden(). Same as above.
get_tab_idx_from_control() added. Allows the get the correct tab index, ignoring top-level and internal nodes (very useful for us to use internally, see "Caveats").
clip_tabs member added. Now the clipping can be disabled.
set_tab_title() now returns a tab to its default name (the one of the child) if set to blank.
AlignmentMode constants have been removed from TabContainer, now making use of the ones in TabBar.

Caveats:

This is for you, fellow Godot maintainers: use get_tab_idx_from_control() instead of child->get_index() when manipulating controls in a TabContainer. As said before, this method is a helper to get the correct index of the tab.
all_tabs_in_front is now more literal than before. The old behavior always made the selected tab be in front no matter what, now, due to technical limitations, all tabs must be either in front or back (the default is back).

Sooo many bugs

the most obvious one is Import tab overlapping scene tree at startup

also for some reason, the scene that opens at start is the first tab instead of the last one
can't switch scene tabs
if you move e.g. Import tab next to FileSystem and then back, you get this

this button can't be clicked anymore

after adding Controls to TabContainer, they are all visible until you change tab

(previously any new control would be auto-hidden)

The last bug isn't fully fixed yet. The Control is visible and doesn't get updated immediately after being added in editor:

(hiding it would be enough though)
btw, it might be useful to have get_tab_bar() method in the TabContainer. Some compound controls (e.g. ScrollContainer) have such methods.

btw, it might be useful to have get_tab_bar() method in the TabContainer. Some compound controls (e.g. ScrollContainer) have such methods.

The whole thing depends on TabBar to be in perfect sync with the stuff in TabContainer, changing anything could easily make it crash. Would that function still be desirable?

Well, depends if TabBar has some useful properties that can be customized and aren't available in TabContainer. ScrollContainer will also crash if you free the scroll bars and it's just mentioned in the docs.

Minimum size does not work:

Regardless of use_hidden_tabs_for_min_size.

@KoBeWi Bugs (hopefully) fixed, and docs updated. Thanks for helping me test this, by the way!

The side_margin still has problems:

For center align it only moves to the right and for right align it moves to a certain point and then tabs mysteriously disappear. Works good with left align.
Is this intended that get_tab_idx_at_point() ignores upper bounds? I.e. it returns correct id no matter how high above tab you check. Doesn't happen with other sides.

I'm looking at the code and I wonder, should left alignment be default instead of center? Every single TabContainer in editor changes it to left. Also left is used commonly in other software (browsers, code editors, everything uses it). I think it would be better to change it.
btw this PR fixes #56140

I'm looking at the code and I wonder, should left alignment be default instead of center? Every single TabContainer in editor changes it to left. Also left is used commonly in other software (browsers, code editors, everything uses it). I think it would be better to change it.

Probably, but this should be done in another PR. :wink:

Should it? You are modifying all alignment usage anyways. You could remove these lines instead of changing them.

Minimum size calculation still doesn't work correctly. It seems to not take tab size into account:

Here, I have 64x64 TextureRect. Minimum size should be tab height + TextureRect size, but it's only the latter.

Changes made.

Thanks! Amazing work :)


New pass() API

Similar to 'spy()` but falls back to network not recording anything

API ideas

.spy(matcher, transparent) - transparent is Bool
.pass()


Collection-repeat compatibility

Hi Aliok, Any progress on the support for Collection-repeat? I want to integrate your ion-affix sticky element in a collection repeat with a dynamically generated dividers (via a directive). Here's the article and logic I'm using:
http://gonehybrid.com/how-to-group-items-in-ionics-collection-repeat/

This project is not maintained anymore. We cancelled our free-time hobby project with Ionic because of too much changed recently in our personal lives and we don't have enough time at all.
I would be happy to reference anyone who would fork the project and maintain it.


allow choice of tqdm submodule

Sometimes its convenient to choose the tqdm submodule the constructs the progress bar. For example, to manually select the 'std' backend rather than 'notebook' when working in a jupyter notebook. Before, jax-tqdm only allowed automatic selection using tqdm.auto.tqdm. Here, the user can specify the tqdm_type argument as one of 'auto', 'std', or 'notebook' (default 'auto') which uses the corresponding option, e.g., tqdm_type='std' uses getattr(tqdm, 'std').tqdm.
I only included those three options. If there's a more general way to specify any tqdm submodule that would be better.

LGTM @jeremiecoullon any thoughts?

Sounds great! : )

Bonza, will merge and setup a release later


Implement '+' CSS selector

'x + y' CSS selector, which means "get y next to x" doesn't work in Nokogiri.
For example:
doc = Nokogiri::HTML(open("https://docs.microsoft.com/en-us/windows/win32/api/winuser/"))
puts doc.css('h2#functions + div')

doesn't output anything but should output a <div class="table-scroll-wrapper"> which contains the table of functions from winuser.h Win32 header.

Hi @lem0nify, thanks for asking about this functionality.
Nokogiri has implemented the CSS + selector!

see tests for converting CSS queries into XPath queries
see tests asserting on the results of CSS queries

I haven't looked more deeply at your example of unexpected behavior, but is it possible that libxml2 is parsing that document differently than you expect (and differently from your browser) because it's malformed? And thus the resulting DOM is different enough to make this query return different results?
/me types for a minute ...
Looks like that's a reasonable conclusion, see:
$ nokogiri "https://docs.microsoft.com/en-us/windows/win32/api/winuser/"
Your document is stored in @doc...
2.7.0 :001 > puts @doc.errors
157:86: ERROR: Tag header invalid
174:65: ERROR: Tag nav invalid
227:323: ERROR: Tag nav invalid
254:385: ERROR: htmlParseEntityRef: expecting ';'
254:395: ERROR: htmlParseEntityRef: expecting ';'
333:116: ERROR: Tag nav invalid
336:89: ERROR: Tag section invalid
342:96: ERROR: Tag main invalid
350:105: ERROR: Tag time invalid
356:147: ERROR: Tag nav invalid
4030:165: ERROR: Tag nav invalid
4081:76: ERROR: Tag footer invalid
4104:28: ERROR: Tag svg invalid
4105:71: ERROR: Tag rect invalid
4106:76: ERROR: Tag rect invalid
4107:75: ERROR: Tag rect invalid
4108:75: ERROR: Tag rect invalid
4109:76: ERROR: Tag rect invalid
4110:75: ERROR: Tag rect invalid
4111:83: ERROR: Tag rect invalid
4112:83: ERROR: Tag rect invalid
4128:28: ERROR: Tag svg invalid
4129:71: ERROR: Tag rect invalid
4130:76: ERROR: Tag rect invalid
4131:75: ERROR: Tag rect invalid
4132:75: ERROR: Tag rect invalid
4133:76: ERROR: Tag rect invalid
4134:75: ERROR: Tag rect invalid
4135:83: ERROR: Tag rect invalid
4136:83: ERROR: Tag rect invalid
4152:28: ERROR: Tag svg invalid
4153:71: ERROR: Tag rect invalid
4154:76: ERROR: Tag rect invalid
4155:75: ERROR: Tag rect invalid
4156:75: ERROR: Tag rect invalid
4157:76: ERROR: Tag rect invalid
4158:75: ERROR: Tag rect invalid
4159:83: ERROR: Tag rect invalid
4160:83: ERROR: Tag rect invalid
4220:159: ERROR: Tag nav invalid
4223:108: ERROR: Tag nav invalid
4235:123: ERROR: Tag aside invalid
4245:109: ERROR: Tag footer invalid
4258:56: ERROR: ID theme-menu already defined
4268:28: ERROR: Tag svg invalid
4269:71: ERROR: Tag rect invalid
4270:76: ERROR: Tag rect invalid
4271:75: ERROR: Tag rect invalid
4272:75: ERROR: Tag rect invalid
4273:76: ERROR: Tag rect invalid
4274:75: ERROR: Tag rect invalid
4275:83: ERROR: Tag rect invalid
4276:83: ERROR: Tag rect invalid
4292:28: ERROR: Tag svg invalid
4293:71: ERROR: Tag rect invalid
4294:76: ERROR: Tag rect invalid
4295:75: ERROR: Tag rect invalid
4296:75: ERROR: Tag rect invalid
4297:76: ERROR: Tag rect invalid
4298:75: ERROR: Tag rect invalid
4299:83: ERROR: Tag rect invalid
4300:83: ERROR: Tag rect invalid
4316:28: ERROR: Tag svg invalid
4317:71: ERROR: Tag rect invalid
4318:76: ERROR: Tag rect invalid
4319:75: ERROR: Tag rect invalid
4320:75: ERROR: Tag rect invalid
4321:76: ERROR: Tag rect invalid
4322:75: ERROR: Tag rect invalid
4323:83: ERROR: Tag rect invalid
4324:83: ERROR: Tag rect invalid

If I'm incorrect in that conclusion, and you can provide a small, self-contained way to reproduce this issue, I'd be happy to look more deeply.

@flavorjones It's ok for me:
C:\Users\lem0nify>nokogiri "https://docs.microsoft.com/en-us/windows/win32/api/winuser/"
Your document is stored in @doc...
irb(main):001:0>

Also, if I write like this:
doc = Nokogiri::HTML(open("https://docs.microsoft.com/en-us/windows/win32/api/winuser/"))
puts doc.css('h2#functions').first.next_element

instead of
doc = Nokogiri::HTML(open("https://docs.microsoft.com/en-us/windows/win32/api/winuser/"))
puts doc.css('h2#functions + div')

it works fine.

@lem0nify OK, I'm guessing that I didn't explain myself as clearly as I could, so I'll do work I was asking you to do to further investigate.
#! /usr/bin/env ruby

require "nokogiri"
require "open-uri"

html = URI.open("https://docs.microsoft.com/en-us/windows/win32/api/winuser/")

doc = Nokogiri::HTML(html)

File.open("bar.html", "w") { |f| f.write doc.to_html(:indent => 2) }

doc.at_css("h2#functions").parent.children.map(&:name)
# => ["text",
#     "h1",
#     "text",
#     "ul",
#     "text",
#     "nav",
#     "text",
#     "comment",
#     "text",
#     "p",
#     "text",
#     "ul",
#     "text",
#     "h2",
#     "text",
#     "table",
#     "text",
#     "h2",
#     "text",
#     "table",
#     "text",
#     "h2",
#     "text",
#     "table",
#     "text",
#     "h2",
#     "text",
#     "table",
#     "text",
#     "comment",
#     "text"]

doc.css("h2#functions").first.next_element.name
# => "table"

The first query demonstrates that there are no sibling div elements.
The second query demonstrates that the code you wrote above (css().first.next_element) is a table, which is specifically not what the CSS query is searching for.
Nokogiri is performing CSS queries using + correctly, as far as I can tell; nothing in this issue demonstrates anything that would cause me to investigate further.

@flavorjones Ok I see.
But then why do I see div element in Chrome dev tools on this page? Do they really wrap table into div or not depending on the User-Agent? 🤔

From my comment above:

is it possible that libxml2 is parsing that document differently than you expect (and differently from your browser) because it's malformed? And thus the resulting DOM is different enough to make this query return different results?

HTML and XML parsers, generally speaking, will parse well-formed markup identically, because there's a spec. There is no spec and no formal W3C guidance on how to correct or "fix up" malformed markup, and so every parser seems to do it differently. You're likely seeing differences between your browser's parser and libxml2 (which is what Nokogiri uses). There's nothing we can easily do to change this behavior, unfortunately.

Oh, and one other possibility: libxml2 does not handle HTML5, and so there are likely to be differences between your browser's DOM and Nokogiri's DOM for any HTML5 document. If this is the case here, you may want to try using https://github.com/rubys/nokogumbo to parse the doc!

Oh! And one more possibility: this site looks like it might be using javascript to add wrapper elements to the DOM.

Confirmed: if I turn off javascript in my browser, the <div> element is not present (is not created).
I hope I've been able to help?

Confirmed: if I turn off javascript in my browser, the <div> element is not present (is not created).
I hope I've been able to help?

Yes. Thank you!


increase pmw3901 publish rate and max_ground_distance

the optical flow sensor was publishing at 20 hz and limited estimation to 5m above ground.
the increased publishing rate of 45 hz enables better yaw estimation performance.
the increased max_ground_distance enables more use cases of the optical flow sensor.
FYI @priseborough

Looks good to me

@AnnaDaiZH Can you also try changing from LPWORK to HPWORK. I wanted this to go in because LP seems to give too much jitter.

@DanielePettenuzzo I tested it an it gives less jitter .
before

after

I think we can merge it

Tested successfully with PixHawk V4 and Pesky PMW3901 breakout using master with this PR. I am using it as fallback for vision fusion (EKF2 mask 43):

Just being curious about some random spikes:

@dagar any idea how we can make space on fmu-v2 for this small pr? This really improves the performance of flow with the pmw3901

@ecmnet thanks for testing

@dagar any idea how we can make space on fmu-v2 for this small pr? This really improves the performance of flow with the pmw3901

It's a separate issue from the change here. Can you rebase?


Move fault injection filter before Mixer

With this PR: https://github.com/istio/proxy/pull/771
Proxy can call Mixer report for any requests rejected by other filters regardless before or after Mixer filter.
It is better to move fault injection filter before Mixer Check so quota is not wasted if quota is enforced.

Should Mixer allow requests rejected by the fault filter?
Conversely, should the fault filter reject requests allowed by Mixer?
What about any other extensions filter like external auth one?


build error when 2 server certificates of size 5k and 5.5k is used with https

I am getting following build error when i use 2 server certificates since i need to send request to 2 different server:
../project/build/main/libmain.a(firmware_status_update.o):(.literal.fw_update_task+0x4): undefined reference to _binary_server1_pem_end' ../project/build/main/libmain.a(firmware_status_update.o):(.literal.fw_update_task+0x8): undefined reference to _binary_server1_pem_start'
collect2: error: ld returned 1 exit status
When i use one certificate it works fine.
output of make partition_table:
Partition table binary generated. Contents:

Espressif ESP32 Partition Table
Name, Type, SubType, Offset, Size, Flags
nvs,data,nvs,0x9000,16K,
otadata,data,ota,0xd000,8K,
phy_init,data,phy,0xf000,4K,
ota_0,app,ota_0,0x10000,1536K,
ota_1,app,ota_1,0x190000,1536K,

and make size output:
Total sizes:
DRAM .data size:   14752 bytes
DRAM .bss  size:   57304 bytes
Used static DRAM:   72056 bytes ( 190088 available, 27.5% used)
Used static IRAM:   75028 bytes (  56044 available, 57.2% used)
Flash code:  832003 bytes
Flash rodata:  289396 bytes
Total image size:~1211179 bytes (.bin may be padded larger)
size of server1.pem is 5K
size of server2.pem is 5.5K
I have read all binary embedded goes to .rodata section of flash, but i don't know what is happening at memory level.
Please help.

What is your component.mk look like?

Components.mk file:

#
#  "main" pseudo-component makefile.
#
# (Uses default behaviour of compiling all source files in directory, adding 'include' to include path.)
# embed files from the "certs" directory as binary data symbols
# in the app
#COMPONENT_EMBED_TXTFILES := wpa2_ca.pem
#COMPONENT_EMBED_TXTFILES += wpa2_client.crt
#COMPONENT_EMBED_TXTFILES += wpa2_client.key
COMPONENT_EMBED_TXTFILES := server1.pem
COMPONENT_EMBED_TXTFILES := server2.pem

Seems like you missed '+':
COMPONENT_EMBED_TXTFILES += server2.pem

Thanks jkearins,
I was missing that only, i dont know how i can miss that, this was there in commented certificates. It's building now.  :)


Add --max-already-downloaded option

Before submitting a pull request make sure you have:

[x] Searched the bugtracker for similar pull requests
[x] Read adding new extractor tutorial
[x] Read youtube-dl coding conventions and adjusted the code to meet them
[ ] Covered the code with tests (note that PRs without tests will be REJECTED)

👉   I don't have tests for this yet. I couldn't find a test for max_downloads either; any pointers on how to best test this?

[x] Checked the code with flake8

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

[x] I am the original author of this code and I am willing to release it under Unlicense

What is the purpose of your pull request?

[x] New feature

Description of your pull request and other information
This PR adds a new --max-already-downloaded switch, similar to --max-downloads, but it only takes into account files that would have been redownloaded.
To facilitate that,

⚠ it minutely changes the API for FileDownloader.download() to, instead of True, return a string representing the reason a download succeeded; currently it's either "ok" or "already_downloaded". I wouldn't expect any downstream users to use if ...download() is True, and strings are truthy anyway, so this should work just fine
it adds a new super DownloadLimitReached exception class from whence MaxDownloadsReached and the new MaxAlreadyDownloadedReached exceptions are derived, and catches it wherever MaxDownloadsReached used to be caught before.
it adds the new option to the argument parser and plops it into the ytdl instance's options as you might expect
it adds the necessary counter to the ytdl object

As a side effect, it also removes an unused copy of report_file_already_downloaded.

Guess not 😁


error response from daemon

My config file is in the "app" folder, I have checkedout dev, my docker with its image are ok.
I am new to Docker so I can't find the solution.
btw I tried the different ways to call the directory in the console.

thanks

Hey @chomanoh ,
Use PowerShell, rather than CMD. $PWD doesnt work in CMD.

Ok thanks !


Add helper script for building NATFIXMEs

Note to self: start working on a YAML lib, so we can run the new script under Natalie, and offload the YAML generation in the test runner to the lib.
Or maybe I should just finish all the things I have in progress first. :wink:

This didn't prove as useful as I thought it would. Closing for now.


Best way to handle failed checks in middleware

Hi -- I have a bunch of middleware classes that check that certain headers are in place and correct before everything is passed on to the routes.
My reading of the code is that the way to handle cases where the checks in the middleware fails is as follows:

Middleware sees that a header is incorrect
Middleware sets the response object's code to 500 (or whatever)
Route checks the response object's code -- if it isn't the default 200, then handle as some sort of error

That's what I've been going with, anyway, but I thought I'd ask here in case I'd missed some other, more idiomatic way of handling these cases.
Oh, and crow totally rocks, I'm really enjoying the process of learning it -- thank you! :)
Doug.

Do you want to stop propagate the invalid request to the handler?
Then you can use res.end() in mw.before_handle with appropriate response.
You can see the example code here: https://github.com/ipkn/crow/blob/master/tests/unittest.cpp#L744
If you want to fix wrong headers in the middleware and just to notify to the handler, Saving that information to a variable in the middleware's context is better I think.
You can place one middleware for notifying purpose and use it from another middlewares like  all_ctx.template get<NotifyMW>().has_error = true, so the handler only need to check NotifyMW's context.


#544756: [templates/nextjs-sxa] added cdn for font-awesome font

[x] This PR follows the Contribution Guide
[x] Changelog updated

Description / Motivation

There was problem with CORS and getting fonts
Added CDN for FontAwesome fonts

Testing Details

[ ] Unit Test Added
[x] Manual Test/Other (Please elaborate)

Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

@matkovskyi Please, merge conflicts


Ups broker can now return a 500 error during provisioning when requested

The broker returns the error if you include a "return500": "whatever" in
the ServiceInstance's parameters

I'm not in favor of this. While, as of now, the UPS broker is in a bad state and just a toy (even though its not supposed to be), when the current work to "make it real" is done we do not want to have logic like this in there. Adding it now may make  people expect it in the future. If we want to have logic like this I would suggest we create a test broker (and call it "test") so people know its purpose in life and don't confuse it with something that is meant to be real.

Huh, my bad. I've always thought the UPS broker is just a demo & test broker.

Sadly, today apparently it is  :-(


Implement logging enhancements and other refactoring

unify https://github.com/opencomputeproject/OSF-OSFCI/search?q=SetConfigType idiom
log / track downloads

https://github.com/opencomputeproject/OSF-OSFCI/pull/21 addresses log/track downloads


16111 Webview contentInset docs change

Addresses #16111
Test Plan
No tests needed, non functional change. Test suite ran: PASS

cc @janicduplessis

@facebook-github-bot shipit

@paulbrittain Could you sign the CLA please?

@janicduplessis signed

@facebook-github-bot shipit

@janicduplessis Seems this PR hasnt gone live. The doc isnt reflecting the platform.
Please let me know if I can help with anything here.


Free nanopb mallocs when losing references to them

Fixes #11027.

Tested manually via the quickstart and there's no more leaks.

Resolving comments and addressing feedback in a follow-up to make the release deadline.


Query params as Enums are not working fine with swagger docs??

@tiangolo @SirTelemak  I am facing a strange issue here. I am on version 0.63.0.
I have created an Enum as below:
class DetailsLevel(Enum):
    STANDARD = 'standard'
    FULL = 'full'

And my route looks like:
@router.get(path="",
            description="Retrieve a list of host network object from Checkpoint",
            response_model=CheckpointPaginatedResponse[List[HostResponseModel]],
            responses={
                **CHECKPOINT_RESPONSES,
                404: {
                    "description": "Unable to find requested resource",
                    "model": ErrorModel
                },
            })
async def list_hosts(page_size: int = 500, page: int = 1,
                     x_session_id: str = Header(None, description="Checkpoint session header"),
                     details_level: DetailsLevel = DetailsLevel.STANDARD) -> \
        CheckpointPaginatedResponse[List[HostResponseModel]]

But the generate swagger is not including all the values. What I see is --- and standard. Not sure I am missing anything here.

Try:
class DetailsLevel(str, Enum):
    STANDARD = 'standard'
    FULL = 'full'

I tried that. But it was not working. I was on pydantic 1.7.3 as a random try updating pydantic to 1.8.1 and it worked.

did you take a look at #2910 #2914

Thanks for the help here @Kludex and @valentin994 ! 👏  🙇
@parth7676 if you haven't been able to solve the problem yet, please add a self-contained, minimal, reproducible, example that I can copy-paste to replicate it.
If you solved the original problem, then you can close this issue ✔

Sorry for the long delay! 🙈  I wanted to personally address each issue/PR and they piled up through time, but now I'm checking each one in order.


Add entity and sdf parameters to Server's AddSystem interface

🦟 Bug fix
Fixes #2322
Summary
Entity and XML content information are lost when adding systems through the Server::AddSystem(system) API. To reproduce it:

Create a dummy system plugin:

class DummyPlugin : public gz::sim::System,
  public gz::sim::ISystemConfigure,
{
public:
  void Configure(
    const gz::sim::Entity &_entity,
    const std::shared_ptr<const sdf::Element> &_sdf,
    gz::sim::EntityComponentManager &_ecm,
    gz::sim::EventManager &_eventMgr) override
  {
    std::cout << _sdf->ToString("") << std::endl;
  }
};

Load it with the SystemLoader (extracted from here):

  sim::SystemLoader systemLoader;
  sdf::Plugin sdfPlugin("dummy_plugin", "gz::sim::systems::DummyPlugin", "<test>foo</test>");
  auto mockSystemPlugin = systemLoader.LoadPlugin(sdfPlugin);
  EXPECT_TRUE(*server.AddSystem(mockSystemPlugin.value()));

Without this patch, the code above will output:
<plugin filename='__default__'/>

After the patch, you will be able to use the extended Server::AddSystem(plugin, entity, sdf) API and get:
<plugin filename='dummy_plugin' name='gz::sim::DummyPlugin'>
  <test>foo</test>
</plugin>

Checklist

[x] Signed all commits for DCO
[x] Added tests
[x] Updated documentation (as needed)
[x] Updated migration guide (as needed)
[x] Consider updating Python bindings (if the library has them)
[x] codecheck passed (See contributing)
[x] All tests passed (See test coverage)
[ ] While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge and edit the commit message to match the pull request summary while retaining Signed-off-by messages.

Amended to fix linting issues

That would encapsulate loading the plugin and extracting the _sdf element from sdf::Plugin. Thoughts?

Sounds good to me. May I do it in this PR ?

That would encapsulate loading the plugin and extracting the _sdf element from sdf::Plugin. Thoughts?

Sounds good to me. May I do it in this PR ?

Go for it!

@azeey Comments have been addressed.
Thank you for the review!

@azeey Had to amend again because of another linter issue, sorry about that.
BTW, the contribution guide mentions a codecheck target but the current cmake configuration doesn't seem to be generating that:
$ make codecheck
make: *** No rule to make target 'codecheck'.  Stop.

How should I run cpplint and cppcheck on gz-sim properly?

@azeey Had to amend again because of another linter issue, sorry about that.
BTW, the contribution guide mentions a codecheck target but the current cmake configuration doesn't seem to be generating that:
$ make codecheck
make: *** No rule to make target 'codecheck'.  Stop.

How should I run cpplint and cppcheck on gz-sim properly?

You must install cppcheck first for the codecheck target to be available.

You must install cppcheck first for the codecheck target to be available.

Got it. I think we are good to go now.

No problem. Amended again.

@azeey @iche033 Any idea when this might be released?


rpcd-mod-*: improve postinst script

Usage of killall is replaced with init script. This is cleaner solution
as it does not consider some implementation detail but rather passes
that on to init script implementation.
IPKG_INSTROOT was added to prevent execution when not running in current
root. It is invalid to request reload if install-root is not current
root. In this case it can be considered harmless but it is invalid
nonetheless.
Last change is removal of exit 0. This caused skip of default
postinst. Execution of default postinst does no harm and is more
standard considering possible future expansion of it.

Merged, thanks.

@jow- Would it be possible to include it in OpenWrt 19.07 branch?


Rename from GoblBuilder to GOBLBuilder

This is open to discussion... but in general I think its nice to be consistent with GOBL and use it either as gobl or GOBL when name-spacing.

Yes, agree!


Viewer share upgrades

There is now a feedback button, share button and download button. Each with helpful tooltips. The feedback button no longer copies the s3 link to your clipboard. They have a consistent style in the upper right.

Look fine, seems to be working 👍
I tried submitting some feedback but I'm not sure if/how I can view the s3 bucket, so I haven't double-checked but I'm sure its fine.

Oops there was a problem with the feedback button. It's fixed now. You should be able to see the s3 bucket in the AWS console. When a user clicks feedback we should also get pinged in the dogfooding slack channel.


MLOps Demo: Increase Size of Embeddings

Increased the size of embeddings from 50 to 85
Hypothesis: this makes the models better

/run-full-test

/deploy t9dudzm6


Add check for bad IRIs

#1156 slipped past all our tests. We need a new check to catch bad IRIs.

Can we assume that the RDFXML of the OWL will always come from ROBOT? I can write a simple script to just parse this very quickly.

Maybe start with the first column of build/terms-report.tsv, then remove everything known good, and see if there's anything left. Even sed would work for that, but Python is also fine.

I was thinking we could look at the prefixes in the OWL, and then any term that does not start with a prefix is considered invalid. What do you think of that method, or would you prefer using build/terms-report.tsv?

That sounds fine.

The PR is good. Now I think I'd like to add a few more checks.

I'd like to check that all OBI IDs are seven digits. There's actually one eight digit IRI that I messed up years ago -- see if you can find it!

What's Protege's current default for new terms? It used to be the file's base URI then # then a hyphenated string: http://purl.obolibrary.org/obo/obi.owl#my-label. Although there are exceptions, let's fail if any string starts with "http://purl.obolibrary.org/obo/" and also includes a "#".

Please spend a few minutes researching Protege defaults to see if there are other likely problems.

That caught the bad IRI - http://purl.obolibrary.org/obo/OBI_00001975
Should we just exclude this one from the 7 digit check, and fail otherwise?

Yes, we have to make that one a special case.

OK - the PR is updated!


[release-4.13] OCPBUGS-21556: bump golang.org/x/net to v0.17.0

Manual backport of #20

/retest

/jira refresh
/retest

/lgtm
/approve
/backport-rist-assessed

/label backport-risk-assessed

/retest

/test images

/retest

/retest

/hold

/approve
/label backport-risk-assessed

/retest

/retest

/retest

/retest

/test e2e-vsphere-techpreview

/tide refresh

ci/prow/e2e-vsphere-techpreview seems to be broken for unrelated reasons (CI is stuck building the image).
That is not a big deal as the job is optional and the vpshere component is not even in TechPreview.
/unhold

/approve

/assign @JoelSpeed

/approve
/lgtm
/label backport-risk-assessed

/label cherry-pick-approved

/label cherry-pick-approved

@sunzhaohua2  It looks like something went wrong when applying the cherry-pick-approved label, any idea why?

/override ci/prow/e2e-vsphere-techpreview


Concurrent writes in LanceDB with multiprocessing?

Hello all,
I'm loving LanceDB so far but I am trying to accelerate writes. Since it looks like each row in the database is written as a separate file, it seems possible for multiprocess writes unless there's some locking on the file system in the folders other than data.
Can you advise how to write to LanceDB from multiple processes at once?  Right now I'm doing this:
def trainer():
    global queue
    print(f"Database initializing in process {os.getpid()}")
    db = Db()
    print(f"Initialized database. Beginning doc processing in pid {os.getpid()}")
    db.process_corpus_doc_file()
    print(f"Processing in process {os.getpid()} is complete.")

processes = []
for i in range(num_procs):
    # Stick a None in the queue for each process to consume and exit.
    queue.put(None)
    p = mp.Process(target=trainer)
    p.start()
    processes.append(p)

Each copy of Db() tries to open a connection to the same LanceDB database and table in parallel, but I only spin up the multiple loader processes after I've initialized the table once in the parent process. Is this a problem? Right now it seems to crash my computer entirely, so I'm guessing it might be causing a race condition or something.
Thanks!

Since it looks like each row in the database is written as a separate file

This is true, only if you are writing one row at a time. If this is the case, I think before trying multiple processes you might be better off trying to write multiple rows at a time, which is what LanceDB is optimized for. We have parallelism internally, so we can take advantage of multiple cores without the need for multiprocessing.
How are you getting your rows / batches that you are writing? Our example shows using a Pandas dataframe, but if you have Arrow record batches that is written even more efficiently.

I'm currently reading in a jsonlines file and adding one row at a time. I could batch. However, it takes longer to calculate document embeddings than to save to the database, so it would be most efficient to have many embedding threads writing to the database at once.
I was trying to calculate the embeddings separately in Sentence Transformers and then save them to the database. If I use your with_embeddings functionality, will it be multi-processor by default without me worrying about multi-processing?

However, it takes longer to calculate document embeddings than to save to the database

It sounds like the bottleneck is calculating embeddings rather than writing the data. That makes sense given how computationally intensive it can be.

so it would be most efficient to have many embedding threads writing to the database at once.

Have you considered distributing only the embeddings calculation, and keeping the database write as a single thread? As in: a simple fan-out then fan-in model. That seems like the simplest approach for this use case.
We do support distributed writes, though the API for that isn't well document. But like I said earlier, I don't think this is appropriate for the use case you are describing. This is mostly useful in a distributed system where there are multiple independent nodes; for single-node systems the single-process writer will utilize your IO resources pretty well.
To be clear: if you write one row per file in LanceDB, we can't promise good performance. There is overhead for each file in the table, and this overhead is meant to be amortized over thousands to millions of rows. We'll likely provide utilities functions to compact small files into larger ones to optimize existing tables, but we don't have that yet. So for the sake of good read performance, I'd encourage you to write in batches.

I'm running using your advice now, and I'm using with_embeddings instead of running my embedder in a separate thread, since you said you handle all the optimization of multiprocessing. I see that it's maxing out at 200% on a 4-processor box, so it doesn't really seem to be optimized to use all of the processors.
Can you advise? Am I doing something wrong? See my top screenshot below.

I'm using with_embeddings and writing in batches instead of running my embedder in a separate thread, since you said you handle all the optimization of multiprocessing

Sorry, I didn't mean to imply with_embeddings uses multiple threads. AFAIK it does not.
My suggestion was to write a "fan-out fan-in" program with three process types:
  graph LR;
      read_worker-->embedding_worker_1;
      read_worker-->embedding_worker_2;
      read_worker-->embedding_worker_3;
      read_worker-->embedding_worker_4;
      embedding_worker_1-->write_worker;
      embedding_worker_2-->write_worker;
      embedding_worker_3-->write_worker;
      embedding_worker_4-->write_worker;

read_worker A process that reads the JSON lines and sends them to the embeddings workers
embedding_worker N processes that generate embeddings for rows they are given, then sends the to the write worker
write_worker A process that collects rows into batches and writes them to lance once a sufficient number have been collected.

This way, you parallelize the CPU-bound part (computing embeddings), but batch it all in one writer. Does that make more sense now?
We could eventually make with_embeddings do this for you.

That does make sense. Thanks!
On Wed, Jun 21, 2023 at 9:45 AM Will Jones @.***> wrote:

I'm using with_embeddings and writing in batches instead of running my
embedder in a separate thread, since you said you handle all the
optimization of multiprocessing
Sorry, I didn't mean to imply with_embeddings uses multiple threads.
AFAIK it does not.
My suggestion was to write a "fan-out fan-in" program with three process
types:
graph LR;
read_worker-->embedding_worker_1;
read_worker-->embedding_worker_2;
read_worker-->embedding_worker_3;
read_worker-->embedding_worker_4;
embedding_worker_1-->write_worker;
embedding_worker_2-->write_worker;
embedding_worker_3-->write_worker;
embedding_worker_4-->write_worker;

read_worker A process that reads the JSON lines and sends them to
the embeddings workers
embedding_worker N processes that generate embeddings for rows they
are given, then sends the to the write worker
write_worker A process that collects rows into batches and writes
them to lance once a sufficient number have been collected.

This way, you parallelize the CPU-bound part (computing embeddings), but
batch it all in one writer. Does that make more sense now?
We could eventually make with_embeddings do this for you.
—
Reply to this email directly, view it on GitHub
https://github.com/lancedb/lancedb/issues/213#issuecomment-1601197526,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADNSOFLVWPI2SKGAXAN6HPDXMMQLVANCNFSM6AAAAAAZN3G5GA
.
You are receiving this because you authored the thread.Message ID:
@.***>

Please feel to reopen if you're still having the same issue


add draft content for docs issue 819

Signed-off-by: JENNIFER RONDEAU<EMAIL_ADDRESS>Fixes #819
@rbankston is this what you had in mind? File location is not as specified in original issue, but I think it's more effective here.
Note also that this PR pulls apart the old use case topic as a start at reorganizing use cases/tutorials. Not fully addressing any other docs reorg issue yet, though.

@Bradamant3 this LGTM, can you squash down to 1 commit and I'll merge?

@Bradamant3 reminder, just waiting on you to squash here.


Fixed the "image.addEventListener is not a function" browser error

Fixes #1164
The generated ES6 Image module had this in the Image.__fromBase64() static function:
var image = new Image()
which would instantiate a new object of type lime.graphics.Image as opposed to the intended HTML Image type.

Awesome, thanks!


Move Android environment variables to /secure/

Inventory should do the way SkyService does:

https://github.com/getditto/demoapp-skyservice/pull/27

chat is using secure/ instead of secrets/ -- should we standardize that to make it easier to publish them all to the app store with a script?

This is already done


Workflow execution failed [Screenshot demo]

I am getting error:
Cloud Run long-running operation failed.
in step "run_job", routine "main", line: 71

{
  "message": "Cloud Run long-running operation failed.",
  "status": {
      "cancelledCount": 6,
      "completionTime": "2022-05-02T13:10:56.596220Z",
      "conditions": [
        {
          "lastTransitionTime": "2022-05-02T13:10:58.146380Z",
          "message": "Execution failed.",
          "reason": "NonZeroExitCode",
          "status": "False",
          "type": "Completed"

I couldn't find any documentation reference related with googleapis.run.v1.namespaces.jobs.run.

It's probably because you don't have your project allow-listed for Cloud Run jobs.

Closing this now, please reopen if it's still an issue


[FLINK-3437] [web-dashboard] Fix UI router state for job plan

When viewing a single job the "Plan" page (first tab at top) has nested states (.plan.subtasks, .plan.taskmanagers, ...). Since the "Plan" link triggered the .plan.subtasks state it was inactive for sibling states such as .plan.taskmanagers when selecting secondary tabs from the middle of the page.
This commit changes the "Plan" link to trigger state .plan which is redirected to .plan.subtasks in the state configuration. The "Plan" link is then active for any of the nested sub-states (since a UI router parent state is active when any child state is active).

I'll test the change locally

It seems to work. I'm merging the change.


Fixed gh-1030

#1030

The change here works..however, I needed to check with my employer on the contributor agreement. I created the pull request before I read that section.


stfpy: Added support to get header comments

Hello @bdutro,
We have a demand to get header comments from STFInstReader via stfpy. I dug into this and came up with a patch. However I’m not sure about my implementations as follows:

I need to get a vector of strings as type for header comments. However it was stored in STFReaderBase as std::vector<ConstHandle<CommentRecord>>. It was frustrating to me to implement the same interface in Cython. I kinda cheated it to add another private data member header_comments_str_ to save comments as strings when reading the header. But not sure if that’s the best way to implement it. Do you have thoughts on this?

I added a Cython class StringVector to accommodate the above data from STF C++ library. Not sure if I made(/wrote) it in the right convention/style. Let me know if it doesn’t work for you. Thanks!

I added a Cython class StringVector to accommodate the above data from STF C++ library. Not sure if I made(/wrote) it in the right convention/style. Let me know if it doesn’t work for you. Thanks!

I just renamed it to HeaderCommentsType. Should be fine now.

I need to get a vector of strings as type for header comments. However it was stored in STFReaderBase as std::vector<ConstHandle>. It was frustrating to me to implement the same interface in Cython. I kinda cheated it to add another private data member header_comments_str_ to save comments as strings when reading the header. But not sure if that’s the best way to implement it. Do you have thoughts on this?

Since we generally don't need to read header comments, what if instead you only populated that vector if someone calls getHeaderCommentsString?

I need to get a vector of strings as type for header comments. However it was stored in STFReaderBase as std::vector. It was frustrating to me to implement the same interface in Cython. I kinda cheated it to add another private data member header_comments_str_ to save comments as strings when reading the header. But not sure if that’s the best way to implement it. Do you have thoughts on this?

Since we generally don't need to read header comments, what if instead you only populated that vector if someone calls getHeaderCommentsString?

That works for me! That’s how I did right now. So you’d suggest I could just remove the getHeaderComments method but keep getHeaderCommentsString?

You can leave both methods. It doesn't hurt to give access to the underlying CommentRecords. What I'm saying is leave header_comments_str empty until someone calls getHeaderCommentsString() for the first time.

I see, will do!

You can leave both methods. It doesn't hurt to give access to the underlying CommentRecords. What I'm saying is leave header_comments_str empty until someone calls getHeaderCommentsString() for the first time.

Done!


Use ActionFailure instead of fail for selecting rest of seq

I'm getting a different error now:
Quickstrom Test Report
Failed with error: WebDriverOtherError "E (ResponseError ElementNotInteractable \"element not interactable\\n (Session info: headless chrome=85.0.4183.83)\" \"#0 0x55aeed744aa9 <unknown>\\n\" Nothing (Status {statusCode = 400, statusMessage = \"Bad Request\"}))"

WebDriverOtherError "E (ResponseError ElementNotInteractable \"element not interactable\\n (Session info: headless chrome=85.0.4183.83)\" \"#0 0x55aeed744aa9 <unknown>\\n\" Nothing (Status {statusCode = 400, statusMessage = \"Bad Request\"}))"

I've seen this error many times before but it's never blown up the tests completely. I think you've mentioned that you know the cause of it. Did the behavior somehow change on this one to now cause the tests to blow up completely (rather than continue on)?

That is weird. Looking into it and trying to replicate. Do you know if this happened during a "full" test or during a shrink?

There’s no mention of shrinking. Here’s the full cli output for the html run (and my previous paste was the html contents):
Version: 0.3.0
35

36
Running 5 tests...
37

38
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
39

40
20 actions
41
2021-04-23 17:21:45 ERROR  Error Response: 400 Bad Request
42
{
43
    "data": null,
44
    "error": 400,
45
    "stacktrace": "#0 0x55aeed744aa9 <unknown>\n",
46
    "message": "element not interactable\n  (Session info: headless chrome=85.0.4183.83)"
47
}
48

49
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
50

51
Error: Process completed with exit code 1.

This might be two problems at once:

Some action not being sufficiently validated before picked (causing the "element not interactable" error)
Quickstrom crashing because of this error, not just storing an action failure in the trace

Can you try removing actions from your spec to find out what action (sequence) causes this?


Errors during Build coverage file

Thanks for the workflow template! It is exactly what I was looking for.
Unfortunately, I'm hitting the following error when getting to the Build coverage file:
  python -m pytest --cov='.' tests > pytest-coverage.txt
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.8.8/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.8.8/x64/lib
Error: Process completed with exit code 2.

The repository I'm trying to add this workflow to is here.
Any idea what I'm doing wrong? I tried with and without the python -m. My tests are in the tests folder and all the source code is in the top-directory. The tests run fine locally on my machine, but not within the workflow scripts.
Any help would be appreciated. Thanks!

Found a solution on similar GitHub Action: Pytest Coverage Comment you can give it a try.


please tag v0.8.0 or v0.7.1 containing at least 5a0a231c0

The last tagged release was v0.7.0, from Feb 2018.
For tools that prefer tagged releases (dep, vgo), there is no tagged release containing 5a0a231c0, which means quic-go fails to build against the newest golang.org/x/net (because golang.org/x/net/lex/httphex moved into golang.org/x/net/http/httpguts). Personally I think it was uncool to delete golang.org/x/net/lex/httplex, but that ship has sailed.
I don't know what else you've got going on, but when things are in a stable state (or if you remember a stable state) it would be helpful if you could tag a release containing (or cherry-picking) 5a0a231c0. Thanks.

Hello @rsc,
I was about to make a new release today anyway, so here it is: v0.8.0..


Disable Bootstrap-Select on IE 7

Due to its incompatibility. How can I disable bootstrap select on IE 7?

Check for version, and then call $('.selectpicker').selectpicker('destroy').


Compatibility of Spyder standalone 5.4.4 with Windows Server 2016

I tried to install Spyder standalone 5.4.4 with Windows Server 2016 in a VPS and got the following error.

Hi @GastaoDM thank you for the feedback! Just adding some tracebility here: Issue opened following comments from https://github.com/spyder-ide/spyder/issues/19079

@dalthviz and @mrclary, I think we should remove Spyder-terminal from our installers given that it's unmaintained and the inability of disable it right now.

Just in case, as a workaround in the meantime and if is it okay for @GastaoDM to use an alpha version, we have a differen Windows installer for the 6alpha1 release. I think it doesn't come with the spyder-terminal plugin so the error in the screenshot should not happen


It appears that the downloaded file is corrupt; please try again.

git clone -b master https://github.com/flutter/flutter.git
./flutter/bin/flutter --version
"It appears that the downloaded file is corrupt; please try again."

Steps to Reproduce

...
...
...

Logs

Logs

@nazilaeshghi
Can you provide the entire error log ? Which file is it pointing to, that throws this error ?
Also, you can work around this by deleting your cache, via:
rm -rf /home/ryan/programming/flutter/bin/cache
/home/ryan/programming/flutter/bin/flutter --version

And try again to see if it works for you.


List chown_cyhy_dirs as a dependency for several roles

Actually, I'm not sure the chown_cyhy_dirs bit is correct.  It's not a dependency since it really needs to run after cyhy-reports, for example, is installed.

Nevermind, everything is OK.  The point of the chown_cyhy_dirs is to chown to cyhy:cyhy all the cyhy-stuff that was installed in the packer Ansible.  (We couldn't do it then because the cyhy user didn't exist yet.)
All the configuration that is done in the cyhy_* roles in the terraform Ansible is done with the intended user and group.
Do you agree, @dav3r?

@jsf9k - Yes, that makes sense.


Saving and loading of chat history should be a logic action

Is your suggestion for improvement related to a problem? Please describe.
Saving and loading of chat history is a logic action.
But the logic layer doesn't have the facilities to integrate new loading and storing actions (or, maybe, I haven't found it).
In the GUI layer there are GUIPostOpenAction and SaveDatabaseAction. These classes where a convenient place to implement the chat history saving and loading.
Describe the solution you'd like
As the title says, saving and loading of chat history should be a logic action.
@koppor said, that the Carl is working on moving saving/loading code from GUI to logic. Guess, we have to wait.
What we have know is a workaround that works correctly.

But the logic layer doesn't have the facilities to integrate new loading and storing actions (or, maybe, I haven't found it).

Just create the methods there and call them at SaveDatabaseAction. The GUI layer is the place to call logic actions.
The reason why org.jabref.gui.exporter.SaveDatabaseAction#saveDatabase is still in the UI is the undo manager (see line 276).
As a coding excercise one could move the methods saveDatabase to somewhere in  logic. But it is a rabbit hole, because org.jabref.logic.autosaveandbackup.BackupManager.performBackup also needs to be adapted then.

Closed because of using MVStore #48.


Badge Request: GitHub Check ignore pending builds

📋 Description
I have migrated to https://shields.io/badges/git-hub-branch-status, but compared to the CircleCI badge, it shows the latest build status:
while the CircleCI badge shows the latest non-pending build status. CircleCI seems the expected behavior. The use case is to be able to, with a glance, see if there is something to be worried about (broken CI branch).
It would be great if we could fix this.

Hello @oliviertassinari 👋🏻
Our GitHub branch status badge directly forwards the status returned by the GitHub API (see code) . I believe that this is the same status as the one shown in the GitHub UI (green check, red cross, orange dot). In addition to changing a historical behaviour which users seem happy with (this badge was implemented all the way back in #5973), I feel that introducing an inconsistency and having the badge show something different than the GitHub UI would be misleading and confusing.

I feel that introducing an inconsistency and having the badge show something different than the GitHub UI would be misleading and confusing.

@PyvesB I agree, these are fundamentally two different concepts. The bulk of this issue is to request the introduction of another concept that I would expect to match the most common use case for this badge.