mcw.py:40-43: Stop words filtering is weak. method...

The puzzle 137-9eb7df67 from #137 has to be resolved:
https://github.com/h1alexbel/sr-detection/blob/19e0c17eaeaaba801f533778f767437037793ffd/sr-data/src/sr_data/steps/mcw.py#L40-L43
The puzzle was created by @h1alexbel on 15-Oct-24.
Estimate: 45 minutes,  role: DEV.
If you have any technical questions, don't ask me, submit new tickets instead. The task will be "done" when the problem is fixed and the text of the puzzle is removed from the source code. Here is more about PDD and about me.

@0pdd Summary Main quality problem: Lack of clear problem description. Suggestion: Provide a con...

I see you're talking to me, but I can't reply since I'm not a chat bot.


⚠ Bofrost.de has degraded performance

In 2ec0366, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1586 ms

Resolved: Bofrost.de performance has improved in cea4ad5 after 29 minutes.


Replace rapidjson with nlohmann/json

Commit Message: Replace rapidjson with nlohmann/json
Additional Description:

Schema validation is not supported by the core nlohmann/json library. They endorse an option here https://github.com/nlohmann/json/issues/305 but schema validation is not used anywhere in Envoy core.
Added runtime option to revert back to RapidJson with feature envoy.reloadable_features.remove_rapidjson. It is enabled by default.
There is one usage in core to parse metadata in the HeaderFormatter: https://github.com/envoyproxy/envoy/blob/e073b325890542a3ce3bbc4eefb89a09f390f27e/source/common/router/header_formatter.cc#L61 This could be replaced pretty easily with protobuf's parser in a later PR
Originally I had an implementation that shared the SAX parser between the two implementations, but RapidJson and nlohmann/json have to handle tracking line numbers differently so I split them. (RapidJSON makes it easy to subclass the stream implementation, but nlohmann/json doesn't.)
The  major difference between the two json_internal.cc implementations is the custom stream class (see json_loader.cc line 243-280. It has a reference to the handler to update line numbers)

Risk Level: Low for core, maybe medium for those using JSON on the dataplane.
Release Notes: Added release notes about runtime feature
Testing:

json_loader_test.cc is parametrized over both runtime values.
Fixes https://github.com/envoyproxy/envoy/issues/4705

@asraa I'll let you tag a reviewer once you've got CI sorted, SG?

Yep! This is still in DRAFT mode, I'll move it out of draft mode when I finish some test TODOs (benchmarking)

/wait

There is an existing clang-tidy issue about a potential memory leak in the constructor of rapidjson::SchemaDoument https://github.com/envoyproxy/envoy/blob/12c42b599b053707dcfd13dfdd87c3cd75c95c59/source/common/json/json_loader.cc#L520
I couldn't find related issues, tests in rapidjson support this construction. Im not sure I want to spend too much time on a rapidjson issue
external/com_github_tencent_rapidjson/include/rapidjson/schema.h:1546:39: error: Potential memory leak [clang-analyzer-unix.Malloc,-warnings-as-errors]
        CreateSchemaRecursive(&root_, PointerType(), document, document);
                                      ^
source/common/json/json_internal_legacy.cc:511:3: note: Taking false branch
  if (schema_document.Parse<0>(schema.c_str()).HasParseError()) {
  ^
source/common/json/json_internal_legacy.cc:517:29: note: Calling constructor for 'GenericSchemaDocument<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >, rapidjson::CrtAllocator>'
  rapidjson::SchemaDocument schema_document_for_validator(schema_document);
                            ^
external/com_github_tencent_rapidjson/include/rapidjson/schema.h:1535:14: note: Field 'allocator_' is null
        if (!allocator_)
             ^
external/com_github_tencent_rapidjson/include/rapidjson/schema.h:1535:9: note: Taking true branch
        if (!allocator_)
        ^
external/com_github_tencent_rapidjson/include/rapidjson/schema.h:1539:24: note: 'uri' is null
        uri_.SetString(uri ? uri : noUri, uriLength, *allocator_);
                       ^
external/com_github_tencent_rapidjson/include/rapidjson/schema.h:1539:24: note: '?' condition is false
external/com_github_tencent_rapidjson/include/rapidjson/schema.h:1541:46: note: Calling 'CrtAllocator::Malloc'
        typeless_ = static_cast<SchemaType*>(allocator_->Malloc(sizeof(SchemaType)));
                                             ^
external/com_github_tencent_rapidjson/include/rapidjson/internal/../allocators.h:79:13: note: 'size' is 336
        if (size) //  behavior of malloc(0) is implementation defined.
            ^
external/com_github_tencent_rapidjson/include/rapidjson/internal/../allocators.h:79:9: note: Taking true branch
        if (size) //  behavior of malloc(0) is implementation defined.
        ^
external/com_github_tencent_rapidjson/include/rapidjson/internal/../allocators.h:80:20: note: Memory is allocated
            return std::malloc(size);
                   ^
external/com_github_tencent_rapidjson/include/rapidjson/schema.h:1541:46: note: Returned allocated memory
        typeless_ = static_cast<SchemaType*>(allocator_->Malloc(sizeof(SchemaType)));
                                             ^
external/com_github_tencent_rapidjson/include/rapidjson/schema.h:1546:39: note: Potential memory leak
        CreateSchemaRecursive(&root_, PointerType(), document, document);
                                      ^

There is an existing clang-tidy issue about a potential memory leak in the constructor of rapidjson::SchemaDoument https://github.com/envoyproxy/envoy/blob/12c42b599b053707dcfd13dfdd87c3cd75c95c59/source/common/json/json_loader.cc#L520
I couldn't find related issues, tests in rapidjson support this construction. Im not sure I want to spend too much time on a rapidjson issue
external/com_github_tencent_rapidjson/include/rapidjson/schema.h:1546:39: error: Potential memory leak [clang-analyzer-unix.Malloc,-warnings-as-errors]
        CreateSchemaRecursive(&root_, PointerType(), document, document);
                                      ^
source/common/json/json_internal_legacy.cc:511:3: note: Taking false branch
  if (schema_document.Parse<0>(schema.c_str()).HasParseError()) {
  ^
source/common/json/json_internal_legacy.cc:517:29: note: Calling constructor for 'GenericSchemaDocument<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator> >, rapidjson::CrtAllocator>'
  rapidjson::SchemaDocument schema_document_for_validator(schema_document);
                            ^
external/com_github_tencent_rapidjson/include/rapidjson/schema.h:1535:14: note: Field 'allocator_' is null
        if (!allocator_)
             ^
external/com_github_tencent_rapidjson/include/rapidjson/schema.h:1535:9: note: Taking true branch
        if (!allocator_)
        ^
external/com_github_tencent_rapidjson/include/rapidjson/schema.h:1539:24: note: 'uri' is null
        uri_.SetString(uri ? uri : noUri, uriLength, *allocator_);
                       ^
external/com_github_tencent_rapidjson/include/rapidjson/schema.h:1539:24: note: '?' condition is false
external/com_github_tencent_rapidjson/include/rapidjson/schema.h:1541:46: note: Calling 'CrtAllocator::Malloc'
        typeless_ = static_cast<SchemaType*>(allocator_->Malloc(sizeof(SchemaType)));
                                             ^
external/com_github_tencent_rapidjson/include/rapidjson/internal/../allocators.h:79:13: note: 'size' is 336
        if (size) //  behavior of malloc(0) is implementation defined.
            ^
external/com_github_tencent_rapidjson/include/rapidjson/internal/../allocators.h:79:9: note: Taking true branch
        if (size) //  behavior of malloc(0) is implementation defined.
        ^
external/com_github_tencent_rapidjson/include/rapidjson/internal/../allocators.h:80:20: note: Memory is allocated
            return std::malloc(size);
                   ^
external/com_github_tencent_rapidjson/include/rapidjson/schema.h:1541:46: note: Returned allocated memory
        typeless_ = static_cast<SchemaType*>(allocator_->Malloc(sizeof(SchemaType)));
                                             ^
external/com_github_tencent_rapidjson/include/rapidjson/schema.h:1546:39: note: Potential memory leak
        CreateSchemaRecursive(&root_, PointerType(), document, document);
                                      ^

Only failing existing clang-tidy. Would appreciate any comment! (especially around the new way of holding line_number_)

Only failing existing clang-tidy. Would appreciate any comment! (especially around the new way of holding line_number_)

Would it be helpful if I displayed the diff between the old/new json loader?
The full diff is here https://gist.github.com/asraa/67c51ac7ab48570e471a744fb4c02068
The main differences are:

renaming: SAX callbacks are named in nlohmann as start_object instead of StartObject and have slightly different signatures (all unused params though), asRapidJsonDocument is renamed to asJsonDocument.
The new Json ObjectHandler holds on directly to an updating line_number_ instead of holding on to a LineCountingStringStream with a line number member
The new custom JSON "stream" that counts line numbers is implemented as a custom container that increments line numbers whenever the character at a position is a newline
nlohmann/json had no API to push back items onto a json document with an allocator parameter
No schema validation
Slight differences in error messaging formatting. nlohmann/json will also give you the token at the error.

Would it be helpful if I displayed the diff between the old/new json loader?
The full diff is here https://gist.github.com/asraa/67c51ac7ab48570e471a744fb4c02068
The main differences are:

renaming: SAX callbacks are named in nlohmann as start_object instead of StartObject and have slightly different signatures (all unused params though), asRapidJsonDocument is renamed to asJsonDocument.
The new Json ObjectHandler holds on directly to an updating line_number_ instead of holding on to a LineCountingStringStream with a line number member
The new custom JSON "stream" that counts line numbers is implemented as a custom container that increments line numbers whenever the character at a position is a newline
nlohmann/json had no API to push back items onto a json document with an allocator parameter
No schema validation
Slight differences in error messaging formatting. nlohmann/json will also give you the token at the error.

ping @jmarantz this is ready.
clang-tidy will fail on an existing rapidJSON issue (see https://github.com/envoyproxy/envoy/pull/14467#issuecomment-758016926).

ping @jmarantz this is ready.
clang-tidy will fail on an existing rapidJSON issue (see https://github.com/envoyproxy/envoy/pull/14467#issuecomment-758016926).

@envoyproxy/senior-maintainers  Could you PTAL? This adds the code for a new json parser, does not flip the switch.


add .target0 conditional namespace

This includes the following conditionals:

.target0.mods.[name]: status/res/def/mods check

.target0.element.[name]: durability check

[ ] Verify syntax design.

[ ] Figure out how to handle deprecation better.

closes #905

I'd like to mention that the set_target_pos syntax is 1 indexed. It would be confusing for conditionals to be 0 indexed and the set_target_pos to be 1 indexed.


Phone capable of 4K@30 &1080p@60FPS records 4K@25 & 720p@30FPS

Vince gsmarena.com - 4K@30fps, 1080p@30/60fps
qualcomm.com SD625

This is what your phone implements via the Camera2 API, which is abnormal, but it's common for certain brands not to provide a good Camera2 implementation. It's possible that the phone makes the other frame rate modes available but CameraX doesn't offer frame rate configuration yet. There's already an issue filed about adding frame rate configuration once CameraX supports it. It's likely your phone just doesn't make the modes available via Camera2.

@thestinger Add option to set it in the settings

That's planned, see https://github.com/GrapheneOS/Camera/issues/51. It can't be done yet since CameraX doesn't support it.


Shebang recipe just hangs on Windows 10

As per the title, when I do the following, nothing happens for several second and not even Ctrl-C does anything:
❯ cat justfile
foo:
    #!py
    print("hello, world")

❯ just -vvvvvvvvvvvvvvvvvvvvvvvvv foo
===> Running recipe `foo`...
#!py

print("hello, world")

I'm on the latest version of Windows 10 and willing to provide more info if necessary.

Thanks for reporting this!
-vvvvvvvvvvvvvvvvvvvvvvvvv

😂 I love it. There should be some sort of extra verbosity level that gets unlocked when you use more then ten -v flags.
This will be tricky for me to debug, because I don't have a windows box handy.
Some questions:

What version of just are you using?
Do recipes without a shebang work?
Can you try to run a recipe using powershell? I'm not sure, but I think this will look something like:

foo:
  #!powershell.exe -c
  Write-Host "Foo!"

Here you go, the set shell is important as otherwise just tries to use sh:
❯ cat justfile
set shell := ["pwsh.exe", "-c"]

foo:
    echo foo

bar:
    #!powershell.exe
    Write-Host "Bar!"

baz:
    #!powershell.exe -c
    Write-Host "Baz!"

ham:
    #!py
    print("ham")

spam:
    #!py -c
    print("spam")

❯ just foo
echo foo
foo

❯ just bar
Bar!

❯ just baz
Baz!

❯ just spam
# hangs

❯ just ham
# hangs

❯ just --version
just 0.10.2

Wow, very weird. I don't know if I can debug this, since I don't have a windows dev box, and tests like this are passing on CI, which runs on Windows. Hopefully someone has some insight. Since I have limited resources to debug Windows issues, it would be awesome if a Just user who used Windows wanted to volunteer to help debug.

I'd be willing to look into this. I'm on Windows, obviously, and I know Rust. Could you please give me a few pointers on where to look in the code base?

I think the first question to answer is: Where is the code hanging? Check out Recipe::run, which is what executes a recipe. Shebangs recipes have different code paths from linewise recipes (so-called because they're executed one line at a time), so I assume it's something there. You could try adding some logging statements.
In fact, logging statements would be helpful as a permanent addition, so you could add logging statements that help you figure out what's going on, and then make a PR to add those to the codebase.

And by the way, thank you for taking a look!

Thank you for the pointers! I'll have a look once I have a chance. I'm very busy lately, so it may take me a bit.

You bet, and no worries at all!

I think maybe it is not a just issue. py is the Python launcher which can be used if you want to use different Pythons or just run the latest available.
See this justfile:
set shell := ["powershell.exe", "-c"]

works1:
    #!python
    print("IT WORKS")

works2:
    #!python.exe
    print("IT WORKS")

works3:
    #!python38
    print("IT WORKS")

works4:
    #!python38.exe
    print("IT WORKS")

works5:
    #!C:\Users\MFarkas\scoop\apps\python39\current\python.exe
    print("IT WORKS")

wrong-use:
    #!python -c
    print("Syntax error")

hangs1:
    #!py
    print("It hangs")

hangs2:
    #!py.exe
    print("It hangs")

hangs3:
    #!C:\Users\MFarkas\scoop\apps\python39\current\py.exe
    print("It hangs")

Interesting! I'll close this issue, since it seems like it isn't just-specific. Is it possible that py.exe is printing something to stderr or stdout that's getting swallowed, like a prompt or something?

I do not know exactly the issue, but it works well using python instead of py, so I think it can be closed.
Interestingly this works:
set shell := ["powershell.exe", "-c"]

works:
    py -c "print('Hello')"

This also works:
set shell := ["py.exe", "-c"]

works:
    print('Hello')

But this hangs:
set shell := ["powershell.exe", "-c"]

hangs:
    #!py
    print("It hangs")


chore: add support for Debian and Ubuntu packages

Fixes #103
Execution example: https://github.com/kou/arrow-adbc/actions/runs/3726461219
This also release related codes but they aren't tested yet.
I want to work on it as follow-up tasks.
NOTE: We need to add some files in ci/linux-packages/debian/ to
rat_exclude_files.txt in https://github.com/apache/arrow-adbc/pull/174 .
It's also a follow-up task.

By the way: do you want this to be in the first ADBC release? If so, I'll wait for things to settle before starting the vote

do you want this to be in the first ADBC release?

I don't think that it's a blocker for the first ADBC release. It's just a nice-to-have. If we have ADBC deb/rpm before the next Apache Arrow release, I'll enable https://github.com/apache/arrow/pull/14082 in Apache Arrow deb/rpm.

Ok! I'll take one final look over things this week, then start the vote (aiming for Thursday/Friday).

Can we run packaging.yml workflow in pull request to detect failures soon?
Is it heavy to run for each pull request?

Thanks!


this.Parser.prototype is undefined in astro project

I am using it in the astro project, and when I start the project I get the error Cannot read properties of undefined (reading 'blockTokenizers'). When I debugged the code I found that this.Parser.prototype was undefined in the plugin code.
astro version is "astro": "^3.1.0"
project astro.config.mjs code:
import { defineConfig } from 'astro/config';
import react from "@astrojs/react";

import remarkToc from 'remark-toc';
import remarkAdmonitions from 'remark-admonitions';
import remarkA11yEmoji from '@fec/remark-a11y-emoji';
import remarkCodeTitle from 'remark-code-title';
import remarkCodeBlocks from 'remark-code-blocks'

import mdx from "@astrojs/mdx";

// https://astro.build/config
export default defineConfig({
  markdown: {
    remarkPlugins: [
      remarkToc,
      remarkAdmonitions,
      remarkA11yEmoji,
      remarkCodeTitle,
      remarkCodeBlocks,
    ],
  },
  integrations: [react(), mdx()]
});

I don't know if it's a configuration problem or something went wrong.
**Expectation: **Works on astro project

use remark-directive

@LuHugo Hi! Can you be more specific? I'm having the same problem.:(

@LuHugo Hi! Can you be more specific? I'm having the same problem.:(

Because I used the remark-admonitions plug-in which caused the error, and then I replaced it with remark-directive and got the function I needed. I didn't delve too deeply into the reason.


Failed to connect to Flask Server even though backend doesn't terminate

Describe the bug
Error Log:
"Failed to connect to the flask server, please restart it.
If you think this was a bug, please head over to our GitHub issue tracker, check if someone has already reported the issue, and if not, create a new issue. Please include the full log, all settings that you entered, and a screenshot of this page."

Steps/commands/code to Reproduce
docker run -p 3000:3000 -it -v /var/run/docker.sock:/var/run/docker.sock \
  -e OPENAI_API_KEY="sk-xxxxx" \
  sweagent/swe-agent-run:latest bash start_web_ui.sh \
  python run.py --image_name=sweagent/swe-agent:latest \
  --model_name gpt4 \
  --data_path https://github.com/pvlib/pvlib-python/issues/1603 \
  --config_file config/default_from_url.yaml  --skip_existing=False

Error message/results
Failed to connect to the flask server, please restart it.
If you think this was a bug, please head over to [our GitHub issue tracker](https://github.com/princeton-nlp/swe-agent/issues), check if someone has already reported the issue, and if not, create a new issue. Please include the full log, all settings that you entered, and a screenshot of this page.

System Information
MacOS M1
Checklist

[X] I'm running with the latest docker container/on the latest development version
[X] I've searched the other issues for a duplicate
[X] I have copied the full command/code that I ran (as text, not as screenshot!)
[X] If applicable: I have copied the full log file/error message that was the result (as text, not as screenshot!)
[X] I have enclosed code/log messages in triple backticks (docs) and clicked "Preview" to make sure it's displayed correctly.

Hi @SashvDave, thanks for the feedback.
First, you only need
docker run -p 3000:3000 -it -v /var/run/docker.sock:/var/run/docker.sock \
  -e OPENAI_API_KEY="sk-xxxxx" \
  sweagent/swe-agent-run:latest bash start_web_ui.sh

(not the python run.py etc.).
Second, what was the output of the docker command? Unfortunately I don't think I can help without it. It should have information about why the backend failed.

Hey @klieret, thank you for the help!
The following is what I receive in my terminal when I run the docker command:
(env) sashvdave@Sashvs-MacBook-Pro SWE-agent % docker run -p 3000:3000 -it -v /var/run/docker.sock:/var/run/docker.sock \
   -e OPENAI_API_KEY="sk-xxx" \
  sweagent/swe-agent-run:latest bash start_web_ui.sh

up to date, audited 1819 packages in 4s

349 packages are looking for funding
  run `npm fund` for details

8 vulnerabilities (2 moderate, 6 high)

To address all issues (including breaking changes), run:
  npm audit fix --force

Run `npm audit` for details.

                        -------------

__/\\\\\\\\\\\\\____/\\\\____________/\\\\____/\\\\\\\\\_____
 _\/\\\/////////\\\_\/\\\\\\________/\\\\\\__/\\\///////\\\___
  _\/\\\_______\/\\\_\/\\\//\\\____/\\\//\\\_\///______\//\\\__
   _\/\\\\\\\\\\\\\/__\/\\\\///\\\/\\\/_\/\\\___________/\\\/___
    _\/\\\/////////____\/\\\__\///\\\/___\/\\\________/\\\//_____
     _\/\\\_____________\/\\\____\///_____\/\\\_____/\\\//________
      _\/\\\_____________\/\\\_____________\/\\\___/\\\/___________
       _\/\\\_____________\/\\\_____________\/\\\__/\\\\\\\\\\\\\\\_
        _\///______________\///______________\///__\///////////////__

                          Runtime Edition

        PM2 is a Production Process Manager for Node.js applications
                     with a built-in Load Balancer.

                Start and Daemonize any application:
                $ pm2 start app.js

                Load Balance 4 instances of api.js:
                $ pm2 start api.js -i 4

                Monitor in production:
                $ pm2 monitor

                Make pm2 auto-boot at server restart:
                $ pm2 startup

                To go further checkout:
                http://pm2.io/

                        -------------

[PM2] Spawning PM2 daemon with pm2_home=/root/.pm2
[PM2] PM2 Successfully daemonized
[PM2] Starting /usr/bin/npm in fork_mode (1 instance)
[PM2] Done.
┌────┬────────────────────┬──────────┬──────┬───────────┬──────────┬──────────┐
│ id │ name               │ mode     │ ↺    │ status    │ cpu      │ memory   │
├────┼────────────────────┼──────────┼──────┼───────────┼──────────┼──────────┤
│ 0  │ swe-agent          │ fork     │ 0    │ online    │ 0%       │ 23.4mb   │
└────┴────────────────────┴──────────┴──────┴───────────┴──────────┴──────────┘
* If you are running on your own machine, then a browser window
  should have already opened. If not, wait a few more seconds, then
  open your browser at http://localhost:3000
* If you are running in github codespaces, please click the popup
  that offers to forward port 3000 (not 8000!).
  Missed it? Find more information at
  https://princeton-nlp.github.io/SWE-agent/installation/codespaces#running-the-web-ui
* Something went wrong? Please check
  web_api.log for error messages!

I get no error messages in my web_api.log

Hmmm. Perhaps it's actually running but the web interface is confused... I've had this a few times and just had to wait for a few seconds... But perhaps I should revisit this and make it more stable.
The best way is to set up swe-agent from source and then manually start both parts of the web server (see here)

Hmm, I tried starting both parts of the web server individually, still getting the same issue - i wait for a few seconds and it still doesn't go away. Here's the output I receive in my terminal:
Proxy error: Could not proxy request /socket.io/?EIO=4&transport=polling&t=O_88m_u from localhost:3000 to http://localhost:8000.
See https://nodejs.org/api/errors.html#errors_common_system_errors for more information (ECONNREFUSED).

I checked out the link but it just provides the error code, not the solution. tried a few solutions from stack overflow but no bueno

Were you starting them from a development install or also with the docker container?
Also, was there any more output?
For example, if your port 8000 is taken, the flask backend will serve somewhere else and the frontend doesn't know about this...

Let me perhaps add a default route to the backend so you can check in your web server if it running.

Hey, sorry I missed your earlier message.
Yes, I am starting from a development install, not with the docker container.
No more output, that was it.

Hmm, I may have found the issue:
It doesn't log any issues when I open the server, but I navigated to the URL and it seems to have an issue. Pasted below
2024-06-01 11:38:32,893 - werkzeug - INFO - WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://<IP_ADDRESS>:8000
2024-06-01 11:38:32,893 - werkzeug - INFO - Press CTRL+C to quit
2024-06-01 11:38:32,894 - werkzeug - INFO -  * Restarting with stat
2024-06-01 11:38:34,529 - werkzeug - WARNING -  * Debugger is active!
2024-06-01 11:38:34,540 - werkzeug - INFO -  * Debugger PIN: 128-207-556
2024-06-01 11:38:41,670 - werkzeug - INFO - <IP_ADDRESS> - - [01/Jun/2024 11:38:41] "GET / HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/Users/sashvdave/SWE-agent/env/lib/python3.12/site-packages/flask/app.py", line 1498, in __call__
    return self.wsgi_app(environ, start_response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sashvdave/SWE-agent/env/lib/python3.12/site-packages/flask_socketio/__init__.py", line 43, in __call__
    return super(_SocketIOMiddleware, self).__call__(environ,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sashvdave/SWE-agent/env/lib/python3.12/site-packages/engineio/middleware.py", line 74, in __call__
    return self.wsgi_app(environ, start_response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sashvdave/SWE-agent/env/lib/python3.12/site-packages/flask/app.py", line 1476, in wsgi_app
    response = self.handle_exception(e)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sashvdave/SWE-agent/env/lib/python3.12/site-packages/flask_cors/extension.py", line 178, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
                                                ^^^^^^^^^^^^^^^^^^^^
  File "/Users/sashvdave/SWE-agent/env/lib/python3.12/site-packages/flask/app.py", line 1473, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sashvdave/SWE-agent/env/lib/python3.12/site-packages/flask/app.py", line 882, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sashvdave/SWE-agent/env/lib/python3.12/site-packages/flask_cors/extension.py", line 178, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
                                                ^^^^^^^^^^^^^^^^^^^^
  File "/Users/sashvdave/SWE-agent/env/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sashvdave/SWE-agent/env/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sashvdave/SWE-agent/sweagent/api/server.py", line 102, in index
    return render_template("index.html")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sashvdave/SWE-agent/env/lib/python3.12/site-packages/flask/templating.py", line 149, in render_template
    template = app.jinja_env.get_or_select_template(template_name_or_list)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sashvdave/SWE-agent/env/lib/python3.12/site-packages/jinja2/environment.py", line 1084, in get_or_select_template
    return self.get_template(template_name_or_list, parent, globals)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sashvdave/SWE-agent/env/lib/python3.12/site-packages/jinja2/environment.py", line 1013, in get_template
    return self._load_template(name, globals)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sashvdave/SWE-agent/env/lib/python3.12/site-packages/jinja2/environment.py", line 972, in _load_template
    template = self.loader.load(self, name, self.make_globals(globals))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sashvdave/SWE-agent/env/lib/python3.12/site-packages/jinja2/loaders.py", line 126, in load
    source, filename, uptodate = self.get_source(environment, name)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sashvdave/SWE-agent/env/lib/python3.12/site-packages/flask/templating.py", line 65, in get_source
    return self._get_source_fast(environment, template)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sashvdave/SWE-agent/env/lib/python3.12/site-packages/flask/templating.py", line 99, in _get_source_fast
    raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: index.html

I've also been getting tripped up by Flask server startup and existing processes when using start_web_ui.sh; this might help avoid some confusion: https://github.com/princeton-nlp/SWE-agent/pull/479

It doesn't log any issues when I open the server, but I navigated to the URL and it seems to have an issue. Pasted below

Yes, that's to be expected, currently we only serve the API on port 8000, not a website. But I've opened https://github.com/princeton-nlp/SWE-agent/issues/490 to do that for debugging purposes (trying to get this done today or tmr.)
Just to double check, you are opening your browser at http://localhost:3000/, right? (not 8000).
While doing that, could you check that the backend is running by checking
lsof -i :8000

(this checks for anything serving port 8000. it should show the flask server).

Hey, I'm facing the same issue.
Also tried running the service and frontend individually outside of docker, no luck.

Strange, strange. Let me make it a priority to get to the bottom of this.
Possible solutions:

https://stackoverflow.com/a/63231052/ (use 127...)
https://stackoverflow.com/a/57532447/ (trailing slash)
https://stackoverflow.com/a/46574808/ (--ignore-client)
https://stackoverflow.com/a/73739848/ (change axio configuration)
https://create-react-app.dev/docs/proxying-api-requests-in-development/#configuring-the-proxy-manually

Let me attempt the first two for now

yup I just solved it by using http://<IP_ADDRESS>:8000 as the proxy in package.json.
Thanks!


The sdk dependency in the gradle file has to be updated

The dependency to the facebook sdk is still 11.1.0, but has to be updated to 12.1.0, according to facebook.

Thanks for issue. We can use v3.4.0.

Thanks for issue. We can use v3.4.0.

Hi @rdlabo when you will release 3.4.0 to npm?


Deprecation Trial Not Working

Hi Team, I have had some troubles enabling this deprecation trial so I wanted to check if I was doing anything wrong.. I have successfully added my origin-trial meta tag, which looks like tag looks like  to my top-level page. The token is the right token for my domain. When I go to the Application tab in Chrome DevTools, I still see the storage being partitioned by top-level site. Any ideas what might be going wrong?

In addition, I also registered my token to work in a third party context and tried to add the JavaScript to add the meta tag to the page so that when it gets iframed it will request to opt in to the deprecation trial. In this case, I also successfully inserted the origin-trial meta tag but I still see the storage getting partitioned.

@Octophi would you mind pasting your test OT token here? (It's just a base64 encoded object - nothing super secret about it). I'm curious if you selected the subdomain matching checkbox.
Also, here's a pretty simple example using sessionStorage, but the particular storage interface shouldn't matter:
https://rogue-lace-join.glitch.me/
See also https://miketaylr.com/misc/3pspdt.js and view-source:https://miketaylr.com/misc/sptest.html to understand the setup.
The basic premise is that you should inject the 3P OT token in the top-level frame's head before creating and injecting any 3P iframes.

Sure, my token is AysPnqWJeE+bXqBmzsgbertfahuz7WDiHl03RGtRm/e5Kw309x5CWqk0/CQSpn1sRxV9SNxioru5KZNfu3jaFwgAAACOeyJvcmlnaW4iOiJodHRwczovL2ZvcmNlLmNvbTo0NDMiLCJmZWF0dXJlIjoiRGlzYWJsZVRoaXJkUGFydHlTdG9yYWdlUGFydGl0aW9uaW5nIiwiZXhwaXJ5IjoxNzI1NDA3OTk5LCJpc1N1YmRvbWFpbiI6dHJ1ZSwiaXNUaGlyZFBhcnR5Ijp0cnVlfQ==
I can see it inserted into a meta tag on the top-level page but it doesn't seem to be stopping the partitioning from occurring.

Also, are there any additional complications if we have an iframe within an iframe within a page?

Let me tag in @arichiv on the question of iframes within iframes... can you describe the setup a little bit more?

I have an endpoint which I control, which iframes in another endpoint which I control, which I then use to iframe in a third-party endpoint which I do not control. I want to disable storage partitioning and have been trying to do so by adding the meta tag to the HTML for the top-level site. However, it doesn't appear to be working at the moment.

OK, so you have a.com which frames b.com which frames c.com and you want unpartitioned storage for c.com, correct?

Yup, exactly

Can you describe the specific origins that are embedding each other? What level is https://force.com:443?

I ask because your token is for https://force.com:443 and is a Third-Party token. There are two options if we consider the example of a.com embedding b.com
(1) a.com can include a first-party token for a.com, this will unpartition all subframes on any origins
(2) a.com can include a third-party token for b.com, this will unpartition all subframes for b.com only

Also, it's important to note that in case (2), the token must be injected via a script from b.com. a.com cannot inject it itself.

Ok, got it, so we cannot use the same token for both a third-party context and a first-party context.
Currently, I have a page https://joey-dev-ed--c.develop.vf.localhost.sfdcdev.force.com:6101/apex/CanvasPage which iframes in a page on the salesforce.com registrable domain, which then iframes in a page in herokuapp.com. I am inserting a meta tag on the top-level page which looks like

I believe this is a first-party token associated with force.com. It still doesn't appear to be working, though. Am I missing something?

That should be possible, yes.

Ok after some more debugging I figured out why it wasn't working - my dev environment was not running https on the default port 443, so the token wasn't associated with the correct origin. Thanks y'all for answering all my questions so promptly.

Glad to hear you got it sorted!


NDRS-453 - carry over finalized deploys from joiner to validator phase

This PR adds the linear_chain as an argument to the DeployBuffer's constructor, allowing it to load the finalized deploys from storage.

@Fraser999 and I had a call where we discussed some changes (move method to storage, call SmallVec::from), and those are my last commit.

@Fraser999 and I had a call where we discussed some changes (move method to storage, call SmallVec::from), and those are my last commit.

Yep, looks OK. Thanks.

bors r+


fix: http transformer e2e test flakiness

Description of changes:

fix flaky http-transformer-e2e tests by deploying a modified jsonserver to lambda+api gateway with CDK
enhance all tests with stricter checking on responses

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report

Merging #3978 into master will decrease coverage by 1.12%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3978      +/-   ##
==========================================
- Coverage   60.31%   59.19%   -1.13%
==========================================
  Files         300      312      +12
  Lines       13776    14037     +261
  Branches     2774     2651     -123
==========================================
  Hits         8309     8309
- Misses       5031     5313     +282
+ Partials      436      415      -21

Impacted Files
Coverage Δ

packages/amplify-util-mock/src/api/api.ts
0.00% <0.00%> (ø)

packages/graphql-mapping-template/src/print.ts
35.71% <0.00%> (ø)

...ages/amplify-appsync-simulator/src/server/index.ts
64.28% <0.00%> (ø)

...ges/amplify-util-mock/src/CFNParser/stack/index.ts
88.09% <0.00%> (ø)

...ges/graphql-transformer-core/src/util/fileUtils.ts
11.76% <0.00%> (ø)

...es/amplify-util-mock/src/api/resolver-overrides.ts
0.00% <0.00%> (ø)

...es/graphql-transformer-core/src/stripDirectives.ts
35.29% <0.00%> (ø)

.../amplify-util-mock/src/utils/lambda/loadMinimal.ts
0.00% <0.00%> (ø)

...amplify-appsync-simulator/src/server/operations.ts
28.12% <0.00%> (ø)

...graphql-transformer-core/src/TransformerContext.ts
26.20% <0.00%> (ø)

... and 22 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 60559c5...c8e9675. Read the comment docs.


SELinux: Validate platform gcp in enforcing mode

Validate platform gcp in enforcing mode
General
After all packaging and pipeline related stuff regarding SELinux got merged, we need to perform new platforms tests. This issue covers the integration tests with SELinux in enforcing mode for the platform type gcp.
Do not start with this tests before these ones are done:

#1200
#1201
#1202

Task
Important: Before performing the platform tests make sure, that your SELinux is running in enforcing mode.

Run the platforms related tests on gcp
Log and add all avc messages to this issue (or extend the gardenlinux-selinux-module by providing a PR)
Rerun the tests after adding the rules

Additional hyperscaler related rules are missing. Providing a PR to fix this, asap.

This get added within my PR https://gitlab.com/gardenlinux/gardenlinux-package-gardenlinux-selinux-module/-/merge_requests/10
Reviewer: @nanory

Just released version gardenlinux-selinux-module-1.0.7 for GCP hyperscaler (changelog).
This needs retesting with 1.0.7.

Validated on GCP:
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             default
Current mode:                   enforcing
Mode from config file:          enforcing
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Memory protection checking:     actual (secure)
Max kernel policy version:      33

Process contexts:
Current context:                unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
Init context:                   system_u:system_r:init_t:s0

File contexts:
Controlling terminal:           unconfined_u:object_r:user_devpts_t:s0
/etc/passwd                     system_u:object_r:etc_t:s0
/etc/shadow                     system_u:object_r:shadow_t:s0
/bin/bash                       system_u:object_r:shell_exec_t:s0
/bin/login                      system_u:object_r:login_exec_t:s0
/bin/sh                         system_u:object_r:bin_t:s0 -> system_u:object_r:shell_exec_t:s0
/sbin/agetty                    system_u:object_r:getty_exec_t:s0
/sbin/init                      system_u:object_r:bin_t:s0 -> system_u:object_r:init_exec_t:s0
/usr/sbin/sshd                  system_u:object_r:sshd_exec_t:s0

Found a minor issue for systemd-notify; will fix this tomorrow.

Just release version 1.0.8 of gardenlinux-selinux-module that fixes this minor issue.


Update the challenge registry API spec

The project that contains the API spec of the challenge registry has not been updated in a while.

[x] #739
[x] #743
[x] #745

Future work

Review how the documentation site of the API spec is generated

Done


Change vendor from Eclipse Foundation to Eclipse Adoptium

As discussed in the marketing group meeting and in the Adoptium Workgroup Steering Committee meetings this week. This should remove ambiguity to end users by clarifying that this is the "Adoptium" build.
Signed-off-by: Stewart X Addison<EMAIL_ADDRESS>

LGTM

Could you please tell where exactly vendor will be changed - in API, in release file, in default installation paths, registry paths?

IIRC at this stage it is only:

The vendor property in the json meta data - OpenJDK17-jdk_x64_windows_hotspot_17_35.msi.json
java.vendor Java system property

We're looking at whether we need to change the installer paths as well.  More on this shortly

@karianna Thanks!


[FEATURE]Support Kafka event processing

Is your feature request related to a problem? Please describe.
We need to plug events streamed through Kafka to streaming algorithms so that upon arrival of Kafka messages, we can update train/predict using streaming algorithms. In particular, we are interested in using this library in a smart manufacturing context, to monitor and predict machine performance and downtimes.
Describe the solution you'd like
Provide some adaptor, that allows connecting to Kafka and stream the events to scikit-multiflow
Describe alternatives you've considered
We are developing a solution in this fork, where we reframe scikit-multiflow to process events on demand instead of actively poll data sources for new events.
Additional context
We have industry requirements for streaming algorithms, but we experience an adoption barrier on the lack of support of scikit-multiflow to certain widely used stream sources, such as Kafka; and the fact that the library was designed to pull events, not to listen for them. Providing such adaptors would cover a wide range of new scenarios and ease the adoption of the library.

Hello,
We have similar need, but follow another path: We use the model implementations from scikit-multiflow and integrate them directly in our framework that already supports kafka as a source.
The on-going work is available here and works with vanilla scikit-multiflow:
https://github.com/maki-nage/rxsci-multiflow
Feeding data from kafka is done with maki nage: makinage.org
It looks like several streaming/reactive frameworks want to integrate with scikit-multiflow (cf #106). It would be nice to make this possible without needing specific integration in scikit-multiflow. Using the different models directly is quite easy, but not the evaluators. I think that splitting the evaluator functions into smaller parts would make it: This would not change the existing scikit-multiflow evaluators API/behavior, and allow other frameworks to reuse most of this logic while only replacing the data feed part.

@MainRo thank you for sharing! We agree regarding the evaluators behaviour. In our implementation we refactored them, providing different evaluation strategies.

interesting, where did you move them ? The evaluation module is empty on your master.

We have that in the skmultiflow/data/observer package. We thought about it as event observers with different trigger policies.

+1 for this feature. This will be a great feature since Kafka is widely used in many companies.
We can create a new issue to discuss the design of the new API of the evaluator in order to support other sources like Kafka, MQTT, or similar service.
I think it's better to work on the same project and add new features rather than create new forks.

@garawalid we submitted a proposal in #278 which provides a working implementation. Seems the gh-actions are not detecting the package structure changed, thus failing the tests, which all run if executed locally.
The way we developed it, it is ready to support other sources such as RabbitMQ and others as well. I would suggest continuing the conversation on this issue, to avoid splitting the case over multiple issues.


Ler QR Mas nao libera acesso

(node: 1413) UnhandledPromiseRejectionkarning: Error: Authenticating via JSON session is not supported for MultiDevice-enabled Wh
atsApp accounts.
at LegacySessionAuth.getAuthEventPayload (/home/whats/uhat icket/backend/node_modules/uhatsapp-web. js/src/authStrategies/Lega
cySessionfuth. js:57:24)
at runMicrotasks ()
at processTicksAndRejections (internal/pracess/task_queues. js )
at async Client. initialize (/home/uhats/uhat icket/backend/node_modules/whatsapp-web. js/src/Client . js:217:34)
(node: 1413) UnhandledPromiseReject ionWarning: Unhandled promise rejection. This error originated either by throwing inside of an
async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node prac
ess on unhandled promise rejection, use the CLI flag ‘--unhandled-rejections=strict’ (see https://nodejs.org/api/cli.html#cli_un
handled_rejections_mode). (rejection id: 1)
(node:1413) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that ar
e not handled will terminate the Node. js process with a non-zero exit cade.

Mesmo ocorreu aqui.

try changing wbot.ts and package.json for multi device support
wbot.ts and package.json
https://github.com/diazzaid/whaticket-1/blob/master/backend/src/libs/wbot.ts
https://github.com/diazzaid/whaticket-1/blob/master/backend/package.json
npm run build
and ignore if there is an error message
npm start

Fiz os procedimentos porem problema persiste!

Pelo Jeiro modo Beta virou definitivo para whatsapp, creio q a biblioteca whatsapp-web.js n suporta multi-devices

tentar desativar a opção de multi-devices no whatsapp app. estava ocorrendo o mesmo erro para mim.


Assistance needed: Unable to reclaim space in Kubernetes cluster on corporate macOS machine

hi Sir/Madam, Need help resolve or recalim space from the manifests created and not releasing even with no pods running. Does this mean need underlying root privileges reqd. note: this is an corp administered work m/c. Pls guide to resolve if its a k8s/kind/macos underlying. Hard to tell. If kind and k8s can create, it can reclaim not happening. Pls advice.
Topic Ref:
https://kubernetes.io/docs/tasks/run-application/run-replicated-stateful-application/#scaling-the-number-of-replicas
=====================================================================================
Darwin XXXXXXXXXXXX 23.2.0 Darwin Kernel Version 23.2.0: Wed Nov 15 21:54:10 PST 2023; root:xnu-10002.61.3~2/RELEASE_X86_64 x86_64
Name: kube-apiserver-cert-control-plane
Namespace: kube-system
Priority:<PHONE_NUMBER>
Priority Class Name: system-node-critical
Node: cert-control-plane/<IP_ADDRESS>
Start Time: Wed, 20 Dec 2023 21:14:02 -0500
Labels: component=kube-apiserver
tier=control-plane
Annotations: kubeadm.kubernetes.io/kube-apiserver.advertise-address.endpoint: <IP_ADDRESS>:6443
kubernetes.io/config.hash: b5fdc31de1e0bdcf64496afe6ba25407
kubernetes.io/config.mirror: b5fdc31de1e0bdcf64496afe6ba25407
kubernetes.io/config.seen: ages 2023-12-20T19:06:50.347173259Z
kubernetes.io/config.source: file
Status: Running
SeccompProfile: RuntimeDefault
IP: <IP_ADDRESS>
IPs:
IP: <IP_ADDRESS>
Controlled By: Node/cert-control-plane
Containers:
kube-apiserver:
Container ID:  containerd://007aa3e658392c9f87140627c13b1d6a6e263fe4388a2af2d0477103f5002a08
Image: registry.k8s.io/kube-apiserver:v1.27.3
NAMESPACE    NAME             STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
mysql-sts    mysql-pv-claim   Bound    mysql-pv-volume                            2Gi        RWO            manual         16h
mysql2-sts   data-mysql-0     Bound    pvc-0b2e495c-3ea2-4398-aba7-71ba1324d161   10Gi       RWO            standard       6h38m
mysql2-sts   data-mysql-1     Bound    pvc-c893b33d-87ce-482c-a114-6806b3d51bf5   10Gi       RWO            standard       6h35m
mysql2-sts   data-mysql-2     Bound    pvc-eaeee3d8-4cd7-4190-a5ae-1cca47b1cdfe   10Gi       RWO            standard       6h35m
mysql2-sts   data-mysql-3     Bound    pvc-f3cc46c2-254d-4e9e-88d7-e29978333c75   10Gi       RWO            standard       103m
mysql2-sts   data-mysql-4     Bound    pvc-7adf8b36-ff75-4049-8b88-c6e36e0145d9   10Gi       RWO            standard       103m
sh-3.2$ k get pv -A
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS        CLAIM                      STORAGECLASS   REASON   AGE
mysql-pv-volume                            2Gi        RWO            Retain           Bound         mysql-sts/mysql-pv-claim   manual                  16h
pvc-0b2e495c-3ea2-4398-aba7-71ba1324d161   10Gi       RWO            Delete           Bound         mysql2-sts/data-mysql-0    standard                6h38m
pvc-7adf8b36-ff75-4049-8b88-c6e36e0145d9   10Gi       RWO            Delete           Terminating   mysql2-sts/data-mysql-4    standard                103m
pvc-c893b33d-87ce-482c-a114-6806b3d51bf5   10Gi       RWO            Delete           Bound         mysql2-sts/data-mysql-1    standard                6h36m
pvc-eaeee3d8-4cd7-4190-a5ae-1cca47b1cdfe   10Gi       RWO            Delete           Bound         mysql2-sts/data-mysql-2    standard                6h35m
pvc-f3cc46c2-254d-4e9e-88d7-e29978333c75   10Gi       RWO            Delete           Terminating   mysql2-sts/data-mysql-3    standard                104m
sh-3.2$ k delete pv pvc-7adf8b36-ff75-4049-8b88-c6e36e0145d9 -n mysql2-sts
Warning: deleting cluster-scoped resources, not scoped to the provided namespace
persistentvolume "pvc-7adf8b36-ff75-4049-8b88-c6e36e0145d9" deleted
pvc-del-kind.txt

Page mentioned in the mentioned issue: https://kubernetes.io/docs/tasks/run-application/run-replicated-stateful-application
/language en

/retitle Assistance needed: Unable to reclaim space in Kubernetes cluster on corporate macOS machine

(Looks like a request for help in its current form)
/kind support

@suriya786  Thank you for reaching out and seeking assistance. We appreciate your query, but the GitHub issue tracker may not be the most suitable platform for obtaining advice on Kubernetes. We recommend exploring alternative resources such as the Kubernetes Slack community (http://slack.k8s.io/) or the Discuss Kubernetes forum (https://discuss.kubernetes.io/) for the support you need.

/transfer kubernetes


Singsong- allsang; song-fremføring type to bmm

New subclip type "Sing-along" , export them as "Singsong" for BMM

This is added in MB.


GSD Request

--- GSD JSON ---
{
  "vendor_name": "Linux",
  "product_name": "Kernel",
  "product_version": "versions from v4.0 to before v5.4.170",
  "vulnerability_type": "unspecified",
  "affected_component": "unspecified",
  "attack_vector": "unspecified",
  "impact": "unspecified",
  "credit": "",
  "references": [
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=5e33f6fdf735cda1d4580fe6f1878da05718fe73",
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=240fc586e83d645912accce081a48aa63a45f6ee"
  ],
  "extended_references": [
    {
      "type": "commit",
      "value": "5e33f6fdf735cda1d4580fe6f1878da05718fe73",
      "note": "introduced"
    },
    {
      "type": "commit",
      "value": "240fc586e83d645912accce081a48aa63a45f6ee",
      "note": "fixed"
    }
  ],
  "reporter": "joshbressers",
  "reporter_id": 1692786,
  "notes": "",
  "description": "usb: gadget: f_fs: Clear ffs_eventfd in ffs_data_clear.\n\nThis is an automated ID intended to aid in discovery of potential security vulnerabilities. The actual impact and attack plausibility have not yet been proven.\nThis ID is fixed in Linux Kernel version v5.4.170 by commit 240fc586e83d645912accce081a48aa63a45f6ee, it was introduced in version v4.0 by commit 5e33f6fdf735cda1d4580fe6f1878da05718fe73. For more details please see the references link."
}
--- GSD JSON ---

This issue has been assigned GSD-2022-1000033


homer: init 24.11.1

fix #152343
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

ping @tecosaur how did you come up with the patch? it does not build for me.

Result of nixpkgs-review pr 354293 run on x86_64-linux 1

1 package built:

homer

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 354293

x86_64-linux

:white_check_mark: 1 package built:

homer

aarch64-linux

:white_check_mark: 1 package built:

homer

LGTM.

@PatrickDaG i just updated the package again to its latest version.

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 354293

x86_64-linux

:white_check_mark: 1 package built:

homer


Use like-bazel.cmake to generate protobuf intermediate files

I am importing https://github.com/wangkuiyi/fluid/blob/develop/tools/cmake/like-bazel.cmake, which is a simplification of https://github.com/PaddlePaddle/Paddle/blob/develop/cmake/generic.cmake and has been verified working.


greatly reduce bandwith to openstreet

returns a lot less unneeded node data, especially on freeways. tested

Please close this, looks like speedlimitd.py is not even being used they're being handled in mapd.


Add API for device support

Based on the discussion in gh-39.

The summary of feedback received on this PR so far was:

given how different current libraries are w.r.t. device handling, let's keep it minimal and focus only on the use case of dealing with devices in library code.
That use case does not require a device object in the API standard or identifying specific physical/logical devices.
So remove the object, and the string representation that allows identifying a specific device in a portable way across libraries.
Keep only:

the .device attribute, which gives back a device object that only needs to be able to compare (__eq/neq__) itself with other devices from the same library
the device= keyword for creation functions.
a method named to_device on the array object, to move arrays.

PR updated. And asked about the link to the device_type:device_id on https://github.com/data-apis/consortium-feedback/issues/1. I suspect that the device_id happens to always work there, because libraries will typically use the same ordering as CUDA (as shown by, e.g., nvidia-smi) will give, but that it's technically not guaranteed.

I will note that it's kind of odd to have a .device attribute that returns some device object that itself is not part of the API. It will work, but it will make it harder to instantiate (it'll either be in a different namespace, or - for numpy - completely missing) and use in type annotations.

What is the behavior in symbolic execution model where devices may be set post hoc and may even change across multiple calls to the same function?  Also, is there some notion of symbolic devices ?
As a concrete example, does y = zeros([2], x.device), where x has not been placed on any device yet, imply a colocation directive for x and y ?

What is the behavior in symbolic execution model where devices may be set post hoc and may even change across multiple calls to the same function?

If the default of device=None is implementation-defined, then that will be completely compatible with symbolic execution right?

Also, is there some notion of symbolic devices ?

It may be good to mention explicitly that symbolic execution is a thing and that the current design must allow for it. Other than that, I think all the current version of this PR says is that a library will have a device object and that it must implement __eq__ and __neq__. That should be fine for symbolic devices too.
The text says "to a specific physical or logical device". That may mean different things to different people, "logical" and "symbolic" sound similar to me; happy to add "symbolic" explicitly though.

As a concrete example, does y = zeros([2], x.device), where x has not been placed on any device yet, imply a colocation directive for x and y ?

That co-location is current a recommendation, not a hard requirement, in the "semantics" section of this PR. I'd expect also a symbolic execution model to do co-location and raise otherwise, but I'm not sure if that's guaranteed or not (TFs placement policies are a little tricky to wrap my head around just from the docs).

The text says "to a specific physical or logical device". That may mean different things to different people, "logical" and "symbolic" sound similar to me; happy to add "symbolic" explicitly though.

I searched the TensorFlow more thoroughly, and there's no such thing as a "symbolic device" there. I'm pretty sure it's LogicalDevice instead, so the current text in this PR seems clear to me.

No more comments in the last couple of weeks, so I'll go ahead and merge this PR so it's available in the published html docs. If there are any more comments, please add them here or open a new issues.

No more comments in the last couple of weeks, so I'll go ahead and merge this PR so it's available in the published html docs. If there are any more comments, please add them here or open a new issues.

LGTM. Agreed if there's any concern we can revisit 🙂

LGTM. Agreed if there's any concern we can revisit 🙂

As a concrete example, does y = zeros([2], x.device), where x has not been placed on any device yet, imply a colocation directive for x and y ?

That co-location is current a recommendation, not a hard requirement, in the "semantics" section of this PR. I'd expect also a symbolic execution model to do co-location and raise otherwise, but I'm not sure if that's guaranteed or not (TFs placement policies are a little tricky to wrap my head around just from the docs).

Note that with symbolic execution, there is a "tracing" phase and an execution phase. If x.device is None at the time the graph is traced, then device=x.device is effectively ignored and colocation will not be done by default. If we want function execution to provide colocation, we need to put extra directives inside the traced graph. Then function runtime can respect those. But this will not be done by default.
A side effect is that there will be different behavior in Eager vs symbolic execution.  For Eager, x.device will be set by the time y is computed, hence colocation will be done, but function execution will ignore it if x.device is not specified at function tracing time. This will generally be the case, since code is mostly written to be agnostic of actual placement and the placement is often done at tf.function call time instead of tf.function tracing time.

As a concrete example, does y = zeros([2], x.device), where x has not been placed on any device yet, imply a colocation directive for x and y ?

That co-location is current a recommendation, not a hard requirement, in the "semantics" section of this PR. I'd expect also a symbolic execution model to do co-location and raise otherwise, but I'm not sure if that's guaranteed or not (TFs placement policies are a little tricky to wrap my head around just from the docs).

Note that with symbolic execution, there is a "tracing" phase and an execution phase. If x.device is None at the time the graph is traced, then device=x.device is effectively ignored and colocation will not be done by default. If we want function execution to provide colocation, we need to put extra directives inside the traced graph. Then function runtime can respect those. But this will not be done by default.
A side effect is that there will be different behavior in Eager vs symbolic execution.  For Eager, x.device will be set by the time y is computed, hence colocation will be done, but function execution will ignore it if x.device is not specified at function tracing time. This will generally be the case, since code is mostly written to be agnostic of actual placement and the placement is often done at tf.function call time instead of tf.function tracing time.

If we want function execution to provide colocation, we need to put extra directives inside the traced graph. Then function runtime can respect those. But this will not be done by default.

Just checking, are you saying that that won't be done now, if you simply add the device= keyword in the most naive way? Or you do not want to do this in TensorFlow by default?
It seems desirable to just always put those extra directives in I'd think, and not let eager/symbolic behavior diverge?

If we want function execution to provide colocation, we need to put extra directives inside the traced graph. Then function runtime can respect those. But this will not be done by default.

Just checking, are you saying that that won't be done now, if you simply add the device= keyword in the most naive way? Or you do not want to do this in TensorFlow by default?
It seems desirable to just always put those extra directives in I'd think, and not let eager/symbolic behavior diverge?

If we want function execution to provide colocation, we need to put extra directives inside the traced graph. Then function runtime can respect those. But this will not be done by default.

Just checking, are you saying that that won't be done now, if you simply add the device= keyword in the most naive way? Or you do not want to do this in TensorFlow by default?
It seems desirable to just always put those extra directives in I'd think, and not let eager/symbolic behavior diverge?

We will likely make this work best-effort in TensorFlow when we try to adhere to these new APIs. Given that this device placement seems like a recommendation instead of a requirement, it should be ok I think. This is a subtle implementation issue for frameworks implementing symbolic execution.

If we want function execution to provide colocation, we need to put extra directives inside the traced graph. Then function runtime can respect those. But this will not be done by default.

Just checking, are you saying that that won't be done now, if you simply add the device= keyword in the most naive way? Or you do not want to do this in TensorFlow by default?
It seems desirable to just always put those extra directives in I'd think, and not let eager/symbolic behavior diverge?

We will likely make this work best-effort in TensorFlow when we try to adhere to these new APIs. Given that this device placement seems like a recommendation instead of a requirement, it should be ok I think. This is a subtle implementation issue for frameworks implementing symbolic execution.


Feature: Replace click-outside-modal interaction with close icon

Type of feature
🍕 Feature
Current behavior

Suggested solution

Additional context
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Contributing Docs

[X] I agree to follow this project's Contribution Docs

Thanks for mentioning this pain point when you gave workspaces a go @KevinVandy. This should be fixed in our current sprint.


Remove mount dependency

review @jsingle  do not cd or export path with that directory if mount is not specified.  closes issue #449

:tada:


Events are in random order if they have the same timestamp

Can you please expand on this subject as I'm not completely following. Are you referring to the ui?

Yes in the UI.
After some more testing, it looks like it's random.
If i submit two events right after each other, the order in the UI is not always the same they were submitted in.
If i do a sleep of 1 ms between the two submit, the order is correct.

This could because they are submitted with the same exact date time. When it's created with the same exact date time it will show up in the order they were returned as from elastic search. Can you please confirm this by looking at the date on the two created events. What is it set to? Are they the same time?

I think they are the same. I can't see milliseconds, but if i insert a sleep for 1 millisesond then the order is correct.

Ahh. I didn't see the api link. I will try that and get back to you.

You can get your api key by logging in here.

The datetimes are exactly the same.

When we get the events back from elasticsearch we tell it to sort by date and nothing else for most views so it should be in the order it's received (but it's not guaranteed). I'm not sure what should happen here as it's by design.
@ejsmith thoughts?

Hmm... Not without some sequential id.

It's created with a semi sequential id but it's created in the order
they are added.Are the id's close?
On 4/9/15, Kasper Kirkegaard<EMAIL_ADDRESS>wrote:

Hmm... Not without some sequential id.

Reply to this email directly or view it on GitHub:
https://github.com/exceptionless/Exceptionless/issues/86#issuecomment-91348600

--
Thanks
-Blake Niemyjski

Do you have the id's that created it, I'm curious if they are sequential. Is there any chance you could put a unit test together that reproduces this and we could take a closer look. I would think that the events would be processed in the order they are sent in. If they are sent in via bulk, I'd think we'd process them in the order they exist in the json array.

@KasperLK any chance you could please take a look at my previous response.

I'm closing this as we will process events based on the order they are received as well as there timestamp.


[Breaking] Align all text-style related types to better match the use-cases of workspace

This PR aligns all text-style related types to better match the use-cases of workspace.
Related tickets

WRS-NUMBER

Screenshots

This PR should be merged into long-lived v2 branch, to fix the issue here we could create a non breaking union type


Add freopen

Add freopen to Sys::IO

Implemented.
https://github.com/yuki-kimoto/SPVM-Sys/commit/2de8410f2381060b10049b9b18ba0dc591bbf216


Code quality improvements, bugfixes, additional tests for spork/data

Added additional tests to /test/suite0021.janet , applying to spork/data/diff.
This revealed bugs in the way diff was handling larger, more deeply-nested structures.
While fixing bugs, simplified diff's logic which allowed eliminating several small "helper" functions. The eliminated function calls probably improve performance of diff, but I haven't benchmarked before/after.
All tests, including newly-added ones, now pass.

May be minor point but I think start-suite's argument is currently optional.  May be it's not necessary to add one in this case?
I tried running the tests with and without an argument and the output didn't seem different wrt the presence of this argument -- though I might have missed something (^^;
When reorganizing janet's tests, we made a similar change and I think it made it a bit easier for maintenance.  Possibly a similar thing holds here.

I was just making this one match all the others that are currently out there. I noticed it because the output from jpm test is inconsistent between all the other test suites and suite0021:

I don't really have an opinion either way on which way is preferable.  ¯_(ツ)_/¯

...Of course, now I notice that there's already tests out there organized by module instead of just being opaquely numbered. 😜

I noticed it because the output from jpm test is inconsistent between all the other test suites and suite0021

Ah right -- I guess I did miss the difference :)

there's already tests out there organized by module instead of just being opaquely numbered. 😜

I don't know if it would be seen as worth it, but I suppose we might be able to reorganize in a manner somewhat similar to what we did for Janet itself.
Though may be things are already gradually heading in that direction and it's another thing I failed to notice (^^;


feat: export UniStyle(View|Text|Image) types

Summary

I want to create some reusable Object and type them correctly
type Font = {
    [key: string]: UnistyleText
}

export const fonts: Font = {
    body: {
        fontSize: 16,
        fontWeight: 400,
        lineHeight: 24
    }
}

const stylesheet = createStyleSheet(theme => ({
  body: {
    ...fonts.body
  }
}));

It throws error if UnistyleText isn't used

Hey @jpudysz, if this change makes sense, can we get a quick release?
I am happy to take any better alternative solution.

@jpudysz Fixed the linting issue. Thank you.


Unknown shorthand flag

when trying to run gobuster in kali:
gobuster -u http://website.com -w wordlist
I get an error:
Error:unknown shorthand flag: 'u' in -u

Please read the README, the command line has changed. Thanks.


manually grafting a parent isn't updating the ancestry of children

Occasionally Ratatosk imports subtrees (eg a species grafted to a genus) but is unable to graft these to the main tree:

When a curator edits the parent (eg genus) and manually grafts it to the the main tree, the ancestry of the parent (eg genus) is updated

But the ancestry of the child (eg species) is not

This ancestry glitch is the underlying issue resulting in most instances of active taxa with inactive ancestors following things like taxon changes, so would be good to minimize the incidence of these ancestry glitches

I just tried replicating this in a spec and on staging with a new genus and species, but everything worked fine. Does this only happen for taxa that came in through Ratatosk or something? When you say "manually grafted" what do you mean, exactly? Like you go to taxa/edit and change the parent? Or some other way of grafting?

By manually grafting I mean go to taxa/edit and alter the parent
I just clicked 'search external names providers' with 'Parapilinurgus-masumotoi' on the staging ID form on Obs/show

and Ratatosk imported the subtree and failed to graft as I described:
https://staging.inaturalist.org/taxa/885736-Parapilinurgus-masumotoi
Try setting the parent of Parapilinurgus to Cetoniinae here https://staging.inaturalist.org/taxa/885736/edit

Try setting the parent of Parapilinurgus to Cetoniinae here https://staging.inaturalist.org/taxa/885736/edit

That's impossible. https://staging.inaturalist.org/taxa/885736/edit allows you to edit Parapilinurgus masumotoi (885736). Do you mean https://staging.inaturalist.org/taxa/885737/edit?

whoops sorry - I did mean https://staging.inaturalist.org/taxa/885737/edit
I haven't tried replicating it with manually created taxa. But all of the ancestry glitches I've found come from species (e.g. Parapilinurgus masumotoi) that were originally imported fro external providers. But that could just be that thats the only source of ungrafted subtrees

Ok, I set the parent of https://staging.inaturalist.org/taxa/885737-Parapilinurgus to https://staging.inaturalist.org/taxa/127588-Cetoniinae and https://staging.inaturalist.org/taxa/885736-Parapilinurgus-masumotoi now has its proper ancestry, so clearly I am still missing something.

interesting - thats not the behavior I've seen nor what happened to me on production when I handled  Parapilinurgus masumotoi that way this morning. let me find another example

hmm - I deleted Paratrikona->Paratrikona-albomaculata on staging (which is another example of this problem I cam across) and reimported and also found that it grafted successfully this time https://staging.inaturalist.org/taxa/885738-Paratrikona-albomaculata
frustrating
These are generated every day so lets wait until I find another one which will maybe give us more insight. But if its not consistent, thats probably going to be a pain...

@kueda I've been tracking these instances of taxa with ancestries of just a parent ID
taxa_sql = <<-SQL
  SELECT id FROM taxa WHERE ancestry ~ '^(?!.*\/).*$' AND is_active = TRUE AND rank_level != 70;
SQL
data = ActiveRecord::Base.connection.execute(taxa_sql)
data.map{|a| a["id"].to_i}.count

data.map{|a| a["id"].to_i}.each do |taxon_id|
  puts Taxon.where(id: taxon_id).first.name
  puts Taxon.where(id: taxon_id).first.parent.name
  puts ""
end

and they all seem to be generated as follows: someone enters a name and clicks 'search external name providers' such as 'Sigara limitata'. Ratatosk imports the taxon and one or more child taxa (e.g. 'Sigara limitata limitata'). Ratatosk first grafts the child to the parent (e.g. Sigara limitata limitata -> Sigara limitata) and then when grafting the parent to the tree (e.g. Sigara limitata -> Sigara), somehow something craps out and the ancestry of Sigara limitata limitata is not updated. Here are the taxa child parent pairs that have this issue that have been generated over the past week:
Sigara limitata limitata
Sigara limitata

Satsuma arisana arisana
Satsuma arisana

Saldula pilosella pilosella
Saldula pilosella

Orthotylus adenocarpi purgantis
Orthotylus adenocarpi

Globiceps sordidus sordidus
Globiceps sordidus

Nabis meridionalis meridionalis
Nabis meridionalis

Orius laticollis laticollis
Orius laticollis

Satsuma arisana takkiriensis
Satsuma arisana

Satsuma arisana tani
Satsuma arisana

Orthotylus ericetorum ericetorum
Orthotylus ericetorum

Tuponia suturalis suturalis
Tuponia suturalis

Acalypta musci musci
Acalypta musci

Cantacader quadricornis quadricornis
Cantacader quadricornis

Oedemera rufofemorata rufofemorata
Oedemera rufofemorata

Ilex sugerokii longipedunculata
Ilex sugerokii

Heterometrus indus indus
Heterometrus indus

Neptunides polychrous polychrous
Neptunides polychrous

Isturgia perviaria perviaria
Isturgia perviaria

Tachysphex helveticus helveticus
Tachysphex helveticus

Jatropha hernandiifolia portoricensis
Jatropha hernandiifolia

Tenthredo solitaria solitaria
Tenthredo solitaria

Hibiscus indicus integrilobus
Hibiscus indicus

Amphidromus schomburgki schomburgki
Amphidromus schomburgki

Deramas livens livens
Deramas livens

Notobalanus flosculus flosculus
Notobalanus flosculus

Acmaeodera rubromaculata rubromaculata
Acmaeodera rubromaculata

I've been unable to consistently reproduce this via the UI:
Step 1 - make sure these taxa don't exist
open staging console and run the following
  t.destroy
end
if t = Taxon.where(name: "Notobalanus flosculus").first
  t.destroy
end

Step 2 - add the taxa through the UI
log in to staging
go to https://staging.inaturalist.org/observations/24653887
click 'suggest an ID'
enter "Notobalanus flosculus"
click 'Search external name providers'
navigate to "Notobalanus-flosculus" note that its properly grafted
navigate to "Notobalanus flosculus flosculus" note that its also properly grafted (ie can't reproduce)
or through the console only:
Step 1 - make sure these taxa don't exist
open staging console and run the following
if t = Taxon.where(name: "Notobalanus flosculus flosculus").first
  t.destroy
end
if t = Taxon.where(name: "Notobalanus flosculus").first
  t.destroy
end

Step 2 - import the taxon
ext_names = TaxonName.find_external(params[:q], :src => params[:external_src])
ext_taxon_ids = ext_names.map(&:taxon_id).compact
puts ext_taxon_ids.join(",")
@external_taxa = Taxon.find( ext_taxon_ids ) unless ext_taxon_ids.blank?
@external_taxa.each do |external_taxon|
  external_taxon.delay(:priority => USER_INTEGRITY_PRIORITY).graft_silently unless external_taxon.grafted?
end

note that both taxa are properly grafted (ie can't reproduce)
Since I can't consistently reproduce this, something must be occasionally crapping out after the child is grafted to the parent when the parent is grafted to the tree that prevents the child ancestry from updating.


Jigsaw Component Integration

Added a desk at the front of the classroom that when clicked, allows the user to interact with a jigsaw puzzle.
https://www.loom.com/share/755f7621ebd5421786bc91a90b0660fb

Looks awesome from a design/product perspective, Vivian. Great work as always.


pkg/backup: change getEndpointWithMaxRev to getClientWithMaxRev

Since getEndpointWithMaxRev already creates etcdclient in order to determine the endpoint with max rev, it can smartly return the etcdclient with max rev instead.
Then backupmanager can use that client directly to retrieve snapshot instead of create a new client based on the max rev endpoint that getEndpointWithMaxRev originally returns.

cc/ @hongchaodeng

LGTM


Caelid missing a boss

Hi,
You are missing the Erdtree Avatar in Caelid.

Hello,
I think you might be confusing the Erdtree avatar with the Putrid Avatar

You are right, please close it :)

Let me know if you have any other missing content on the website, thank you!


Ansible uses outdate version of ciao-cert causing deployment failures

I tried to install ciao via the ansible scripts this morning, and failed.  The reason for the failure seems to be that I have an old version of ciao-cert in /tmp/go/bin left over from an install I did last year.  This version is missing an option that the ansible scripts rely on and so the build fails.  I guess ansible should rebuild ciao-cert before using it.  Maybe a -u is missing go get is called.
TASK [ciao-controller : Create Scheduler key and CA certificate] ***************
fatal: [ciao-ctl.ld.intel.com]: FAILED! => {"changed": true, "cmd": ["/tmp/go/bin/ciao-cert", "-anchor", "-role", "scheduler"<EMAIL_ADDRESS>"-organization=Example Inc.", "-ip=<IP_ADDRESS>", "-host=ciao-ctl.ld.intel.com", "-verify"], "delta": "0:00:00.012555", "end": "2017-02-17 10:17:59.746441", "failed": true, "rc": 2, "start": "2017-02-17 10:17:59.733886", "stderr": "flag provided but not defined: -anchor\nUsage of /tmp/go/bin/ciao-cert:\n  -alsologtostderr\n    \tlog to standard error as well as files\n  -directory string\n    \tInstallation directory (default \".\")\n  -dump string\n    \tPrint details about provided certificate\n  -elliptic-key\n    \tUse elliptic curve algorithms\n  -email string\n    \tCertificate email address (default \"ciao-devel@lists.clearlinux.org\")\n  -host string\n    \tComma-separated hostnames to generate a certificate for\n  -ip string\n    \tComma-separated IPs to generate a certificate for\n  -log_backtrace_at value\n    \twhen logging hits line file:N, emit a stack trace\n  -log_dir string\n    \tIf non-empty, write log files in this directory\n  -logtostderr\n    \tlog to standard error instead of files\n  -organization string\n    \tCertificates organization\n  -role value\n    \tComma separated list of SSNTP role [agent, scheduler, controller, netagent, server, cnciagent]\n  -server\n    \tWhether this cert should be a server one\n  -server-cert string\n    \tServer certificate for signing a client one\n  -stderrthreshold value\n    \tlogs at or above this threshold go to stderr\n  -v value\n    \tlog level for V logs\n  -verify\n    \tVerify client certificate\n  -vmodule value\n    \tcomma-separated list of pattern=N settings for file-filtered logging", "stdout": "", "stdout_lines": [], "warnings": []}

The workaround is to delete /tmp/go

I ran cleanup and the directory was deleted.
Is it a requirement that you always run cleanup before installing?

It is not required to run cleanup.yml before deploy.
The reason of not doing -u was to allow developer to make changes on the ciao directory and get those changes deployed.
What do you think on bringing ciao_dev option back to skip -u when ciao_dev=True and do the update when ciao_dev=False ?

It is not required to run cleanup.yml before deploy.

But what happens if the user makes some change to the cnci_agent.  If he just runs the playbook again will the cnci be rebuilt?

What do you think on bringing ciao_dev option back to skip -u when ciao_dev=True and do the update when ciao_dev=False ?

That sounds good.  What does ciao_dev actually do?  Does it use your GOPATH rather than /tmp/go?

But what happens if the user makes some change to the cnci_agent. If he just runs the playbook again will the cnci be rebuilt?

I suppose what I'm trying to say is I think there are currently some cases where you do need to do a cleanup to before an install.  Maybe what we need to do is to simply identify these cases, treat them as bugs and fix them in the playbook.

But what happens if the user makes some change to the cnci_agent. If he just runs the playbook again will the cnci be rebuilt?

No, unless the user removes images/.cnci_ready and removes the image from ciao-image

That sounds good. What does ciao_dev actually do? Does it use your GOPATH rather than /tmp/go?

Nothing, the option was removed when we switched to deploy from source code

Nothing, the option was removed when we switched to deploy from source code

What would it do if we brought it back?

No, unless the user removes images/.cnci_ready and removes the image from ciao-image

And this is likely to cause a lot of confusion for people who are working on the cnci.  It would be great if we could identify all of these places where you currently do need to do a cleanup and fix them.


Update Switcher.tsx to fix build error

Fix build error #163, suggested by bigsk1.

No need to re-add this useless parameter, just remove them directly from EmptyChat.tsx and Navbar.tsx, that'll work fine. Refer to this PR #168


Generate Helm config options from values.md

This PR reads values.yaml and generates a table of options automatically, assuming the values.md is documented correctly.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: sdake
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: linsun
Assign the PR to them by writing /assign @linsun in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/assign @geeknoid

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: sdake
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: geeknoid
Assign the PR to them by writing /assign @geeknoid in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Correct install command

This suppresses a warning saying that the package --package is not found.

im just confused with this install command
apparently it worked for you even without the package??? but in #2 it didn't??

You need to specify the package, but it is not a flag.
Per cargo install --help:
Usage: cargo.exe install [OPTIONS] [crate]...

You just need to specify the crate list at the end of the options.


Inconsistent interpretation of model output between self-conditioning step and prediction step

I think there is a bug in the interface self-conditioning in rin_pytorch.py.
The model output is interpreted differently during the self-conditioning stage compared to the prediction stage.
Currently we have (pseudocode):
self_cond = x0_to_target_modification(model_output)           # Treat model prediction as x0 convert it to x0, eps, or v
pred = self.model(..., self_cond, ...)                        # Self-condition on prediction for x0, eps, or v and predict x0, eps, or v
target = x0_to_target_modification(x0)
loss = F.mse_loss(pred, target)

In the current implementation, the interface prediction is interpreted as x0 during self-conditioning, but as the target (x0, eps, or v) at the prediction step.
I see two ways that we could do interface self-conditioning that would be consistent.
We could either:

make the model always predict x0. Then we would have

self_cond = model_output                                      # Where self_cond is a prediction for x0
pred = self.model(..., self_cond, ...)                        # Self-condition on x0 prediction and predict x0
pred = x0_to_target_modification(pred).                       # Convert x0 prediction into prediction for x0, eps, or v
target = x0_to_target_modification(x0)
loss = F.mse_loss(pred, target)

or

make the model always predict what it is intended to predict (x0, eps, or v). Then we would have

self_cond = model_output                                      # Where self_cond is a prediction for the target (x0, eps, or v)
pred = self.model(..., self_cond, ...)                        # Self-condition on prediction for x0, eps, or v and predict x0, eps, or v
target = x0_to_target_modification(x0)
loss = F.mse_loss(pred, target)

In contrast to the current implementation, in my two proposals, the interpretation of the interface prediction is the same between the self-conditioning step and the prediction step. Would you agree that there is inconsistency here and that either of these proposals solves it?
Here is the current code:
if random() < self.train_prob_self_cond:
    with torch.no_grad():
        model_output, self_latents = self.model(noised_img, times, return_latents = True)
        self_latents = self_latents.detach()

        if self.objective == 'x0':
            self_cond = model_output

        elif self.objective == 'eps':
            self_cond = safe_div(noised_img - sigma * model_output, alpha)

        elif self.objective == 'v':
            self_cond = alpha * noised_img - sigma * model_output

        self_cond.clamp_(-1., 1.)
        self_cond = self_cond.detach()

# predict and take gradient step

pred = self.model(noised_img, times, self_cond, self_latents)

...

loss = F.mse_loss(pred, target, reduction = 'none')
loss = reduce(loss, 'b ... -> b', 'mean')

@jsternabsci i have only seen self conditioning done with the predicted x0 (correct me if i'm wrong)
there's nothing inconsistent as during inference, self conditioning is also done with the predicted x0
however, i get what you are saying. i could offer both options, if you are running experiments to see which way is better?


Progress

Fix https://github.com/microsoft/testfx/issues/3728
initial implementation of showing tests in progress / test node detail

Closing in favor of #4221


How to make wagtail_textract work with Media Storage Backend

Hi there,
I am trying to use wagtail_textract for my project. I tried previously using just textract but am interested in some of the helper utilities of wagtail_textract. I am wondering how wagtail_textract will work in production with Docker and a Media Storage backend, such as Azure.
The line here is referencing a file path:
text = textract.process(document.file.path).strip()
but when in production using a Media Storage backend, it seems like this will fail because it does not have a proper file system. Has this been tested or does anybody know how I might be able to get this to work? Any help would be much appreciated! Let me know if you need any more info about my project setup.

Hi @allcaps ,
Thanks for your response. I ended up using tempfile to make document indexing work with my Azure Media Storage backend. The solution I made accounts for both local development and production environments. It uses the normal file.path when in your local dev environment, but then in production uses tempfile to read the file and then close and delete.  Below is the snippet I used:
import tempfile

def index_documents(self):
        """Loops through all the documents in a ResourcePage and returns a string with all the extracted text."""
        alltext = ''
        for block in self.body:
            if block.block_type == 'resource_document':
                if block.value['document'].file_extension.lower() in ['pdf', 'pptx', 'html', 'htm', 'xls', 'xlsx', 'doc', 'docx', 'rtf', '.txt']:
                    try:
                        path = block.value['document'].file.path
                        text = self.extract_text(path)
                    except NotImplementedError:
                        logger.info('Downloading for search index %s' % block.value['document'].file.url)
                        remote_file_url = block.value['document'].file.url
                        f = tempfile.NamedTemporaryFile('w+b', suffix='.%s' % block.value['document'].file_extension)
                        urlretrieve(remote_file_url, filename=f.name)
                        path = f.name
                        text = self.extract_text(path)
                        f.close()
                    alltext += text.decode("utf-8")
        return alltext

The relevant section is in the except statement. In my case, I am looping through a StreamField where users add documents to the page and indexing the individual documents. This could be extended to the way that wagtail_textract indexes documents through the transcribe_document handler.
Thoughts on the approach?

Hi @danieltomasku ,
Thanks for bringing this use case to attention.
That approach looks pretty good to me.
Is there some way we can make this easier? Maybe the transcribe_document() method could have some pluggable helper method?

Seems like Wagtail does the 'is-this-a-local-file' check in the same way: https://github.com/wagtail/wagtail/blob/7034cd131774b8971ff3c7424999a28164480f29/wagtail/documents/views/serve.py#L35

Hi @danieltomasku, I'd be happy to accept a PR if that helps.


Only initialize runner once in NeymanConstructor

This is a further fix of https://github.com/XENONnT/alea/pull/100. We do not need to initialize runner for each **{**nominal_values, **generate_values}, but we can pass generate_values, nominal_values to runner.update_poi to pass correct nominal_values to get the expectations. Additionally, I made modifications:

Require parameter's ptype to be in ["rate", "shape", "efficiency", "livetime"], the default ptype is "shape".
Recover StatisticalModel.set_nominal_values and Parameters.set_nominal_values, add option to modify the static parameter so that user is aware of the modification.

This PR can be closed now, right @dachengx ?

Suspended by https://github.com/XENONnT/alea/pull/108


Faceted datasets, when sourced do not inherit children

Given a faceted dataset A and another dataset B whose source is A:
Since the resulting groups in B do not inherit their _id from A, the GroupBuilder attempts to associate B's groups with non-existent datasets named vg_bb instead of vg_aa.
This is demonstrated by the spec below, which draws two trendlines by faceting flat_data into grouped_data then sourcing it as sourced_group_data. Rendering the marks with "from": { "data": "grouped_data" } works as expected, but "from": { "data": "sourced_grouped_data" }, as shown, breaks it.
{
  "width": 400,
  "height": 400,
  "padding": { "top": 0, "left": 0, "right": 0, "bottom": 0 },

  "data": [
    {
      "name": "flat_data",
      "values": [
        { "category": "A", "x": 0,   "y": 300 },
        { "category": "A", "x": 100, "y": 400 },
        { "category": "A", "x": 200, "y": 300 },
        { "category": "A", "x": 300, "y": 100 },
        { "category": "A", "x": 400, "y": 200 },
        { "category": "B", "x": 0,   "y": 200 },
        { "category": "B", "x": 100, "y": 100 },
        { "category": "B", "x": 200, "y": 400 },
        { "category": "B", "x": 300, "y": 300 },
        { "category": "B", "x": 400, "y": 400 }
      ]
    },
    {
      "name": "grouped_data",
      "source": "flat_data",
      "transform": [
        {
          "type": "facet",
          "groupby": "category"
        }
      ]
    },
    {
      "name": "sourced_grouped_data",
      "source": "grouped_data"
    }
  ],

  "marks": [
    {
      "type": "group",
      "from": { "data": "sourced_grouped_data" },
      "marks": [
        {
          "type": "line",
          "properties": {
            "enter": {
              "x": { "field": "x" },
              "y": { "field": "y" },
              "stroke": { "value": "black" },
              "strokeWidth": { "value": 3 }
            }
          }
        }
      ]
    }
  ]
}

A quick fix for this is to change line 207 to
inherit = "vg_"+(Object.getPrototypeOf(group.datum)._id || group.datum._id);

i.e, checking for the _id that group.datum would have inherited from __proto__. This fix passed all tests.
Not sure where to go from here. Happy to contrib, but would need some more info, like:
Is the issue more complex than what's addressed here?
Should a proper fix iterate over the prototype chain instead of assuming a single level?
Is it necessary to use a polyfill for accessing the __proto__ object?
Thanks!
Amit

This is great catch, Amit! Setting aside polyfill, there are a couple of issues I can see with us sticking with using tuple IDs to map to the faceted datasets: deriving a C data set from B, and faceting the derived C data set itself. In either case, the logic for determining the faceted datasource's name would require logic to iterate up the prototype chain and/or verify the datasource exists to know when to stop iterating.
Instead, I would propose introducing a _facetID field on tuples. The Facet operator could maintain a counter (much like we do with tuples). On every new facet, the counter would be incremented, the new datasource would be named vg_{facetID} and this would be stored in the corresponding tuple's _facetID field. With this strategy, I believe we can just rely on JavaScript's native prototypal inheritance. Derived tuples in B and C would still have the correct _facetIDs from their upstream counterparts, and if C was further faceted, new _facetIDs would be assigned to its tuples.
What do you think of this strategy?
If you would like to implement the fix, I'd be happy to merge in the pull request when it's ready! If not, I can take care of it too. Just a heads up, there are a couple of places that this inheritance is hardcoded (e.g., in Scale). So another upshot of the _facetID approach is centralizing the naming of faceted datasources.

Thanks for filing the issue, @meetamit. I went ahead and fixed the bug in 221df8c. Let me know if you still experience any issues. Thanks!

@arvind, oops — I was hoping to get to it before you and save you the work, but you beat me to it. Thank you!

No worries! Thanks for catching and tracing the issue. It made implementing the fix orders of magnitude easier :)


Greenkeeper/snyk 1.313.1

Pull Request Test Coverage Report for Build 4234

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 4230:
0.0%

Covered Lines:
128

Relevant Lines:
128

💛  - Coveralls


Travis should run the website tests

Currently he just runs the unit tests :disappointed:

@Sjelin We are running protractor tests as part of Travis using headless browser... and have not seen issues so far. We found that phantomjs doesn't behave properly.

We aren't running the website's tests

what tests are you running or trying to run ? I think I didn't get your point completely

There's a separate set of tests in the /website/ folder

Since the website doesn't update except when a new version is released, I think this might just slow down the test suite - is there something that slipped through the cracks that this would have caught?

Yeah, I noticed when reviewing marcy's PR that she broke some of the website tests and I didn't notice until I was looking through it for the second or third time.  It is true that it didn't make it all the way to the website though.  Maybe a better solution is to have the generate-docs.sh refuse to run unless you've run the tests on the current version.

We decided against this ages ago


Fix docstring syntax for INIT

Codecov Report

Merging #34 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #34   +/-   ##
=======================================
  Coverage   90.74%   90.74%
=======================================
  Files           2        2
  Lines          54       54
=======================================
  Hits           49       49
  Misses          5        5

Impacted Files
Coverage Δ

src/InitialValues.jl
87.17% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5a4798b...8b52bdd. Read the comment docs.