Compile time (CodeRemote)

Hi, Kapil here from CodeRemote. I wanted to continue our discussion from r/rust, and figured it would be easier to track here.
Sounds like you're facing bad compile times here, and I'd like to see how I can help alleviate that

Sure, tracking the timings here makes sense.

I profiled incremental builds (i.e. cargo [cmd], touch src/main.rs, cargo [cmd] again) in debate-map/app-server!
Default compiler cargo check: 2.86 s ; My modded compiler cargo check: 2.33 s
Default compiler cargo build: 6.88 s ; My modded compiler cargo build: 6.40 s
I did a simple git clone and haven't messed with the code at all. There is a 17% improvement in cargo check time, but I'm not sure why you said an incremental cargo build (on the default compiler) is taking several minutes? FWIW I am running this locally on a Linux machine, sans Docker.

The slow increment cargo build happens after changing a single line of code here: https://github.com/debate-map/app/blob/8def743f6db26dbe3b495d018d7b452629baefd6/Packages/app-server/src/main.rs#L59
I simply add a "1" to the end of that log message, and that's when I see the slow incremental recompiles.
Here are my compile timings, when using cargo build: (ie. debug build)
# first build (not 100% pristine since forgot to clear "target" folder beforehand)
   [...]
   Compiling rust-shared v0.0.1 (C:\Root\Apps\@V\DebateMap\Main\Packages\rust-shared)
   Compiling app-server v0.1.0 (C:\Root\Apps\@V\DebateMap\Main\Packages\app-server)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 3m 48s

# recompile (no code change)
PS C:\Root\Apps\@V\DebateMap\Main\Packages\app-server> cargo build
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 5.80s

# recompile (no code change)
PS C:\Root\Apps\@V\DebateMap\Main\Packages\app-server> cargo build
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.94s

# recompile (after changing that one log line)
PS C:\Root\Apps\@V\DebateMap\Main\Packages\app-server> cargo build
   Compiling app-server v0.1.0 (C:\Root\Apps\@V\DebateMap\Main\Packages\app-server)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1m 02s

And here it is for cargo build --release:
# first build (not 100% pristine since forgot to clear "target" folder beforehand)
   [...]
   Compiling dyn-clone v1.0.6
   Compiling app-server v0.1.0 (C:\Root\Apps\@V\DebateMap\Main\Packages\app-server)
    Finished `release` profile [optimized] target(s) in 7m 57s

# recompile (no code change)
PS C:\Root\Apps\@V\DebateMap\Main\Packages\app-server> cargo build --release
    Finished `release` profile [optimized] target(s) in 1.48s

# recompile (no code change)
PS C:\Root\Apps\@V\DebateMap\Main\Packages\app-server> cargo build --release
    Finished `release` profile [optimized] target(s) in 1.12s

# recompile (after changing that one log line)
PS C:\Root\Apps\@V\DebateMap\Main\Packages\app-server> cargo build --release
   Compiling app-server v0.1.0 (C:\Root\Apps\@V\DebateMap\Main\Packages\app-server)
    Finished `release` profile [optimized] target(s) in 4m 52s

While I normally do the builds in docker, the timings above are done on my host machine (OS: Windows 10, CPU: Ryzen 7 1700X, 8-Core). The rest of my timings in this thread (unless otherwise specified) will also be done on my host machine, for consistency -- and to rule out the possibility of a Docker misconfiguration.
Anyway, as seen from the timings above, incremental recompiles are really quite slow for having changed only a single logging line. So quite curious to see what results you have with your modified rust! (which for reference for other readers, adds caching of the expansions of proc-macros when the token-stream input for those macros has not changed since the last compile)

@Venryx My incremental builds are on the order of 14 seconds when I make that 1-line change. I'm happy to get you set up on a remote machine with my compiler, if you want to try it out. I just need your ssh pubkey for it. Feel free to email me at<EMAIL_ADDRESS>

@Venryx My incremental builds are on the order of 14 seconds when I make that 1-line change. I'm happy to get you set up on a remote machine with my compiler, if you want to try it out. I just need your ssh pubkey for it. Feel free to email me at<EMAIL_ADDRESS>
I'm guessing the 14 seconds is for an incremental debug build, with your fork of rust?
For completeness' sake, it'd be cool to also know: (with the one-line change in each case)

Incremental debug build, on your machine, with stock rust.
Incremental release build, on your machine, with stock rust.
Incremental release build, on your machine, with your modified rust.

But in any case, 14 seconds sounds great! Will send you in an email in a bit.

I'll include a summary here of my cargo check/build timings (also included in the email I just sent).
Notes:

All of these timings are the "incremental" build/check times, as executed after changing the log-message of the single println call in Packages/app-server/src/main.rs.
Naturally, my custom "wrap_slow_macros" macro is disabled for all of these benchmarkings. (the macro technically gets called, but just immediately returns its input tokens)
All of my local machine timings (well actually, all the timings below in general) are not done within Docker or anything. (mentioned since my normal build process for debate-map occurs within Docker; for these timings of course, I wanted to remove the Docker variable from the benchmarking process)
For most of the timings below (excluding eg. my local timings, though some of those are in my comment above), you can see the raw logs for the attempts here: https://github.com/debate-map/misc/blob/main/RustCompileBenchmarks/2024-03-26.txt

On my local machine, the timings are:

My local machine, cargo check, stock rust: ~8s
My local machine, cargo check --release, stock rust: ~34s
My local machine, cargo build, stock rust: ~1m2s
My local machine, cargo build --release, stock rust: ~4m52s

On the CodeRemote build machine, the timings are:

CR build machine, cargo check, stock rust: ~3s
CR build machine, cargo check, modded rust: ~3s
CR build machine, cargo check --release, stock rust: ~12s
CR build machine, cargo check --release, modded rust: ~5s
CR build machine, cargo build, stock rust: ~8s
CR build machine, cargo build, modded rust: ~8s
CR build machine, cargo build --release, stock rust: ~1m20s
CR build machine, cargo build --release, modded rust: ~1m20s

First observation: The CodeRemote build machine's CPU (Ryzen 5 3600) should only be +17% faster (cpu benchmark here) than my CPU (Ryzen 7 1700x), yet it somehow is +300% faster for compiling this particular project!
So I will need to investigate and see what the heck is causing such a big compile-time difference between CPUs with similar specs.

Summary: For this particular project, the modded compiler appears to be helpful for "cargo check --release", but not for the other check/build configs. This is surprising, since I thought my project was bottlenecked by the proc-macro expansions, for incremental release builds. However, I am grateful to have been prompted to do these tests, since I can see plainly now how slow the compiles are on my local machine for some reason, which I will now investigate.
(And thank you to @kapilsinha for taking the time to set up a test environment for me on his build server! It was nice to get to try a mod of the compiler with caching of proc-macro expansions, and I wish him the best in his continued development of it, given the substantial speedups it's shown in some other benchmarked projects.)

The benchmarkings you posted http://github.com/debate-map/app/issues/285#issuecomment-2020222609 are very interesting.
Is Code Remote building on Windows like yourself? Or on Linux?

Is Code Remote building on Windows like yourself? Or on Linux?

The Code Remote building is done on Linux. (and not within a Docker container or anything from what I could tell)

The Code Remote building is done on Linux. (and not within a Docker container or anything from what I could tell)

So you are comparing completely different things.
Try linux on windows subsystem with mold and tell us the results.

Okay, I have redone the tests on my local machine, except within WSL2 rather than on Windows: (and naturally, I used a fresh clone of my repo within the linux file-system, to avoid cross-OS filesystem overhead)

My local machine, WSL2, cargo build, stock rust: ~20s [compared to ~27s outside of WSL2]
My local machine, WSL2, cargo build --release, stock rust: ~3m01s [compared to ~4m52s outside of WSL2]

So indeed, it seems that building in Linux rather than Windows does give a substantial speed improvement.
Still slower than expected relative to the CodeRemote build machine though -- which should only be +17% faster (based on relative cpu power/generic-benchmarks), but is still much more than that:

CR build machine, cargo build, stock rust: ~8s
CR build machine, cargo build --release, stock rust: ~1m20s


Does it work with vuejs 0.12?

Hey,
I'm editing a old project that was written with vuejs 0.12. And when i try to enable vue devtools, i always get that its detected but is in production mode or disabled by the user.

Vue.js is detected on this page. Devtools inspection is not available because it's in production mode or explicitly disabled by the author.

Im setting the debug to true
Vue.config.debug = true;

So I'm wondering if devtools even work with vue 0.12 and if it does, how do i enable it.

It doesn't seem like they do. If you manage to find a fix for it, we're open to a PR. But unfortunately, we're not going to do it because it's not worth anymore.


Azure Shell Integration

Explore the option of integrating the Azure Shell into the site allowing readers to try out the script in their environment.

The try it functions maybe locked down to a specific location - lets try it though.


Use ActiveSupport::Reloader in Rails 5

Rails 5.1 will remove the cleanup! and prepare! methods from ActionDispatch::Reloader, as they have been replaced with ActiveSupport::Reloader reload!.
DEPRECATION WARNING: cleanup! is deprecated and will be removed from Rails 5.1 (use Rails.application.reloader.reload! instead of cleanup + prepare) (called from <main> at bin/delayed_job:5)
DEPRECATION WARNING: prepare! is deprecated and will be removed from Rails 5.1 (use Rails.application.reloader.prepare! instead) (called from <main> at bin/delayed_job:5)

Coverage decreased (-0.8%) to 90.32% when pulling 80c6f9c1de99b653e9f1e380241e25ac2971bcc4 on stuwil:activesupport-reloader into e3772d4f0c8470d0fcba00c86ca3bc4f5e876830 on collectiveidea:master.

Coverage decreased (-0.6%) to 90.508% when pulling 2498b67f88c81a31fa25a2c8fc63f7e17ad379c1 on stuwil:activesupport-reloader into e3772d4f0c8470d0fcba00c86ca3bc4f5e876830 on collectiveidea:master.

Coverage decreased (-0.6%) to 90.508% when pulling 2498b67f88c81a31fa25a2c8fc63f7e17ad379c1 on stuwil:activesupport-reloader into e3772d4f0c8470d0fcba00c86ca3bc4f5e876830 on collectiveidea:master.

It's October now. Is there any interest in merging this? The deprecation warnings spam our logs.

Handled by #948


plugin produces compile error

I used cocoon and phonegap
both give following compile error:
No resource identifier found for attribute 'appComponentFactory' in package 'android'
com.android.ide.common.process.ProcessException: Failed to execute aapt
then i removed cordova-plugin-crosswalk-webview plugin and compile worked
So what now?

Having the exact same problem building with cordova locally. Removing the plugin fixes my issue, but I need the plugin.
/path/to/platforms/android/build/intermediates/manifests/full/armv7/release/AndroidManifest.xml:57: AAPT: No resource identifier found for attribute 'appComponentFactory' in package 'android'
/path/to/platforms/android/build/intermediates/manifests/full/armv7/release/AndroidManifest.xml:57: error: No resource identifier found for attribute 'appComponentFactory' in package 'android'
FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':processArmv7ReleaseResources'.

com.android.ide.common.process.ProcessException: Failed to execute aapt

Same problem building with PhoneGap cli-6.5.2. Setting PhoneGap cli back to 6.0.0 fixed the problem.

I don't feel like that's a fix. I don't even use PhoneGap cli.

As pointed out by @Scuttman in #205, this is caused when building with Android Support Library v28.
To avoid needing to manually create a build-extras.gradle file, use cordova-android-support-gradle-release plugin to specify v27:
cordova plugin add cordova-android-support-gradle-release  --variable ANDROID_SUPPORT_VERSION=27.+

@dpa99c this method works for me, thank you dude u save me

@dpa99c this method works for me and thank you so much.

@dpa99c i love you

I feel like installing a plugin to fix a bug is a bit like killing a mosquito with a hammer. Is this really the correct solution?

@Moghul There's plugins and there's Plugins...
This "plugin" is just Gradle config which is deployed by a nodejs script fired by Cordova.
There's no native code.
It's a generic solution to a problem which is inherently caused by the fact that cordova-android  doesn't provide a mechanism by which you can take fine-grain control of Gradle dependency versions.
The "correct" solution would be to add such as mechanism to the cordova-android platform.

@ #

That worked for me! Thanks @dpa99c !!

@dpa99c thanks man , that worked for me ! , thanks

@dpa99c thanks man

@dpa99c thank you very much!


[IMPROVEMENT] Configure fastview projection miss threshold

We observe some warning from perfectly well running systems. This threshold is hard coded, a bit of liberty can be nice.

Merge or not?


[5.9 🍒][Explicit Module Builds] Remove logic for Swift-client-specific Clang module dependency jobs

Cherry-pick of #1332
We have moved towards a build model where the output of the dependency scanner is the canonical collection of modules that must be built. The driver's logic to generate potentially multiple examples of a given Clang dependency, with different command sets, as required per Swift client module in the build graph, is no longer valid - even module output paths are no longer determined by the driver. Remaining logic that is removed in this PR may result in mistakenly producing duplicate jobs for the same Clang PCM build job.
Change affects explicit module builds only and does not have any impact on implicit build code-path.

@swift-ci test


forEach is not working with my query.....??

Hi Sir,
db.collection.find( { } ).forEach(function(docs){
 console.log(docs);
})

this query is not working..
while applying only find then its working...
please provide me some solution on it..

db.collection.find( { } ) returns a cursor which has no forEach function. To make this query working you could use
db.collection.find( { } ).toArray(function (err, docs) { docs.forEach ... })

or simply
db.collection.find( { }, function(docs) { docs.forEach ... })

Or use the cursor API https://github.com/mafintosh/mongojs#cursor

ok thanks its working for me :)

@saintedlama I thought there was a forEach method in the cursor. It's even in the API
https://github.com/mafintosh/mongojs#cursorforeachfunction

@sorribas You got me :smile: there is a forEach function defined in the cursor prototype. @Ashh007 What's your exact issue with the forEach on the cursor? Any error message?

i think the forEach callback is (err, value) and not (value)

@saintedlama  while using in mongo console its working withe following code...
db.collection.find( { } ).forEach(function(docs){  console.log(docs); })
but not working with mongojs... getting empty response..

@Ashh007 its function (err, doc)

@mafintosh yes sir but in forEach we got only one value in function..

@Ashh007 this is how forEach is expected to work. If you prefer to get all documents just stick to the method outlined above.


dotnet-sdk: move license files to the proper folder

Description of changes
Fixes #193656
Moves LICENSE.txt and ThirdPartyNotices.txt to share/dotnet-sdk/<version/ directory.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

This looks good to me.
There is still a dotnet file at the root of the directory though. Do we want to move the binary to bin/ instead of having a symlink from $out/bin/dotnet to $out/dotnet?

Result of nixpkgs-review pr 194061 run on x86_64-linux 1

1 package blacklisted:

tests.writers

3 packages failed to build:

jetbrains.rider
vscode-extensions.ms-vsliveshare.vsliveshare
xivlauncher

43 packages built:

ArchiSteamFarm
alttpr-opentracker
azure-functions-core-tools
baget
boogie (dotnetPackages.Boogie)
btcpayserver
depotdownloader
discordchatexporter-cli
dotnet-aspnetcore (dotnet-aspnetcore_6 ,dotnetCorePackages.aspnetcore_6_0)
dotnet-aspnetcore_3 (dotnetCorePackages.aspnetcore_3_1)
dotnet-aspnetcore_5 (dotnetCorePackages.aspnetcore_5_0)
dotnet-aspnetcore_7 (dotnetCorePackages.aspnetcore_7_0)
dotnet-runtime (dotnet-runtime_6 ,dotnetCorePackages.runtime_6_0)
dotnet-runtime_3 (dotnetCorePackages.runtime_3_1)
dotnet-runtime_5 (dotnetCorePackages.runtime_5_0)
dotnet-runtime_7 (dotnetCorePackages.runtime_7_0)
dotnet-sdk (dotnet-sdk_6 ,dotnetCorePackages.sdk_6_0)
dotnet-sdk_3 (dotnetCorePackages.sdk_3_1)
dotnet-sdk_5 (dotnetCorePackages.sdk_5_0)
dotnet-sdk_7 (dotnetCorePackages.sdk_7_0)
eventstore
formula
github-runner
inklecate
jackett
jellyfin
msbuild
nbxplorer
netcoredbg
omnisharp-roslyn
opentabletdriver
osu-lazer
pinta
pre-commit
prowlarr
ps3-disc-dumper
python-language-server
radarr
roslyn
ryujinx
scarab
wasabibackend
wasabiwallet

This looks good to me.
There is still a dotnet file at the root of the directory though. Do we want to move the binary to bin/ instead of having a symlink from $out/bin/dotnet to $out/dotnet?

That does make more sense to me, not sure why it wasn't done in the first place.

If nobody complains, I will merge the PR soon.

About the dotnet binary: I still have to test if it doesnt break the package. I'm in favor of leaving it, since it will only cause conflicts with other dotnet versions, and in that case the user should use combinePackages, as explained in the docs.

I'm in favor of leaving it, since it will only cause conflicts with other dotnet versions,

I completely agree. We shouldn't defer this PR because of that.
Thank you for your work!


Add Markdown support to Jupyter book with SQL kernel

Currently Jupyter book with SQL kernel does not support Markdown

Hi @AndreyQC can you elaborate a bit? Markdown ("text") cells should be rendering just fine when using the SQL kernel. Are you referring to cells inside of a notebook or .md files inside of a book? Neither should be associated with the kernel chosen.


Add Chef support

Add the ability to install chef client to use a Box as a Chef workstation

I would also use this -- would allow me to share recipes from production in my codio dev environments.


How to get pointcloud2 by realsense d435 using Matlab

| Camera Model                       | D435 |
| Operating System & Version |   Win 10  |
| Platform                                 | PC  |
| Language                            |  Matlab    |
|
Issue Description
I want to process .xyz file from D435 in Matlab, but it Require PointCloud2 object as given in https://in.mathworks.com/help/ros/ref/pointcloud2.html#bu3uilg
I can save the using realsense D435 as .ply but it is not pointCloud2 as required in Matlab.
There are many methods for converting the ply to .xyz but my moto is to direct access of incoming data from realsense to preserve the shape of object......
This method suggested by you is also working but I think there a loss feature by conversion.
So my Aim here is to access the realsense data to pointcloud2.

I think this project will help you.
https://github.com/IntelRealSense/realsense-ros
https://github.com/IntelRealSense/realsense-ros/blob/development/realsense2_camera/src/base_realsense_node.cpp#L805

@sonumathur Did you get chance to try? Looking forward to your update. Thanks!

I have no knowledge about how to work with ROS as replied by @RealSenseCustomerSupport .
Till not able to generate pointcloud2 object using D435 in Matlab.
If there is any method for Matlab then please provide.....
Thanks

Sorry, PointCloud2 is a kind of ROS message.
so far, get pointcloud2 object via ROS is the easiest way.

I think there is some confusion,
@sonumathur - have you reviewed pointcloud_example.m. The team is not very experienced with Matlab, so converting to pointCloud2 is more of a general Matlab question, but you should be able to fetch all the raw data using the provided example

Thanks for your reply @dorodnic
Actually I tried many solutions but still not get the pointcloud2 object.
My main aim is to generate a .xyz data of object under D435. and I think that is generated under pointcloud2.
If there is any way to generate pointcloud2 data using realsense in other Language, Please Provide.
Thanks.

@sonumathur Sorry for late reply! Is this still problem for you? Looking forward to your reply. Thanks!

Till today there is no update in this regard...........

according to the page https://www.mathworks.com/help/ros/ref/pointcloud2.html

The PointCloud2 object is an implementation of the sensor_msgs/PointCloud2 message type in ROS.

You can receive the PointCloud2 object from the ROS wrapper.
Or, try the script in this page to convert xyz data to pointcloud2 object in Matlab.
https://www.mathworks.com/matlabcentral/answers/395620-ros-create-pointcloud2-from-matlab-pointcloud

@sonumathur Did you get chance to try? Looking forward to your update. Thanks!

@RealSenseSupport  I try many times but not able to get desired result.
Under ROS I try following:
roslaunch realsense_ros_camera rs_rgbd.launch
rosrun pcl_ros pointcloud_to_pcd input:=/sensor_msgs/PointCloud2
But there is no output Pointcloud file from this topic /sensor_msgs/PointCloud2

the topic name is "/camera/depth_registered/points" by default.
the type of this topic is "sensor_msgs/PointCloud2"

@sonumathur Any other questions about this? Looking forward to your update. Thanks!


Import react from react

in order to remove deprecation warnings introduced with react-native 0.25.1
Todo:

[ ] replace import React from 'react-native' with import React from 'react' in Components
[ ] do the same in the packages that are included in the vendor folder
[ ] react-native-calendar-picker
[ ] react-native-gifted-messenger
[ ] react-native-scrollable-tab-view

Thanks @p0wl ! Much appreciated.


Agent fails to run if serialized, restored, serialized again, and restored again

Describe the bug
A have a simplistic Agent, basically the example examples/agents/first.ts, changed to work with BAM and wrapped with a Web Server, written using Express. At the end of every interaction the agent is serialized and set into the user's session. When requests come in, if the session has a serialized agent in it, it is restored, otherwise the agent is created from scratch.
This works for two iterations. On the third one it fails, with an exception:
undefined:3
return (...args)=>target.invoke(...args)
                         ^

ReferenceError: target is not defined

To Reproduce

Add extra dependencies:  yarn add express express-session @types/express @types/express-session
yarn start <code below>
In another terminal session:
a. curl -X POST --data-binary '{"text": "Who is the president of the USA"}' -H "Content-Type: application/json" --cookie cookiefile --cookie-jar cookiefile http://localhost:3000
b. curl -X POST --data-binary '{"text": "How old is he?"}' -H "Content-Type: application/json" --cookie cookiefile --cookie-jar cookiefile http://localhost:3000
c. curl -X POST --data-binary '{"text": "Where was he born?"}' -H "Content-Type: application/json" --cookie cookiefile --cookie-jar cookiefile http://localhost:3000

Expected behavior
It should not fail.
Screenshots
Here is the terminal output of the server when it fails:
(node:52233) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
[server]: Server is running at http://localhost:3000
Creating agent
Agent created
Running agent
Agent ran
Serializing agent
Agent serialized
Restoring agent from session
Agent restored from session
Running agent
Agent ran
Serializing agent
Agent serialized
Restoring agent from session
Agent restored from session
Running agent
undefined:3
return (...args)=>target.invoke(...args)
                         ^

ReferenceError: target is not defined
    at Object.eval [as callback] (eval at <anonymous> (file:///Users/kallner/Projects/ai-agents/bee-agent-framework/src/serializer/serializer.ts:1:11657), <anonymous>:3:26)
    at run (/Users/kallner/Projects/ai-agents/bee-agent-framework/src/emitter/emitter.ts:203:40)
    at _Emitter.invoke (/Users/kallner/Projects/ai-agents/bee-agent-framework/src/emitter/emitter.ts:204:60)
    at Object.on.toBoundedFunction.value (/Users/kallner/Projects/ai-agents/bee-agent-framework/src/emitter/emitter.ts:106:29)
    at run (/Users/kallner/Projects/ai-agents/bee-agent-framework/src/emitter/emitter.ts:203:40)
    at _Emitter.invoke (/Users/kallner/Projects/ai-agents/bee-agent-framework/src/emitter/emitter.ts:204:60)
    at Object.on.toBoundedFunction.value (/Users/kallner/Projects/ai-agents/bee-agent-framework/src/emitter/emitter.ts:106:29)
    at run (/Users/kallner/Projects/ai-agents/bee-agent-framework/src/emitter/emitter.ts:203:40)
    at _Emitter.invoke (/Users/kallner/Projects/ai-agents/bee-agent-framework/src/emitter/emitter.ts:204:60)
    at _Emitter.emit (/Users/kallner/Projects/ai-agents/bee-agent-framework/src/emitter/emitter.ts:189:23)

Code snippets
Here is the code being run above:
import "dotenv/config.js";
import express, {Express, Request, Response } from 'express';
import session from 'express-session';

import { BeeAgent } from "bee-agent-framework/agents/bee/agent";
import { BAMChatLLM } from "bee-agent-framework/adapters/bam/chat";
import { TokenMemory } from "bee-agent-framework/memory/tokenMemory";
import { DuckDuckGoSearchTool } from "bee-agent-framework/tools/search/duckDuckGoSearch";
import { OpenMeteoTool } from "bee-agent-framework/tools/weather/openMeteo";

const app: Express = express();
const port = process.env.PORT || 3000;

app.use(express.json());
app.use(session({
    secret: 'keyboard cat',
    resave: false,
    saveUninitialized: false
}));

declare module 'express-session' {
    interface SessionData {
        agent: string;
    }
}

app.post("/", async (req: Request, res: Response) => {
    var agent: BeeAgent;

    if (req.session && req.session.agent) {
        console.log("Restoring agent from session");
        agent = BeeAgent.fromSerialized(req.session.agent);
        console.log("Agent restored from session");
    } else {
        console.log("Creating agent");
        const llm = BAMChatLLM.fromPreset("meta-llama/llama-3-8b-instruct");
        agent = new BeeAgent({
            llm, // for more explore 'bee-agent-framework/adapters'
            memory: new TokenMemory({ llm }), // for more explore 'bee-agent-framework/memory'
            tools: [new DuckDuckGoSearchTool(), new OpenMeteoTool()], // for more explore 'bee-agent-framework/tools'
        });
        console.log("Agent created");
    }
    console.log("Running agent");
    const response = await agent.run(
        { prompt: req.body.text },
        {
          execution: {
            maxRetriesPerStep: 3,
            totalMaxRetries: 10,
            maxIterations: 20,
          },
        }
    );
    console.log("Agent ran");

    console.log("Serializing agent");
    req.session.agent = agent.serialize();
    console.log("Agent serialized");

    res.send(response.result.text);
});

app.listen(port, () => {
    console.log(`[server]: Server is running at http://localhost:${port}`);
});

Set-up:

Bee version: [e.g. v0.0.29]
Model provider [e.g. BAM]

Fixed by https://github.com/i-am-bee/bee-agent-framework/commit/58d072f0a16794e96e07302cac8169c74017c14c.

The fix works. Thanks.


Cluster click implementation

Hey all.  I'm trying to implement a way to show a popup list of clustered items when a cluster has been clicked.  Problem is, the cluster factory does not get passed the items that make up the cluster.  So when it is clicked, I don't know which items are in that cluster.
Any chance that the API could be modified such that the factory gets passed the array of markers that make up the cluster?  That would continue to provide the number of points (array length) as well as being able to know which markers make up the cluster.
Thanks for the component!

Hey thanks for the issue it would indeed be possible. I can have a look when I have some time or I would be happy to have some help on this.

Hi, I implemented this feature, with minimal changes to the API (no breaking changes).
https://github.com/alex3165/react-mapbox-gl/pull/306

Release with version 2.2.0, see: https://github.com/alex3165/react-mapbox-gl/releases/tag/v2.2.0


os_sem: possible token overflow

When releasing a semaphore os_sem_release, token count is incremented, but no check is done to handle the case where it will overflow

This was discussed on https://github.com/apache/mynewt-core/pull/1897, we concluded overflowing uint16 in this case was very unlikely.


Changes for compatibility release 0.6.1

Coverage remained the same at 87.062% when pulling<PHONE_NUMBER>75ffdd8508cdd333b98a53bb7851d0 on INM-6:release061 into 7ebd0c8579b3004ab2cea59431beb44aa03bf7f4 on NeuralEnsemble:master.


sfdx sfpowerkit:pool:fetch auto logon feature not working in combination  with --json parameter

When using sfdx sfpowerkit:pool:fetch --tag  --json command the output doesn't contain information
the authorization was successful and is also not executed.
To Reproduce
Steps to reproduce the behavior:
sfdx auth:sfdxurl:store -f user.json --setdefaultdevhubusername --setalias DevHub (login to devhub)
RESULT=$(sfdx sfpowerkit:pool:fetch --tag testpool --json)                                       (claim scratch org)
USERNAME=$(echo "$RESULT" | jq -r '.result.username')                                           (get scratch org username)
sfdx alias:set SO="$USERNAME"                                                                                (set alias)
sfdx force:config:set defaultusername=SO                                                                 (set default)
error: ERROR running config:set: No AuthInfo found for name
Expected behavior
After claiming a scratch org, it is succesful authorized and the defaultusername is correctly set
Screenshots
Desktop (please complete the following information):

OS: Windows
Node Version: v12.18.4
Version of sfpowerkit : v3.1.5
Salesforce Org Version : Summer21 release

Additional context
https://github.com/Accenture/sfpowerkit/discussions/530

While using JSON, the autologin is currently disabled. You might have to use the details in the JSON and authenticate to the org using sfdx auth commands
I will have a look whether to improve it.
@aly76

Thaks @stoevie . We will be fixing this in the next release

Any reason for doing this? Facing this limitation now makes it a real pain to handle errors which come back


two January 2024 dates in screenshots

Two screenshots in this article use January 2024 dates. They should be updated to reflect September 30th 2024. It's confusing othwewise.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 0d9e50bc-931a-80b2-7363-da6eec0643db
Version Independent ID: 03575b7a-0bfd-3b6d-cb49-c00d929e68ea
Content: How to migrate to the Authentication methods policy - Microsoft Entra
Content Source: articles/active-directory/authentication/how-to-authentication-methods-manage.md
Service: active-directory
Sub-service: authentication
GitHub Login: @Justinha
Microsoft Alias: justinha

@W00tFoo
Thanks for your feedback! We will investigate and update as appropriate.

@Justinha
Could you please review add comments on this, update as appropriate.

@W00tFoo
Thank you for bringing this to our attention.
I've delegated this to content author @Justinha, who will review it and offer their insightful opinions.

this is fixed. #please-close


DatePanel with markFocused does not style the active datetime with Calendar set to multiple and range

This is my component
    <Calendar
       style={{ width: "100%" }} // Adjust the width and height as needed
       className="bg-dark"
       multiple
       range
       rangeHover
       format="MM/DD/YYYY HH:mm"
       plugins={[
         <TimePicker hideSeconds position="bottom" />,
         <DatePanel markFocused />,
       ]}
     />

Im guessing markFocus doesnt support multiple range dates in terms of styling, the functionality is there.

#319 my attempt at a solution


Update redirection link

Updated footer by replacing expired link and included portfolio link of @aditya-singh9

Please do not make these kinds of prs and make some useful prs which are adding features to the functionality of the website.
You can just point these mistakes by openeing an issue

Sorry for your inconvenience


Building on Mac OS X

Building the toolchain using make toolchain fails currently. Some issues:

env: python2: No such file or directory
No package manager support

I can work on this and submit a pull request. I'll try not to break existing builds...
The python2 error can be ignored while building the toolchain.

Too many build issues, it's not worth pursuing this...

Yup! Is really complicated adapt the build system to OS X, some time ago I suggested to convert the build system to scons but isn't a aim for the project.

For me, this is a very important issue... I don't like use virtual machines to build the OS, the current workflow is very hard for macOS users!

If you have Mac OS, you can try building on Mac OS. My fork contains the work that I did so far

Ok... I will try see that in the next days

In the end though, it's probably not worth it. Running QEMU in Mac is incredibly slow, due to the lack of KVM acceleration. Terminal is probably fine, but running the compositor slows QEMU down to a crawl.

That's true, but you can use VirtualBox or similar... Now, there is a script for that (run-virtualbox.sh)

Note that script relies on grub-mkrescue, which is probably not a fun thing to install on macOS. We could probably adapt the automated package installation for homebrew. As for why some other things aren't building, that's a tougher problem to tackle.

GRUB is actually pretty easy to build from source. It's just cross
compiling things that's really hard.
On Dec 9, 2016 6:52 PM, "Kevin Lange"<EMAIL_ADDRESS>wrote:
Note that script relies on grub-mkrescue, which is probably not a fun thing
to install on macOS. We could probably adapt the automated package
installation for homebrew. As for why some other things aren't building,
that's a tougher problem to tackle.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/klange/toaruos/issues/136#issuecomment-266163541, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AMKdXBefPKXsVS672UvSmLUK_OFzwDh9ks5rGfezgaJpZM4LJH0v
.


Bug fix profile page and moved the info banner from App into Signin and Signup

A fix to solve the disabled state of the Save button in the Profile page when using OAuth, moved the info banner to signin and signup to prevent the banner from rendering in the review page.

Thanks, I will need some help to fix the oauth button problem in the main branch as well. The infobar will be return in a different shape and doesn't need to be ported.


Add full support of horizontal scrolling

Full support of horizontal scrolling, as well as relative function.
scroll_to_... functions are all using tween now.
I changed set_h/v_scroll() to get_h/v_scroll_bar().set_value_with_no_signal() and queue_redraw, as the former function tries to pull content back to boundary, which caused flicker like #2.

Might not be a large issue but the horizontal over-drag seems to snap back further than the vertical one.
https://user-images.githubusercontent.com/57133330/216006112-d2fc0fa6-2413-4645-acf8-cc7de84a65d8.mp4

Hi, SpyrexDE. It's actually the size of scroll bar and I fix it. And I simplify bounce methods.

Using the newest Godot Beta (v4.0.beta16.official [518b9e580]) I get some really weird behavior:
https://user-images.githubusercontent.com/57133330/216141057-286a22ac-9c41-4533-a4c6-196a5e7872f1.mp4

I'm using the same version, but I didn't find that issue. Have you checked the content node minimum size? Or is that convenient to provide a minimal reproduction project?

Yeah, I just edited the scroll container in example.tscn to have a wider minimum x-size, so it would create an horizontal scroll bar.

Perhaps I misunderstood something. Is that any weird behavior?

How would you go about creating a horizontal scroll? Normal scrollbars work with the control's minimum size as well.

I guess you mean you don't want h scroll bar even if content node is wider than scroll container? Creating scroll bar is actually original scroll container behavior, which is controlled by Horizontal/Vertical Scroll Mode.

No, sorry for the communication issue. What I wanted to point out is that depending on what Godot 4 Beta I am using there appears to be a bug. I opened exactly the same project once using Beta 5 and once using Beta 16, but the results are different:
Beta 5:
https://user-images.githubusercontent.com/57133330/216604255-787f8dc7-a020-4b95-8a98-ba293432e953.mov
Beta 16:

Ahh okay, that was exactly it. Thank you very much :)


RuntimeException - Too many arguments

When I run the code, I get the following error from the cron:
Symfony\Component\Console\Exception\RuntimeException]    Too many arguments.

What is the command you are trying to run?

/usr/bin/php mysqldumper.phar --keep-local >/dev/null 2>&1

I am using the master version.

What happens if you try without the /dev/null ?

Same issue with just /usr/bin/php mysqldumper.phar.

When I run just /usr/bin/php mysqldumper.phar, I now get this error:
 No mysqldumper.json found

mysqldumper [--dir DIR] [--keep-local] [--skip-remote] [--ignore-table IGNORE-TABLE] [--self-update]```

Even though `mysqldumper.json` is in the same directory as `mysqldumper.phar`.

Vey weird. We are running this no problem on some production servers.
Can't look into it at the moment, as I'm away for the festive period and don't have access to my dev machine.
Will take a look when I'm home in a few days.
Let me know if you figure anything out in the mean time
Thanks

@shefik Did you get anywhere with this?

Not yet.


Initial state not working and not able to be set.

Library Version : com.github.skydoves:flexible-bottomsheet-material3:0.1.2
Affected Device(s) : Pixel 8 Pro SDK 34

Initial state not working or not able to be set.
Tracing the library code, calling rememberFlexibleBottomSheetState calls a private function rememberFlexibleSheetState which has a default parameter of initialValue: FlexibleSheetValue = FlexibleSheetValue.Hidden. The public method does not take that parameter, however, the initial state should evaluate to hidden regardless.
In my most basic implementation, the sheet always starts off a not hidden, intermediately expanded.
 val sheetState = rememberFlexibleBottomSheetState(
      isModal = false
 )

FlexibleBottomSheet(
        windowInsets = WindowInsets(0, 0, 0, 0),
        onDismissRequest = { },
        sheetState = sheetState,
      ) {
        CircularProgressIndicator()
      }
}

It's not clear how to set the initial state, or if it's supported. Am I missing something obvious?
Wrapping the entire FlexibleBottomSheet in an if statement and check some state before showing works, but then it gets awkward to have a nice animation to hide it when that state changes, it would just abruptly disappear.
Expected Behaviour:

there is a way to set the initial state value
initial state value is respected.

I had the same problem, any idea how to solve this @skydoves ?
Thanks for your time!

Any updates on this issue?

Seeing the same issue on 0.1.5, the state gets a signal to be intermittently expanded immediately


Fix: Rename docs to catalog to keep routes consistent

Changing related routes from docs to catalog
both cli commands sqlmesh ui --mode docs and sqlmesh ui --mode catalog spin up SQLMesh UI in docs mode but with routing name catalog
Changing internal references form Docs to Catalog

Does our documentation need to be updated to reflect the renaming of docs to catalog?

so sqlmesh ui --mode docs should work as before and change mostly internal, and i don't think we have any references to/docs  url

Verified opening sqlmesh ui/docs/catalog routes to correct page.


Custom colors not working

I have the following config:
import { defineConfig } from 'vite-plugin-windicss'
import colors from 'windicss/colors'
import typography from 'windicss/plugin/typography'

export default defineConfig({
  darkMode: 'class',
  plugins: [
    typography(),
  ],
  theme: {
    extend: {
      colors: {
        primary: colors.blue,
        secondary: colors.amber,
        harmonydark: {
          50: '#f4f4f5',
          100: '#e9e9eb',
          200: '#c7c7cc',
          300: '#a5a5ad',
          400: '#626270',
          500: '#1f1e33',
          600: '#1c1b2e',
          700: '#171726',
          800: '#13121f',
          900: '#0f0f19',
        },
      },
    },
  },
})

However, autocomplete does not work for harmonydark, primary, or secondary.

@Bluskript cannot reproduce

Could you try reproducing this in a new project setup with the Vitesse template? My project is based on that.

can you share your project, then i can test? setting up projects is out of scope for us

oh. I think this issue caused by windi/tailwind.config.ts, current typescript config not support yet.

@voorjaar also it is that in your config u are usingimport / exportand not require / module.exports I am looking for the issue we had about this

@Bluescript https://github.com/windicss/windicss-intellisense/issues/24


[vcpkg] Enhance detection of Anti-Virus messing with builds

I recently encountered AV related build failures and wondered why the retry logic in vcpkg_execute_build_process.cmake didn't kick in. The error message generated on the failing machines includes the mt-executable's file-extension. The detection logic previously accounted for this until the function was overhauled in #18838 / #18488 / 6c9da67fba1d19c2e09ddf02f2d0e4ec1ecc124c. However, I believe this to be an oversight.

/azp run

the line was while (ITERATION LESS 3 AND (out_contents MATCHES "mt : general error c101008d: " OR out_contents MATCHES "mt.exe : general error c101008d: ")), contemplating both cases. It has been overhauled with only one and patched now with only the other... I suppose the original was still the best one...


chore(deps): upgrade dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-2.x" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 374

Questions ?
Please refer to the Backport tool documentation


feat(filter-button): add filter button variant to radio comp

add filter button variant to radio comp
feat #376
Description
Add filter button 'select-one' variant as a variant for the existing Radio component.
Fixes # (issue)
Requirements
Storybook link: https://deploy-preview-36--legal-and-general-canopy.netlify.app/?path=/story/components-filter-button--select-one
Design link: https://legalandgeneral.invisionapp.com/share/2SZ192MTHZY#/432421864_Filter_Buttons
Screenshot:

Checklist:

[x] The commit messages follow the convention for this project
[x] I have provided an adequate amount of test coverage
[x] I have added the functionality to the test app
[x] I have provided a story in storybook to document the changes
[x] I have provided documentation in the notes section of the story
[ ] I have added any new public feature modules to public-api.ts
[x] I have linked the new component to adobe DSM (if appropriate)

Looks great @AlexandruHoisan a few comments/ideas, but nothing to contentious I hope. The approach looks good.
Does it need to be a breaking change though? I wasn't sure if anything we did here would affect an existing application?
For anyone new to this, we are aware that it carries a slight overhead in that you will end up importing all of the style variants even though you are only using one. We looked at component inheritance as a way to get round this, but in the end favoured the simplicity of this approach. I think we should document the downside in an issue though, then at some stage in the future we may be able to find a way of only including the css that is needed

It actually just affects the segment PR. Sorry about the misleading breaking change.

I'll pop a 'request changes' on there, just to make sure we don't forget and accidentally merge the breaking changes commit msg. 😁

rewrote the message and got rid of the Breaking changes from it. 👍

Just wondering what feat #376 is in the commit message?

Just wondering what feat #376 is in the commit message?

I was referencing the segment changes but I realise that does not make sense to have it here.
I'll take it out.


insert into select bug

CrateDB version:
3.2
Environment description:

JVM version: java1.8
Distribution: centos 6.5
Number of nodes; 4 node

Problem description:
table a 200 record
table b 0 record
insert into b select * from a limit 10
and the result is 40 records has been insert !!!!(my cluster has 4 nodes)
What's the matter?

Could you post the exact query you're using? Currently the query clause of a INSERT INTO isn't allowed to contain a limit at all. The query you posted would result in an error:  Using limit, offset or order by is not supported on insert using a sub-query

insert into b select * from (select * from a limit 10)c

Thank you for the bug report, we've a fix for this (https://github.com/crate/crate/pull/8555) which will be part of the next 3.2 and 3.3 releases.


Allow launching multiple instances

Is your feature request related to a problem? Please describe.
I find this app useful to check if two files are identical, right now I have to copy two hashes into a text editor and compare them there.
Describe the solution you'd like
Allow opening multiple windows of the app.

Thanks for raising this issue!
Allowing multiple instances does sound interesting and I'll definitely get on it, however your use case seems to be already covered:

Open the first file by clicking on the "Open File" button in the welcome view

Select the verify tab from the headbar:

Click on the button under the title "File" and select the file you want to check against:

If that is not your use case, please explain it below!
If that was your use case, it appears that the current design of the Verify view is not clear enough...
(just to explain the usage of the verify tab: they are tools to check against the current open file
the text filed checks if the input matches any of the hashes of foo.txt
the file button checks if a hash of the selected file matches any of the hashes of foo.txt)

Oh! I guess I'm just blind and didn't check out the verify tab in a hurry, otherwise it's pretty obvious what this tab does. Anyway, I think multiple instances is still a good thing to have

With the ability to open files directly from the command line, you are “theoretically” able to open multiple files with Collision.
But for now, the program just doesn't handle it correctly!
It seems that launching a new instance of Collision......merges the window content with the already-existing window! xD

First file
Second file
Sum of both

Nice catch! Thanks for the report!
It should be fixed as of #124


DNS not working properly in Docker

Server:
# working
docker run -it --rm -v /dev/net/tun:/dev/net/tun --sysctl net.ipv6.conf.default.disable_ipv6=0 --add-host host.docker.internal:host-gateway --cap-add NET_ADMIN --name tun2proxy tun2proxy --proxy socks5://host.docker.internal:7930 --dns-addr <IP_ADDRESS> --dns virtual

# not working
docker run -it --rm -v /dev/net/tun:/dev/net/tun --sysctl net.ipv6.conf.default.disable_ipv6=0 --add-host host.docker.internal:host-gateway --cap-add NET_ADMIN --name tun2proxy tun2proxy --proxy socks5://host.docker.internal:7930 --dns-addr <IP_ADDRESS> --dns over-tcp

# not working
docker run -it --rm -v /dev/net/tun:/dev/net/tun --sysctl net.ipv6.conf.default.disable_ipv6=0 --add-host host.docker.internal:host-gateway --cap-add NET_ADMIN --name tun2proxy tun2proxy --proxy socks5://host.docker.internal:7930 --dns-addr <IP_ADDRESS>

Client:
docker run --rm -it --network "container:tun2proxy" alpine:3.7

Note that even if --dns virtual works it will not reveal the real IP of public domains.

Hmm. I think the problem locate at --proxy socks5://host.docker.internal:7930 paramter. Need more thinking.

Hmm. I think the problem locate at --proxy socks5://host.docker.internal:7930 parameter. Need more digging.
Here can not work.
https://github.com/tun2proxy/tun2proxy/blob/48f527ad81f5c114ca0159d037a655977526f446/src/args.rs#L332-L336
I hope you can debug in your Docker environment and then provide further feedback information.

@James4Ever0
如果你在启动命令里加上 --privileged 将会正常启动。这里失败的原因是bind mount需要增加Linux cap，docker container默认没有这个cap。
即使能够正常运行，--privileged 是一个危险的选项，这将会让docker container拥有完整root权限，在生产上是十分危险的。
@ssrlive  我查询了Linux Cap表，Bind mount需要的cap是SYS_ADMIN，但是我加上了 --cap-add SYS_ADMIN 但是没有效果。所以我有个疑问，在这个 [2024-07-30T05:11:52Z WARN  tproxy_config::linux] failed to bind mount custom resolv.conf onto /etc/resolv.conf, resorting to direct write 日志打印的时候是否调用了其他Linux cap。

@ssrlive
假设增加 SYS_ADMIN 这个权限可以正常运行，尽管如此，CAP_ADMIN 这个能力的范围非常大，权限太高，要谨慎操作。也许，判断在docker内的话可以通过echo "nameserver <IP_ADDRESS>" > /etc/resolv.conf 而不是 bind mount。
Linux Cap没有更小颗粒度的能力控制...但也许...我没找到？

CC @one-d-wide

Would like to borrow some code from tun2proxy
My use case scenario can be viewed here

[@Paper-Dragon's comment translated by google]
I checked the Linux Cap table, and the cap required for bind mount is SYS_ADMIN, but I added --cap-add SYS_ADMIN but it didn't work. So I have a question, whether other Linux caps are called when this [2024-07-30T05:11:52Z WARN tproxy_config::linux] failed to bind mount custom resolv.conf onto /etc/resolv.conf, resorting to direct write log is printed.

Tun2proxy dosn't need a mount capability to work. You can see in the logs that after an unsuccessful attempt it resorting to direct write. And that's what actually fails (though the precise cause of the error isn't logged properly). So making the /etc/resolv.conf writable by a tun2proxy process should probably be sufficient to make it work.
But I guess we may want to reconsider hard failing on this step. And also actually document this use-case,
instead of just pointing to the system docs, since suggesting to run a network-facing application as a root is just asking for a trouble.

Assuming that adding the SYS_ADMIN permission can run normally, however, the scope of the SYS_ADMIN capability is very large and the permissions are too high, so be careful. Maybe, if you are in docker, you can use echo "nameserver <IP_ADDRESS>" > /etc/resolv.conf instead of bind mount.
Linux Cap does not have a more granular ability control... but maybe... I didn't find it?

FYI separating mounting into a separate capability is kind of pointless, since with an unrestricted ability to do mounts on the global fs tree, you can basically hijack any executable or config file.

Server:
# working
docker run -it --rm -v /dev/net/tun:/dev/net/tun --sysctl net.ipv6.conf.default.disable_ipv6=0 --add-host host.docker.internal:host-gateway --cap-add NET_ADMIN --name tun2proxy tun2proxy --proxy socks5://host.docker.internal:7930 --dns-addr <IP_ADDRESS> --dns virtual

# not working
docker run -it --rm -v /dev/net/tun:/dev/net/tun --sysctl net.ipv6.conf.default.disable_ipv6=0 --add-host host.docker.internal:host-gateway --cap-add NET_ADMIN --name tun2proxy tun2proxy --proxy socks5://host.docker.internal:7930 --dns-addr <IP_ADDRESS> --dns over-tcp

# not working
docker run -it --rm -v /dev/net/tun:/dev/net/tun --sysctl net.ipv6.conf.default.disable_ipv6=0 --add-host host.docker.internal:host-gateway --cap-add NET_ADMIN --name tun2proxy tun2proxy --proxy socks5://host.docker.internal:7930 --dns-addr <IP_ADDRESS>

Client:
docker run --rm -it --network "container:tun2proxy" alpine:3.7

Note that even if --dns virtual works it will not reveal the real IP of public domains.

I can't reproduce this with Docker running under root and I don't have any intuition on why it would work with virtual DNS while not working with other DNS modes because they all rely on being able to intercept DNS requests to the server . Is Docker running rootless in that case? Please provide /etc/resolv.conf contents and log outputs for all cases. Ideally, you would provide a Vagrantfile allowing me to reproduce your issue.

Hmm. I think the problem locate at --proxy socks5://host.docker.internal:7930 parameter. Need more digging.

I don't think this is the problem. This name resolution should work through /etc/hosts inside the container.
Apart from that, please pull the latest container image. Maybe that already solves it.

Regarding the privilege discussion: tun2proxy needs privileges for at least the following:

tun device creation (CAP_NET_ADMIN),
routing table manipulation (CAP_NET_ADMIN) with --setup, and
modifying /etc/resolv.conf with --setup (CAP_SYS_ADMIN for mount or CAP_DAC_OVERRIDE for direct writing, both effectively mean full root).

Convenience comes at a cost. You can use tun2proxy with an already existing tun interface and you do not have to use --setup. I know a thing or two about security, so in 15703a4823fe9ec2468eb5078b0e87d45c17cae8, I implemented a privilege drop such that the network-facing part does not run with root privileges:
https://github.com/tun2proxy/tun2proxy/blob/15703a4823fe9ec2468eb5078b0e87d45c17cae8/src/main.rs#L84
However, that seems to have gotten lost in one of the refactorings. I don't currently have the time to re-implement that. (The solution was a quick shot and not ideal anyway. Instead of changing the user ID, the capabilities should have been dropped, so it could still have run as UID 0 but without capabilities.)

It is more than just the DNS problem. For the same socks5 proxy tun2socks is able to correctly route traffic while tun2proxy unable to.
I can demonstrate the issue here.


Wrong behavior with translate_tabs_to_spaces for non-ascii characters

Found in eyelash/xi-gtk#47
When pressing tab for table alignment, an incorrect number of spaces is inserted when the line contains multi-byte character sequences.
To reproduce:
Make sure translate_tabs_to_spaces = true
Open an editor window and insert a<tab>a<enter> æ<tab>a<enter> €<tab>a<enter> 😃<tab>a<enter>
Observed behavior:
The editor buffer contains
a   a
æ  a
€ a
😃    a

This is rather obviously wrong
Fixing this properly is tricky, because some code points are two characters wide (like CJK characters and emoji) or zero-width, but I believe counting a code point as one character is better than counting bytes.
Ideally, this would be expected:
a   a
æ   a
€   a
😃  a

but I think this is also acceptable:
a   a
æ   a
€   a
😃   a

Wouldn't it be even better to do this on a grapheme level instead codepoint?
I would like to take a stab at this. First I'll look how others do it.

@mqrzy the final version of this should be using measured width; see #565 for a detailed writeup on that problem.
The simplest interim solution would be to switch from bytes to chars, which will still be wrong but will be much less wrong than the current behaviour. Switching to graphemes would also be an option, as would using unicode width. All of these solutions are working on the assumption that we can measure width of substring units independent of context, which isn't true in the general case, but I think is fine in the code-editing case. Auto-indent is a code-editing feature, so I wouldn't get too hung up on the problem of width measurement generally?

Why is this fine in the code-editing case?
I'll think about unicode-width vs graphemes and implement some interim solution

My thinking is that the code editing case is generally ascii-heavy, and generally uses fixed-width fonts; if we assume that graphemes have equal width in this case we'll get in trouble occasionally, but I think it would be acceptable until real width measurement lands.


Unable update service task names

Describe the bug

Unable update service task names
Steps to reproduce

Create a workflow, add a service task and save.
Modify the name of the service task step.

Result

The name did not change. But the approval task name can be updated succefully.

Expected behavior

Environment

Walrus version: main 065ee92e55e14044fa0777be4c631f7082fadc0e
UI version: dev-b9a6538

Additional context
Add any other context about the problem here.

It displays the task name as a service name, so the task name update fails.

Walrus version: main a3e5c0343d3e831ca30e4fd76a160212bc5bba7f
UI version: dev-4b1fd17

Test Result:  It sometimes jump to the first step after clicking save or sometimes returns an error.

Environment

Walrus version: main a3e5c0343d3e831ca30e4fd76a160212bc5bba7f
UI version: dev-6b82ce8

Test Result: pass


Allow bound method as event handler

Also, in a case like this, tests would be appreciated to highlight if the existing code for this has regressed

@adhami3310 I added a test which is similar to our regression. We are using smth like this for quite a while now and it works fine. However, this is not really a documented/supported feature of reflex. Hence, this PR could be seen as a feature, not a regression bug fix

Since it's similar to the lambda feature, I think this one makes sense. Although it would have been cooler to unite it with that other path.


Change CDN?

Is it possible to change the default template with a specificated CDN URL?

File | Settings | Editor | Live Templates   bootstrap  b4-$ edit variables

@cyanBone Thanks, but your solution seems related to a JetBrains's IDE, I was wondering to do this in VSCode.


GNOME updates 2022-04-01

gnome.aisleriot: 3.22.21 → 3.22.22
https://gitlab.gnome.org/GNOME/aisleriot/-/compare/3.22.21...3.22.22

Look for themes in .local/share/aisleriot/cards. (#268)
Fix build with default guile version we use. (#925)

gnome.five-or-more: 3.32.2 → 3.32.3
https://gitlab.gnome.org/GNOME/five-or-more/-/compare/3.32.2...gnome-3-32 (there is https://download.gnome.org/sources/five-or-more/3.32/LATEST-IS-3.32.3 but this is not tagged on gitlab)

Fix build with latest vala (#26)

gnome.gucharmap: 14.0.2 → 14.0.3
https://gitlab.gnome.org/GNOME/gucharmap/-/compare/14.0.2...14.0.3

Just translations

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

@ofborg build gnome.aisleriot gnome.five-or-more gnome.gucharmap


Remove was_informed_by from WorkflowExecution and update data examples

This PR provides:

removal of was_informed_by slot from WorkflowExecution
update to valid data examples
add invalid data example
delete .json file from data/valid

migration impacts all WorkflowExecution(Activity) sets that use was_informed_by. was_informed_by slot and value will move to new WorkflowChain class. For migration

Can not remove this until WorkflowChain is created. See @aclum's comment in https://github.com/microbiomedata/nmdc-schema/issues/1537#issuecomment-1875968097


cards show complete post, not just first lines

Hi,
when I write a long post the cards show all text and not just the first lines. Please help howto change that.
What code goes into the post to make it  go right?

 Please add that to the archetype!
Description  : "The Markdown Guide is a free and open-source reference guide that explains how to use Markdown, the simple and easy-to-use..."

Updated.


Sell orders are not executed

Hello, when it is ready to sell, the amount it tries to sell is always more than the available amount in my account. How can I solve this type of problem? For example, Iin coins_bought.json file the volume is 37.26 and in my account the amount I own is 37.23, something like that.

So basically the difference is the transaction fee, how can I overcome this problem?


fix: Handling expired APNS certificates properly

At the moment Apns(ConnectionError(hyper::Error(Io, Custom { kind: InvalidData, error: "received fatal alert: CertificateExpired" }))) causing a 500 HTTP error.
We should handle properly expired certificates instead of throwing 500 errors.

When this error happens suspend the tenant: https://walletconnect.slack.com/archives/C058RS0MH38/p1710726001743679
This will result in a 202 response code not a 5xx.


refactor(transformer): add more specific methods to ModuleImportsStore

Make ImportKind private implementation detail of ModuleImports common transform. Provide add_default_import and add_named_import methods instead.

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#6561
#6560  👈
#6559
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @overlookmotel and the rest of your teammates on  Graphite

@Dunqing This is purely a matter of taste, and I'm not tied to it. If you prefer how it was before, please just say so.


Xwayland hidpi

Hi!
The xwayland server code in wlroots was split in https://github.com/swaywm/wlroots/commit/27609ba0d9d140660f0cc077ad7af8670fa0ed21.
This PR rebases the xwayland_hdipi branch on wlroots latest commit.
The changes were not meaningfully substantial, I’ve added myself to the first commit as co-author but I’d be fine with a signoff if you prefer.
You could force-merge the PR.
Thanks!

Sorry, opened against the wrong repo.


Update jasmine dependency to allow users disable default reporter

Please update jasmine dependency to 2.5.2
It allows clearing default reporters so adding custom one won't show duplicated reports in console.

We are waiting a new release from jasmine. The previous discussion starts here. https://github.com/angular/protractor/issues/3811. Created a new issue to jasmine-npm since the hot fix is not out yet. See:  https://github.com/jasmine/jasmine-npm/issues/102.
Leaving this issue open since the duplicate version was previously closed.


Drop ORDER BY clause from copy step when adding a limit

Fixes

Fixes #4381 by @krysal
Description

As explained in the issue, we're ditching this clause here.
Testing Instructions

Set the environment variable for the limit to a number lower than 5000. Eg.:

DATA_REFRESH_LIMIT=1000

Start a data refresh

just api/up && just docker/es/delete-index image-init && \
just ingestion_server/ingest-upstream

Wait for the process to finish; locally, it is very quick (less than a minute), and promote the table.

just ingestion_server/promote

Verify the number of images is correct with just api/pgcli and

SELECT COUNT(*) FROM image;

Checklist

[x] My pull request has a descriptive title (not a vague title likeUpdate index.md).
[x] My pull request targets the default branch of the repository (main) or a parent feature branch.
[x] My commit messages follow best practices.
[x] My code follows the established code style of the repository.
[x] I added or updated tests for the changes I made (if applicable).
[ ] I added or updated documentation (if applicable).
[x] I tried running the project locally and verified that there are no visible errors.
[x] I ran the DAG documentation generator (just catalog/generate-docs for catalog
PRs) or the media properties generator (just catalog/generate-docs media-props
for the catalog or just api/generate-docs for the API) where applicable.

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

@zackkrida It works for me, where did you set DATA_REFRESH_LIMIT? I forgot to add that it should go in the ingestion_server/.env file.


Missing cs files

when I load this project with my visual studio, there come many errors about missing .cs files?!  Would you please release these files?!

when I load this project with my visual studio, there come many errors about missing .cs files?! Would you please release these files?!

My apologies for the delay getting to this. The branch master has now been repaired.


Disable trackupstream for master for now since we are focusing on

Liberty
Keeping this disabled for the next month or so will help us
on focusing on getting the Liberty release out.

Makes sense for me: +1

+1

On the other hand I'm a bit worried that this will make it harder to get Master back to green later... and increase the disconnect towards upstream. e.g. we would not have discovered https://bugs.launchpad.net/bugs/1520096

@bmwiedemann but is only during a month, even if the PR is not merged, not many people will be tracking the status when we are prioritizing other tasks

@bmwiedemann and Master is broken anyway. Maybe the packages build but cleanvm is red since weeks.


3rd party static libraries: Integrate Cocoapods with the CLI Plugins Mechanism

This will require us to support build from the Xcode workspace the cocoapods generate.
We will have to somehow generate metadata for the cocoapods. The best solution would be to generate module maps from the podspecs.
We will have to reasearch what part of the cocoa apis can we consume (e.g there may be C apis that get stripped during the linking, although cocoapods add -ObjC flag)
Later this issue will probably be integrated with the NPM plugins mechanism, and allow pods to be specified in the package.json or a podfile in a plugin.

Moved to: https://github.com/NativeScript/nativescript-cli/issues/729


Feat: Add documentation for Azure Resource ID configuration

Document the new configuration options resulting from the effort to include the x-ms-AzureResourceId header in post requests to the azure log handling service.
#178417450

Thank you @ctlong! This looks great. I'm seeing that this is still marked as a "work in progress". Should I hold off on merging?

This PR is blocked in order to sync up with an upcoming azure nozzle release.

Thank you!


[Bug]: Operators are impacted by downloading and network greatly

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
Operators are impacted by downloading and network greatly
Running operator tests are failed for lack of files or others errors caused by downloading failed frequently.
Fail to run lightningdot - {'model_name': 'lightningdot_base'}
ERROR MESSAGE: unexpected EOF, expected 4305660 more bytes. The file might be corrupted.
Fail to run lightningdot - {'model_name': 'lightningdot_coco_ft'}
ERROR MESSAGE: PytorchStreamReader failed reading zip archive: failed finding central directory
Fail to run lightningdot - {'model_name': 'lightningdot_flickr_ft'}
ERROR MESSAGE: PytorchStreamReader failed reading zip archive: failed finding central directory

Expected Behavior
Could we do some special process for downloading large files?
Or it will be impacted by downloading and network greatly
Steps To Reproduce
Running the following script
import logging
import os
#import ipdb
import subprocess
from towhee import ops

logger = logging.getLogger()
log_path = 'image_text_embedding.log'

image_text_embedding = {
    'lightningdot': [
        {'model_name': 'lightningdot_base'},
        {'model_name': 'lightningdot_coco_ft'},
        {'model_name': 'lightningdot_flickr_ft'}
    ]
}

def test(data_path):
    if not os.path.isfile(data_path):
        subprocess.call(
            f'wget -O {data_path} https://bair.berkeley.edu/images/BAIR_Logo_BlueType_Tag.png',
            shell=True
        )

    image_decoder = ops.image_decode.cv2()
    # Since the image_decoder returns generator, need to extract output from generator for repetitive usage
    #input_data = [x for x in image_decoder(data_path)]
    input_data = image_decoder(data_path)
    handler = logging.FileHandler(log_path)
    logger.addHandler(handler)
    #ipdb.set_trace()
    for k, v in image_text_embedding.items():
        init_op = getattr(ops.image_text_embedding, k)
        for args in v:
            try:
                op_img = init_op(**args, modality='image')
                img_emb = op_img(input_data)
                op_text = init_op(**args, modality='text')
                text_emb = op_text('using a test description')
                assert img_emb.shape[0] > 0
                print('{}, embedding shape is {}'.format(k, img_emb.shape))
                assert img_emb.shape == text_emb.shape
            except Exception as e:
                logger.error('Fail to run %s - %s', k, str(args))
                print(f'ERROR MESSAGE: {e}')
                pass

if __name__ == '__main__':
    test('bear.jpg')

Environment
- Towhee version(e.g. v0.1.3 or 8b23a93): latest
- OS(Ubuntu or CentOS):
- CPU/Memory:
- GPU:
- Others:

Anything else?
No response

we plan to redesign the operator management system in the next version, and the downloading and dependency problems should be solved in that release.


Using specific Spring request annotations where possible

Closes #2161
Tiny change to use specific Spring request annotation where possible.
Behavior not changed.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.David Rodriguez seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.

Aside from fixing the build, please make sure this style is included in the rules.

@mmoayyed, that's interesting, this morning with a different PC it master was failing for the same reason and I thought that it was some configuration problem, but now master is failing with error:
"null value in entry: archivePath=null"
any ideas?
It doesn't make sense this errors, I didn't change any test

Right, it's not your fault. It's gradle having trouble pulling snapshot dependencies from hazelcast. Ignore.
So just to confirm:

We have all annotations [that make sense] switched over to the new model
And we have the build failing if it find anything contradicting those annotations

Correct?

Yeah. As far as I know, @RequestMapping if doesn't specify a method, creates endpoints for all the HTTP methods. So, I created the rule looking for that methods and replaced with the "new" annotations that @RequestMappings with only 1 HTTP method specified
Just to be sure, could the rule match anything not related with this? I mean, what about changing the regex to something like @RequestMapping.*method = RequestMethod

Well this @RequestMapping.*method = RequestMethod is possibly good, but I think it may not capture all cases where you might just have  = GET where you have a static import on RequestMethod perhaps. I should note that we obviously don't have to get this perfect on the first try; there will be cases where this rule might end up clashing with the normal and we'll just accommodate the pattern right then.
I was actually hoping we might be able to define a pattern to capture all @RequestMapping instances but that might be too aggressive, right?

Yeah, I think that too. Let's stay like this and modify it in the future if needed

Very nice. Thanks!


MHub / GC - Add grt123 Model for lung cancer prediction based on lung nodules

Hi, this PR contains the required code for getting our modified version of the publicly available grt123 model, winner of the Data Science Bowl 2017 Kaggle challenge.
Caveats

The PR target is main, but should be something like m-gc-grt123-lung-cancer
In the Dockerfile the MHub model integration is currently marked TODO since it requires the creation of the appropriate branch for this code first.
This implementation still uses a run.py script because the MHAConverter doesn't have the panimg backend yet.

Algorithm I/O

Input should be a CT Lung image (MHA) (This MHub implementation expects a Dicom which is converted to MHA using MhaPanImgConverter)
Output is a JSON file containing the predicted nodule findings (locations) and predicted cancer probability scores.
Test images for this algorithm can be found here

This PR has been updated for the new base image.

Some suggestions

For the final prediction score, add a ValueOutput to the Runner module
Remove thetmp_path config
Use requestTempDir for temporary folders (example)
Remove custom gpu checks if not vital for the model runner (we may implement such a feature globally in mhubio, let's discuss this in our next meeting!)

I just updated and cleaned up this PR:

default.yml

renamed from config.yml
added version and description
updated pipeline with panimg mhaconverter

Dockerfile

added fixed commit hash for grt123 repo git clone
updated entrypoint

LungCancerClassifierRunner.py

removed tmp_path config option
added requestTempDir for tmp_path
added more comments

Removed script files and custom PanImgConverter

This satisfies all but two of your suggestions:

Remove custom gpu checks if not vital for the model runner (we may implement such a feature globally in mhubio, let's discuss this in our next meeting!)

Is required for the model to know how the number of GPUs it has available, so I left it in.

For the final prediction score, add a ValueOutput to the Runner module

This is tricky, since it outputs quite a detailed report per scan (see below). Maybe it is better to leave it as is?
{
    "lungcad": {
        "revision": "9a4ca0415c7fc1d3023a16650bf1cdce86f8bb59",
        "name": "grt123",
        "datetimeofexecution": "09/13/2023 12:28:46",
        "coordinatesystem": "World",
        "computationtimeinseconds": 48.483961
    },
    "imageinfo": {
        "dimensions": [
            512,
            512,
            140
        ],
        "voxelsize": [
            0.820312,
            0.820312,
            2.5
        ],
        "origin": [
            -228.800003,
            -210.0,
            -379.0
        ],
        "orientation": [
            1.0,
            0.0,
            0.0,
            0.0,
            1.0,
            0.0,
            0.0,
            0.0,
            1.0
        ],
        "seriesuid": "dicom"
    },
    "findings": [
        {
            "id": 0,
            "x": -46.800003000000004,
            "y": -31.0,
            "z": -168.0,
            "probability": 0.9999904632568359,
            "cancerprobability": 0.75371253490448
        },
        {
            "id": 1,
            "x": 73.199997,
            "y": 79.0,
            "z": -191.0,
            "probability": 0.999943733215332,
            "cancerprobability": 0.6662029027938843
        },
        {
            "id": 2,
            "x": 24.199996999999982,
            "y": -47.0,
            "z": -171.0,
            "probability": 0.9999247789382935,
            "cancerprobability": 0.47204485535621643
        },
        {
            "id": 3,
            "x": -82.800003,
            "y": 108.0,
            "z": -159.0,
            "probability": 0.8182298541069031,
            "cancerprobability": 0.004258527886122465
        },
        {
            "id": 4,
            "x": 49.199996999999996,
            "y": -67.0,
            "z": -275.0,
            "probability": 0.7932767271995544,
            "cancerprobability": 0.014665956608951092
        },
        {
            "id": 5,
            "x": 69.19999700000001,
            "y": -63.0,
            "z": -287.0,
            "probability": 0.6662660241127014,
            "cancerprobability": 0.0054708984680473804
        },
        {
            "id": 6,
            "x": 126.199997,
            "y": 28.0,
            "z": -334.0,
            "probability": 0.6302552819252014,
            "cancerprobability": 0.008180161006748676
        },
        {
            "id": 7,
            "x": -62.800003000000004,
            "y": -4.0,
            "z": -102.99999999999999,
            "probability": 0.48750755190849304,
            "cancerprobability": 0.00867788027971983
        },
        {
            "id": 8,
            "x": 73.199997,
            "y": 80.0,
            "z": -215.0,
            "probability": 0.36484286189079285,
            "cancerprobability": 0.03693072870373726
        }
    ],
    "cancerinfo": {
        "casecancerprobability": 0.9574154615402222,
        "referencenoduleids": [
            0,
            1,
            2,
            3,
            4
        ]
    }
}

NOTE: This PR proposes a model currently using a dynamic length output. We are working on a method to export dynamic length outputs and will likely introduce a @IO.Out.Data.Many decorator (similar to our @IO.Output -> @IO.Outputs decorators).

NOTE: This PR proposes a model currently using a dynamic length output. We are working on a method to export dynamic length outputs and will likely introduce a @IO.Out.Data.Many decorator (similar to our @IO.Output -> @IO.Outputs decorators).

This is tricky, since it outputs quite a detailed report with multiple scores per nodule per scan (see below). Maybe it is better to leave it as is? Or maybe we could only add a single cancer probability ValueOutput for the whole scan (last probability value in the report after cancerinfo)).

Let's do both, or actually all three :)

Export the original JSON file
Use a Value Output for the overall score
Use a second, dynamic Value Output for all findings.

Dynamic Value Outputs are supported by the newest MHub-IO release now :)
The documentation can be found here and are fully supported in our ReportExporter Module by using an aggregate function (similar to files) but with some additional value operations.
An example implementation would look like this:
@ValueOutput.Name('lnrisk')
@ValueOutput.Label('Lung Nodule Risk-Score.')
@ValueOutput.Type(int)
@ValueOutput.Description('The predicted risk score for a single lung nodule detected by the alggorithm.')
class LNRisk(ValueOutput):
   pass

def getLungNodulesRiskScores(dicom_dir) -> List[int]:
   # ... find lung nodules, and report back an array of risk scores
   return lst_scores

class MyModule(Module):

   @IO.Instance
   @IO.Input('in_data', 'dicom:mod=ct', the='chest CT image')
   @IO.OutputDatas('lnrisks', LNRisk)
   def task(self, instance: Instance, in_data: InstanceData, lnrisks: LNRisk):

      scores = getLungNodulesRiskScores(in_data.abspath)

      for nodule_i, score in enumerate(scores):

         # create value output instance and set the value (we can also modify the description)
         lnrisk = LNRisk()
         lnrisk.description += f" (for nodule {nodule_i})"
         lnrisk.value = score

         # add to collection
         lnrisks.add(lnrisk)

@LennyN95 I have added the case level score and the dynamic scores per finding.
I also found that it might be useful for the dynamic scores to set the metadata for associated values (like position and id) per output value, like so:
https://github.com/MHubAI/models/blob/2d4365ad91a33be3bf86f666a1a4768d8e9dbc81/models/gc_grt123_lung_cancer/utils/LungCancerClassifierRunner.py#L114-L119
Setting the metadata in this way helps with debugging as well, with debug output like:
├── lncancerprob [Lung Nodule cancer probability score.]
│   The predicted cancer probability score for a single lung nodule detected by the algorithm (for nodule 8 at location (73.199997, 80.0, -215.0))
│   └── Lung Nodule cancer probability score. (0.03693072870373726)
│   ├── id: 8
│   ├── x: 73.199997
│   ├── y: 80.0
│   └── z: -215.0

Do you think this would be of added value?
Furthermore, could you give some feedback on the general implementation of the output values, i.e. is it satisfactory?

Adding the ID (and coordinates) to the metadata is excellent (because then, when exporting the report, you could technically filter by these values).
However, metadata is used to query files/data and is not available in the report exporter. So if a value is to be exportable, it needs its own value output. We could of course also implement a directive to export data metadata to the report. However, I find that this could create confusion about the ReportExporter, as it is less clear when information is stored in metadata and when as value output.

Note: test passed (09.01.2024).
DICOM case from NLST on IDC:
https://portal.imaging.datacommons.cancer.gov/explore/filters/?collection_id=nlst

Case ID: 100002
StudyInstanceUID: 1.2.840.1136<IP_ADDRESS>425808326883186792123057288612355322
SeriesInstanceUID: 1.2.840.1136<IP_ADDRESS>650531101716203536241646069123704792

s5cmd --no-sign-request --endpoint-url https://s3.amazonaws.com cp 's3://idc-open-data/b22ed5a6-ad69-4b00-ba26-ae75e96345f8/*' .

Expected output:
---------------
grt123_lung_cancer_findings.json

├── lncancerprob [Lung Nodule cancer probability score.]
│   The predicted cancer probability score for a single lung nodule detected by the algorithm (for nodule 0 at location<PHONE_NUMBER>9999999, -92.0, -260.545013))
│   └── Lung Nodule cancer probability score. (0.01483230385929346)
│   ├── id: 0
│   ├── x:<PHONE_NUMBER>9999999
│   ├── y: -92.0
│   └── z: -260.545013
├── lncancerprob [Lung Nodule cancer probability score.]
│   The predicted cancer probability score for a single lung nodule detected by the algorithm (for nodule 1 at location (-47.60000600000001, 75.0, -245.54501299999998))
│   └── Lung Nodule cancer probability score. (0.01104212086647749)
│   ├── id: 1
│   ├── x: -47.60000600000001
│   ├── y: 75.0
│   └── z: -245.54501299999998
├── clcancerprob [Case level cancer probability score.]
│   Case level probability score
│   └── Case level cancer probability score. (0.025710642337799072)
│   ├── min: 0.0
│   ├── max: 1.0
│   └── type: probability

We have updated the meta.json, please have a look if you agree. There were no changes to the code, so the test should still pass. We should be able to move on with this.

There's one last problem showing up here:
When running the model, the following print out is not captured and pollutes the console: (diag-)image_loader not found, loading dicom will not be possible, caused by this line.
The problem is, that the print statement is executed at import time, which is before we (technically can) start capturing prints.
You may be able to resolve the issue by moving the import inside the task() method of the runner Module.
For the upcoming models, please always check that the print-out is clean and when running in normal mode (no --print or --debug), the output is consistent (any uncaptured outputs like this one will break the appearance).

@LennyN95 Good catch. The print statement issue has been addressed by doing as you suggested. I missed the minor glitch when running it in normal mode the first time I checked it. I'll be more aware of the issue with the upcoming models.


set1axis fixed_tilt_angle for fixed simulations, bug makeScene1axis requests hub_height only and not clearance_height

set1axis fixed_tilt_angle for fixed simulations, bug makeScene1axis requesting hub_height only and not clearance_height
Temporarily ignored for simulations by calculating the hub_height and using it as an input
   ``` clearance_height = 0.5 + y * 0.5 * np.sin(np.radians(tilt))```

IMPORTANT:  need to check if azimuth is working properly ---- remember tracking azimuth gets conerteed from axis_azimuth = 180 to azimuth = 90.....
Example code:
demo = RadianceObj(simulationname,path = testfolder)  # Create a RadianceObj 'object'
demo.setGround(albedo)
hpc = True
metdata = demo.readEPW(epwfile, daydate=daydate, hpc=hpc)
trackerdict = demo.set1axis(metdata = metdata, backtrack = backtrack,
                        gcr = gcr, cumulativesky = cumulativesky, axis_azimuth = azimuth_ang,
                        fixed_tilt_angle = tilt)
trackerdict = demo.gendaylit1axis(hpc=hpc) # entire year.
sceneDict = {'pitch': pitch,'tilt' : tilt, 'azimuth': azimuth_ang, 'clearance_height':clearance_height, 'module_type':moduletype, 'nMods': nMods, 'nRows': nRows}
trackerdict = demo.makeScene1axis(moduletype=moduletype, sceneDict=sceneDict, hpc=hpc)
print("3. Finished makeScen1axis")
trackerdict = demo.makeOct1axis(hpc=hpc)
print("4. Finished Oct1axs")
trackerdict = demo.analysis1axis(sensorsy=sensorsy)
print('\n\r*****RESULTS*****')
print('simulation run in %0.2f seconds' % (time.time() - t))
print('Front %0.2f' % (np.sum(demo.Wm2Front)))
print('Rear %0.2f' % (np.sum(demo.Wm2Back)))
print('BGG: %0.2f ' %(np.sum(demo.Wm2Back) / np.sum(demo.Wm2Front)) )

fixed in development branch


renaming step functions to iteration or update functions?

Minor issue, but I think us using step functions as terminology for the iteration/update steps in accumulate_scan is possibly a bit confusing? Because "step function" is a pretty well known maths concept (https://en.wikipedia.org/wiki/Step_function), and we are using it in a different context e.g. "step-by-step".
This might just be reflecting my personal biases!

e.g. AbstractAccumulationStep -> AbstractAccumulationUpdate

good point. Would your suggestion be iterator or updator or other?

My suggestion would be update or updater because iterator is a bit overloaded in Julia.

so I don't think we should use update as we are likely to want a function called update to i.e update turing models (and in my head update means update an existing object). I don't love that it doesn't clearly identify the iterative process. Hmm

Some thing with recurrence or recurrent or an abbreviation?

hmm maybe

Its the same operation as a recurrent NN (just with a dot product rather than a NN call)... so at least some people would find it intuitive description.

good point

Have we in fact just settled on Steps?

I think we have more paused thinking about it until we have a better vision for a more generic composable iterator vision


There should be a way to get the resulting image size / dimensions.

When using max() or withoutEnlargement() or giving only a width or a height, the user of sharp will not know the resulting image size.
For my project for example, I need to at least know the aspect ratio of the image which the resulting width and height can tell.
#32 covers getting the size before resize operation. This issue is about getting the size after the resize operation.

Is there a way to get the width and height at any interim step? For example, after a resize operation, before a file is written.

Is there a way to get the width and height at any interim step? For example, after a resize operation, before a file is written.


ISPN-9186 Upgrade to protostream 4.3.0.Alpha8

make some of the entities immutable

https://issues.jboss.org/browse/ISPN-9186
https://issues.jboss.org/browse/ISPN-10418

@anistor I like the addition of @ProtoFactory :+1:
There are related failures in the testsuite though:
Exception while calling bridge#set entity class: org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper field bridge: org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge@1caf64f7 java.lang.UnsupportedOperationException: FieldDescriptor.getDefaultValue() called on an embedded message field (only scalars can have a default value)

There are related failures in the testsuite though:

Indeed. This is hopefully a previously hidden bug in infinispan's IndexingTagHandler. investigating now

Just fixed some tests and pushed; let's wait for CI.

run custom performance tests please
https://gist.githubusercontent.com/ryanemerson/40e40efb6e73c8c19c8a5d42c3dfe32f/raw/e57b17d984c9816580519c6b2f8a5aede60e601d/PR-7153.xml
custom_library


Rework the whole datetime library documentation

Hi, I see that this PR is relatively recent, and my issue may be very well related to documentation, so I'll ask my question here (happy to open an issue if the issue is not doc related):
I'm trying to run a py2neo project that I left untouched for about 5 months. The project included dates via the neotime.Date datatype and didn't have any issues before. I believe this was py2neo==4.2.0.
Now, with version py2neo==2021.2.0, I am unable to commit any nodes with date to the DB. When I use neotime.Date, I get this error:
TypeError: Values of type <class 'neotime.Date'> are not supported

And when I replace it with either datetime.date or neo4j.time.Date I get an error like:
TypeError: Cypher literal values of type neo4j.time.Date are not supported

So basically, I don't know how to commit dates, and I didn't find any information from the last few months that could help me towards that end. Particularly, no single mention to dates in the docs. Then I found a few commits in this PR that remove quite a bit of documentation on Dates, so I guess my question is:
Does anyone know a stable way to use dates in py2neo's current version, or where could I find the docs to do so?
Thanks in advance!