[{'code': 400, 'message': 'request must contain between 0 and 100 ids'}]

crowd@crowd-VirtualBox:~$ falcon host_search -f OS=Windows
Falcon Toolkit
Configuration Directory: /home/crowd/.local/share/FalconToolkit
Searching for hosts...
Traceback (most recent call last):
File "/home/crowd/.local/bin/falcon", line 8, in
sys.exit(cli())
File "/home/crowd/.local/pipx/venvs/falcon-toolkit/lib/python3.10/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/home/crowd/.local/pipx/venvs/falcon-toolkit/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/home/crowd/.local/pipx/venvs/falcon-toolkit/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/crowd/.local/pipx/venvs/falcon-toolkit/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/crowd/.local/pipx/venvs/falcon-toolkit/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/home/crowd/.local/pipx/venvs/falcon-toolkit/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), args, **kwargs)
File "/home/crowd/.local/pipx/venvs/falcon-toolkit/lib/python3.10/site-packages/falcon_toolkit/hosts/cli.py", line 53, in cli_host_search
host_search_cmd(client, filters, online_state)
File "/home/crowd/.local/pipx/venvs/falcon-toolkit/lib/python3.10/site-packages/falcon_toolkit/hosts/host_search.py", line 29, in host_search_cmd
host_data = client.hosts.describe_devices(filters=fql, online_state=online_state)
File "/home/crowd/.local/pipx/venvs/falcon-toolkit/lib/python3.10/site-packages/caracara/filters/decorators.py", line 56, in wrapper
return func(_args.args, **_args.kwargs)
File "/home/crowd/.local/pipx/venvs/falcon-toolkit/lib/python3.10/site-packages/caracara/modules/hosts/hosts.py", line 153, in describe_devices
device_state_data = self.get_online_state(device_ids)
File "/home/crowd/.local/pipx/venvs/falcon-toolkit/lib/python3.10/site-packages/caracara/modules/hosts/_online_state.py", line 59, in get_online_state
device_online_state_data = batch_get_data(device_ids, self.hosts_api.get_online_state)
File "/home/crowd/.local/pipx/venvs/falcon-toolkit/lib/python3.10/site-packages/caracara/common/batching.py", line 117, in batch_get_data
raise Exception("At least one thread returned an error: " + str(errors))
Exception: At least one thread returned an error: [{'code': 400, 'message': 'request must contain between 0 and 100 ids'}]

Hi there, thank you for raising this! This bug should be fixed by upgrading Falcon Toolkit to the latest version (pipx upgrade Falcon-Toolkit should do this for you).
If you still get it on the latest release let us know and we'll figure out what's going on here.
CC: @kenoel for awareness

fixed, roll back to an older version

I'm glad you got this to work. If you experience the issue on the latest version of the Toolkit (v3.2.1, or higher), please reopen the ticket and we can look into this again.


DOP-4690: prepare parser for new collapsible directive

Ticket
DOP-4690
Notes

Adds .. collapsible::  directive into our rstspec, and handles new ReSTructured text into expected AST
Adds Error level Diagnostics for missing content block within collapsible directive

README updates

[ ] This PR introduces changes that should be reflected in the README.md and/or HACKING.md, and I have made those updates.

[X] This PR does not introduce changes that should be reflected in the README.md and/or HACKING.md

Just two trivial nits!

sorry for the oversight but from this thread it looks like we want to catch nested collapsibles as well! added a postprocess and a test case for this PTAL


Drop now-illegal "is rw" from array parameter

Fixes template6 for newest rakudo.

Wait, I just realized it was an array with "is rw" on it, which hasn't been used in ages, as both arrays and hashes are mutable references by default.


[WIP] Simpler workers

Previous workers implementation is overly complex and does not work on Brunch 2.x.
They worked by running full-featured brunch instance in every worker. Besides the added complexity, this approach also won't work with deppack, since it needs to store the dependency graphs of all files in one place.
This PR will introduce a simpler implementation of workers. All deppack caches and most of brunch logic will be run inside the main process. The workers are going to handle compilation/linting and optimization. The master will send them the data and they will just call the needed plugins on that, and send the result back.
Workers functionality is opt-in and requires you to pass a CLI option -J <num>. It is better than storing that in the config, because number of workers that yield the speed increase depends on each machine, it's number of cores, so it doesn't belong to the config.
On one of our apps that takes about 15s to compile on 2.4.2, and 25s to compile production (uglified) build on 2.4.2.
With 2 workers, on a dual-core macbook, the dev build is 12s and production one is 20s (so, ~20% increase).
To reference #880:

pipes would not provide much benefit compared to [cp/process].send (which is also async in node 5)
it's not like any tangible amount of time is spent in IPC, maybe 200-400ms tops in the before-mentioned app (2-4%), so probably shouldn't be our priority when optimizing
cluster is not used anymore

Reducing duplicate config load work (which takes maybe a second on that app) is not possible because:

node doesn't have threads and shared memory, so the children have to themselves process the config... or
the master process could send the loaded config to all of its children, after it's been loaded, but it can't, because the config is more than just JSON, it contains closed functions and such in the exports, so we can't just serialize it

The work to be done is to further look how we can achieve a close to 50% increase with 2 cores.
The available options would be to either:

merge this, update the docs about this new functionality, mentioning it's experimental, and incrementally introduce performance optimizations (aka what we did with deppack); or
wait until it's perfect

[x] squash commits
[x] properly terminate the process with children on brunch build
[x] handle brunch restarts so that workers could be used for watch
[x] remove remains of 1.8.x workers
[x] handle errors in workers
[x] update docs
[x] make sure onCompile still works
[x] get rid of a global (use a mediator?)
[x] pass source map to optimizers
[x] ensure production builds work as they should

@paulmillr take a look

Have you perf tested this? I eventually realized after my initial experiment with this was that the primary fatal flaw in what I had put together was using process.send() on the compiled results which you appear to still be doing. process.send() is a blocking sync operation (at least it was at the time) and passing around objects of that size made the performance worse the larger the project (where you'd expect to see the most perf gain from parallelism). I originally thought it was just the initial penalty from having to re-process the config and stuff like that, but that's not the most substantial problem.
Play with it and see if you can find circumstances where using this boosts speed. Better with larger projects than just plain skeletons.
The solution to the problem I've just described that I think merits looking into is to use streaming stdio interfaces to communicate between the processes and avoid process.send() for anything more than trivial signals (or perhaps avoid it altogether).

@es128 as I've stated in the PR description,

pipes would not provide much benefit compared to [cp/process].send (which is also async in node 5)
it's not like any tangible amount of time is spent in IPC, maybe 200-400ms tops in the before-mentioned app (2-4%), so probably shouldn't be our priority when optimizing

Pipes with streamed IO however will introduce much more complexity for no feasible benefit, even if it made IPC take zero time (which it, admittedly, will not)

Have you perf tested this?

If you read into the PR description, the answer to that would be an apparent yes.

On one of our apps that takes about 15s to compile on 2.4.2, and 25s to compile production (uglified) build on 2.4.2.
With 2 workers, on a dual-core macbook, the dev build is 12s and production one is 20s (so, ~20% increase).

I originally thought it was just the initial penalty from having to re-process the config and stuff like that, but that's not the most substantial problem.

For the reference, on the said app, the config takes about a second to parse and such, the optimized config load function for workers made it 200-300ms less. As explained in the PR, there doesn't seem to be a way to avoid that completely, unfortunately.

I apologize. You're right, I didn't read through everything you had written. Just skimmed very quickly and jumped straight to the code to look for process.send. Didn't remember I had written #880.
Good work, looks like you've done a thorough job here.

With 2 workers, on a dual-core macbook, the dev build is 12s and production one is 20s (so, ~20% increase).

Any experiments with greater numbers of workers?

@es128 on a dual-core machine, more workers lead to a decline in speed boost. 3 is slower than 2 (13-13.5ms instead of 12s); 4 were on par with no workers at all (15s). In general, # of workers should equal to the number of cores, tops.

I don't have a machine with more cores to see how that'd improve speed

Interesting, thanks. I've heard anecdotal references that for server applications using cluster some people have found via load testing that the ideal number of workers was slightly less than 2x cores. Probably highly dependent on various factors in system architecture and the application itself.


Windows Terminal seems 30x times slower than mintty/wsltty

Environment
Windows:
Platform ServicePack Version      VersionString
-------- ----------- -------      -------------
 Win32NT             10.0.18363.0 Microsoft Windows NT 10.0.18363.0

WSL: Debian10

Steps to reproduce

start wsl in Windows Terminal
run: time for i in {1..5000}; do echo "line $i"; done
start wsl in wsltty.
run: time for i in {1..5000}; do echo "line $i"; done
compare the real time result.

Expected behavior
According to the roadmap:

Terminal shall be fast and efficient. Input latency should be eliminated wherever possible. Terminal will be very memory-efficient, and will avoid utilizing unnecessary dependencies to minimize memory consumption and disk footprint

Performance is supposed to be in the first place in the roadmap. And Windows Terminal should, at least, not be slower many times than mintty/wsltty before 1.0 release.

Actual behavior

Windows Terminal:

$ time for i in {1..5000}; do echo "line $i"; done
line 1
line 2
line 3
...
line 5000

real    0m0.935s
user    0m0.047s
sys     0m0.125s

screenshot:

It requires almost one second to output 5000 lines.

wsltty:

$ time for i in {1..5000}; do echo "line $i"; done
line 1
line 2
line 3
...
line 5000

real    0m0.034s
user    0m0.016s
sys     0m0.016s

screenshot:

wsltty uses mintty as the backend, which requires only 0.034s to output 5000 lines.
Conclusion
Windows Terminal may have the lowest responsive time, but the throughput is really poor, almost 30x times slower than mintty/wsltty.

So, the more useful comparison might be Windows Terminal vs the vintage console host, where the Terminal is nearly 7x faster:

This might be part of a discussion @miniksa and I were having....

This might be part of a discussion @miniksa and I were having....

Yes, it's probably the overhead of the console driver and/or we need to do wait chain analysis on it.
Our goal for 1.0 was to be faster than the original conhost while using the Windows Console driver. I'm not surprised that using a customized PuTTY talking directly to the TTY streams from the *nix environment is faster given it doesn't use any of the console system.
Long term, we want to get parity with this.

For me WT is actually slower, 9s vs 5s on con. Anyway I think this is dup of https://github.com/microsoft/terminal/issues/4129

/dup #3075


feat(keychain)!: expose clear method instead of clearing in plugin

Let's make keychain application lifecycle agnostic. Hooks are just used for clearing keychain, so let's just expose a clear method wallet module can call

@diegomura some lint errors

@diegomura can u rebase? lint errors were on master, just fixed


Release 0.2.0

v0.2.0
Breaking change!

EMMGMMHyperParams is renamed to EMGMMHyperParams (#55)

New features

Return local metadata from model training to algorithm (#71).

Tasks completed

Update FLAIR preprocessing script to download dataset from HuggingFace, available at https://huggingface.co/datasets/apple/flair (#72).
Update LLM Benchmark Configs (#63).
New improved worker scheduling in distributed simulations. Speeds up FLAIR benchmark by 19% (#73).
Don't pin PyTorch version to 2.0.1 (#69).
Move --noise_cohort_size to add_mechanism_arguments (#70).

@ac554 I see yapf, ruff and mypy is finding mdm and failing on that https://app.circleci.com/pipelines/github/apple/pfl-research/195/workflows/ac6ff5af-e3b3-4711-810e-349e395efd06/jobs/936 . Can you please fix the issues ?

I added same ruff ignores in publications/ as in benchmarks/ in https://github.com/apple/pfl-research/pull/77/commits/19ef43765b07fe514d83c7891fac2bf8181d2617 . After this the remaining issues are very minimal.


Update the run_forseti script to not delete the data model

User should be able to re use the data model created by the cron job for explain purpose. We need to clean up the data model once in a while, maybe we can follow what's implemented in Inventory (forseti inventory purge).

The new indexes has reduced the total model creation time 50X in issue #1509 .  So, that would mitigate the need to do this for 2.0

@blueandgold is this issue still an open?


Odysee Truncated Video Download

Checklist

[X] I'm reporting a broken site
[X] I've verified that I'm running yt-dlp version 2022.05.18 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
USA
Description
Tried to download video from Odysee and video is truncated only 1:10 minutes downloaded 1:20 minutes in web player and it is truncated at the end of the yt-dlp downloaded one.  Feels like there is another part of the blockchain stream its not catching....
:
Verbose log
The log is just going to show that it operated correctly. Seems like its missing some transfer in the way the blockchain streaming works but if it shows up correctly in the webplayer the truncated portion must somehow be available. The log is normal nothing to see here.......  Rather not divulge what I was watching........

Irrespective of the excuses, we need to see log


Bundler.with_clean_env crashes by accessing private method clone on hash

NoMethodError: private method `clone' called for #<Hash:0x007f9c978763e8>
/Library/Ruby/Gems/2.3.0/gems/bundler-1.16.2/lib/bundler/environment_preserver.rb:45:in `restore'
/Library/Ruby/Gems/2.3.0/gems/bundler-1.16.2/lib/bundler.rb:22:in `<module:Bundler>'
/Library/Ruby/Gems/2.3.0/gems/bundler-1.16.2/lib/bundler.rb:20:in `<top (required)>'

This issue happens using ruby 2.4.0 through rvm as well
To reproduce follow setup instructions in this project.
https://github.com/CocoaPods/Strata
Issue happens over here:
https://github.com/CocoaPods/Strata/blob/master/Rakefile#L373
Other things tried:

To rule out that rake-12.3.1 wasn't somehow locking the ENV hash I set up a simple Rakefile with the following:

task default: %w[test]

task :test do
    original = ENV.to_hash
    copy = original.clone
    puts copy
end

Which seems to work well.

On irb also tried the same code

original = ENV.to_hash
    copy = original.clone
    puts copy

which also seems to work well.
Full stack trace
rake aborted!
NoMethodError: private method `clone' called for #<Hash:0x007fcf00a59480>
/Library/Ruby/Gems/2.3.0/gems/bundler-1.16.2/lib/bundler/environment_preserver.rb:45:in `restore'
/Library/Ruby/Gems/2.3.0/gems/bundler-1.16.2/lib/bundler.rb:22:in `<module:Bundler>'
/Library/Ruby/Gems/2.3.0/gems/bundler-1.16.2/lib/bundler.rb:20:in `<top (required)>'
~/Strata/Rakefile:375:in `block (2 levels) in clone_repos'
~/Strata/Rakefile:373:in `chdir'
~/Strata/Rakefile:373:in `block in clone_repos'
~/Strata/Rakefile:363:in `each'

Also tried with bundler <IP_ADDRESS> with same results.
environment_preserver.rb#L45 hasn't been touched for a long time so I'm out of clues.

Looks like it's fixed with the following:
diff --git a/Rakefile b/Rakefile
index 369cf37..41113ee 100644
--- a/Rakefile
+++ b/Rakefile
@@ -29,7 +29,7 @@ DEFAULT_REPOS = WEB_REPOS
 #-----------------------------------------------------------------------------#

 def setup name
-  clone name
+  clone_repo name
   bootstrap name
 end

@@ -59,7 +59,7 @@ def bootstrap name = nil
   puts "\nDisk usage: #{disk_usage}"
 end

-def clone name
+def clone_repo name
   repos = fetch_default_repos
   if name
     repos = [repos.find { |repo| repo['name'] == name }]
@@ -110,7 +110,7 @@ begin

   desc "Clones the web repositories"
   task :clone, :name do |task, args|
-    clone(args.name)
+    clone_repo(args.name)
   end

   # Task install_system_deps
@@ -140,7 +140,7 @@ begin
       bootstrap(name)
     else
       Bundler.with_clean_env do
-        puts `bundle install`
+        sh 'bundle install'
       end
     end
   end

Just tested on Strata, now I don't get the error anymore. Good catch I didn't think the rakefile would define a clone method.
Thanks @segiddins !


Auto scroll to bottom of doc

Hey,
I noticed when I'm in the middle of a document it will auto scroll to the bottom. How do I fix this?
Thanks.

Is there a Solution?
It wasn't like this in the previous version!
Is it possible to download previous versions, cause the current version with it auto scroll down to the bottom is frustrating and slows me down.
I love this app, currently looking for another, which I don't want to do, been using this app for 6+ years.
Please Respond
Thanks
<EMAIL_ADDRESS>


Consider support / implementation of  Arrow Flight SQL

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Arrow Flight SQL has just been officially announced https://arrow.apache.org/blog/2022/02/16/introducing-arrow-flight-sql/
It would be very cool if we had a Rust implementation of this protocol to make it easier to build systems in Rust that used this efficient new protocol and would allow better ecosystem integration.
Self servingly, we would likely use it in influxdb_iox and contribute to such an implementation, but it isn't high on our priority list quite yet.
Describe the solution you'd like
I propose creating an ArrowFlightSQLService type structure that handles the protocol details, and present an idiomatic rust trait for users.
Something like:
pub trait SqlService {
  /// get  list of database schemas as an array of Strings
  fn schemas(&self, catalog_name: Option<&str>, db_schema_filter_pattern: Option<&str>) -> Result<ArrayRef>;

  /// Execute the query and return an async stream of record batches
  fn query(&self, sql: &str) -> Result<SendableRecordBatchStream>;
 ...
}

/// Implementation of Arrow Flight SQL for a `SqlService`
struct ArrowFlightSQLService<S: SqlService>  {
  inner: S
}

/// implement service from https://github.com/apache/arrow/blob/release-7.0.0/format/Flight.proto
/// That handles messages in https://github.com/apache/arrow/blob/release-7.0.0/format/FlightSql.proto
impl protobuf::generated::FlightService for ArrowFlightSQLService {

... code to translate to/from protobuf messages

}

Describe alternatives you've considered
One alternative is to make this part of the official arrow-rs release (perhaps with a sql feature in the arrow-flight crate)?
Another alternative is to make this a separate project (e.g. in https://github.com/apache/arrow-datafusion or in https://github.com/datafusion-contrib )
Additional context
DataFusion

Ideally, the Ballista scheduler should also implement a Flight SQL interface.

Ideally, the Ballista scheduler should also implement a Flight SQL interface.

Yeah, I was thinking that if we had this adapter that handled the lower level protocol details, connecting to Ballista would be fairly straightforward

I have been working on a (very clunky right now) Rust prototype that speaks the server side PostgreSQL wire protocol, parses queries and then will dispatch them to anything that speaks SQL over flight, e.g., our IOx project.
In terms of the translation between a PG query/command and a backend datasource request, I suspect that I am going to end up doing a lot of the work that an implementation of the Arrow Flight SQL protocol would do.
I would be happy to try and make some progress on this, though it's not quite clear to me yet which bits would live in Arrow or Datafusion yet.

Just so that it is clear, Arrow Flight SQL is a replacement for the pg wire protocol (both the command messages and data format).

That's super interesting @nevi-me!
I'n primarily interested in making it possible for services/projects that can talk to PG to be able to talk to IOx, where IOx really means Datafusion. My approach has been to implement the PG wire protocol and then handoff a sub-set of supported PG behaviours to a backend over Flight (e.g., queries). Then translate responses (record batches) in PG responses.
I suspect (assume) a lot of the mapping between PG commands and Flight SQL commands are the same, so making progress on a Rust implementation of the flightSQL spec will get a lot of that work done.
Whilst Flight SQL is intended to replace heterogeneous client/server DB protocols, the existing ones aren't going anywhere any time soon. I think Flight SQL will make it easier to map from specific DB protocols to anything whatever speaks Flight SQL.

I'm primarily interested in making it possible for services/projects that can talk to PG to be able to talk to IOx, where IOx really means Datafusion. My approach has been to implement the PG wire protocol and then handoff a sub-set of supported PG behaviours to a backend over Flight (e.g., queries). Then translate responses (record batches) in PG responses.

That's a great use-case, supporting the Postgres protocol makes sense as tools (esp BI) that support reading the protocol could treat IOx as a source, similar to CockroachDB and others.

I've dumped my rough drafts of the code I mentioned here: https://github.com/nevi-me/datafusion-rdbms-ext. Nothing of decent quality, but could serve as motivation for someone who has the interest and bandwidth to make DataFusion > RDMBS happen

@nevi-me That link is super interesting, thank you!

@nevi-me I checked your code, the idea is to have a to_sql() method for LogicalPlan. I think it's a good idea, actually yesterday I also try to read datafusion's codebase to find this kind of method, but got no clue.
I suggest we implement this method in datacusion, at least can covert to sql dialect datafusion itself can understand.

I have an implementation of datafusion querying directly against postgres I can share. It too uses the catalog but it allows joining a postgres table to a parquet table to now a parquet from a S3 compatible source. It would be easy enough to define a proper trait so multiple implementations could be done.
I also did the auth and part of a rust postgres protocol server (so pgcli can connect to rust) but ran out of time.

@seddonm1 does your implementation have support for Postgres' NUMERIC type? I'm struggling with figuring out how to convert that doe DataType::Decimal(p, s), and could do with some help

FWIW there is a proposed implementation in https://github.com/apache/arrow-rs/pull/1386


Localize Interpreter Errors

Pull Request Description

close #1069
Send the error locations to the IDE together with the error messages.
Changelog:

update: extract diagnostic nodes during the EnsureCompiledJob
update: execute runs only after the successful compilation
update: extract error location from the runtime exceptions
~update: extend execution API executionFailed notification with error locations~
~update: replace executionFailed notification with executionStatus containing the list of diagnostic messages.~
update: handle compilation errors in the EnsureCompiledJob
update: executionFailed notification only signals about the critical errors
add: executionStatus notification containing the list of diagnostic messages
add: extract the stack trace during the runtime errors

Important Notes
API updates:
executionContext/executionFailed
Update executionContext/executionFailed notification to only signal about the critical errors.
{
  contextId: ContextId;
-  message: String;
+
+  /**
+   * The object describing the issue.
+   */
+  failure: ExecutionFailure;
+}

executionContext/exectuionStatus
Add the executionContext/exectuionStatus notification returning a list of diagnostic messages. Diagnostic message contains information about compilation errors and warnings:
{
  /**
   * The identifier of the execution context.
   */
  contextId: ContextId;

  /**
   * The list of encountered problems.
   */
  diagnostics: Diagnostic[];
}

Checklist
Please include the following checklist in your PR:

[x] The documentation has been updated if necessary.
[x] All code conforms to the Scala, Java, and Rust style guides.
[x] All documentation and configuration conforms to the markdown and YAML style guides.
[x] All code has been tested where possible.

You're right, the text field should be called function name. The only purpose of the text field is to show something when the error happens in the builtin node when the location is missing. So, when the error happens in the builtin node, text contains the function name. In other cases, it should also be the function name but I was not able to obtain it from the node. I need your help with it.
I guess, I can revive the original executionFailed notification and use it for the fatal errors. I'll think about how to separate different kinds of errors.

Re 1: Something like stackTraceElement.getTarget().getRootNode() should give you access to a root node, so you can pretty much handle that the same as you do with entering functions. In particular there's getName, but you could also check if that's a MethodRootNode and get everything including including the method name, the qualified name, the type it was defined on and the module scope. If it's not a MethodRootNode, it was a lambda, in which case you can still get the name string from it.
Re 2: Yeah, I think it's worth it, if only for the IDE team to yell at us for having fatal errors in the compiler :P

@kustosz so, I moved critical errors back to executionFailed notification and returning the function name in the stack trace element. The only issue is that without #1097 I cannot resolve it to a function. I suppose I'm doing that issue next.


IPC debug enhancement

Small change to the Debug command on the ipc. Now returns Uuids of all view/containers. This is pre-security considerations, and the Debug command was named as so as not to be relied upon. Later, this will be replaced with more specific commands to get the state of the tree (with varying levels of information) but for now I'm opening up the data to be displayed to make making client applications easier.

Coverage remained the same at 59.225% when pulling 2bc8dd4c2d4ffc3dffc3bcaae2c655e60d0d9586 on ipc-debug-enhancement into 9f76862805246ea005eaa7dbe7abf89d0c85062b on master.

There are an alarming number of changes in this PR throughout the history, and the leaf nodes of the JSON are still just formatted strings, right? It's still got things like
format!("Container w/ layout {:?} and id {:?}", layout, id)

I might be missing something, but shouldn't that actually be a JSON object?
{
    "type": "container",
    "layout": "horizontal",
    "id": "<id>",
    "children": [ ... ]
}

Obviously this whole API still isn't done but if you wanted to enhance it that might be a better way to go. I can open a new PR with those proposed changes.

There we go, fixed the history.
This is just for my benifit, I realize the JSON is awful and totally unparsable. Debug was always meant to just be that -- debugging. There's no other way to get UUIDs of containers (which basically all other commands need) so i just wrote this so I can get the UUIDs of containers easily (again, this command will go away later and be replaced by others)

Coverage remained the same at 59.225% when pulling 76b4369b8618f4dcf26017345aa79a3a31297056 on ipc-debug-enhancement into 92f16568e82b7b7b97e3af60f4272d80ebc41b88 on master.


Wrong Results

EFI firmware version check:
	[-] ATTENTION - Your Mac model (iMac7,1) does not seem to have had any EFI updates released for it :'(

Highest build number check:
	[-] ATTENTION - You are NOT running the latest release build number of your OS version (10.11.6). Your build number is 15G18013, the latest release build number is 15G17023

Up-to-date OS check:
	[+] SUCCESS - You are running the latest major/minor/micro version of the OS you have installed (10.11.6)

First of all, the program doesn't identify my Boot ROM. My version is IM71.007A.B03, which is the latest for the iMac 7,1 (based on this https://support.apple.com/en-us/HT201518 ). It definitely has received Boot ROM updates before. Also, 15G18013 is the latest build for 10.11.6, not 15G17023.

Hi there! Regarding the lack of available EFI updates for your Mac model (iMac7,1) this is because Apple never shipped a newer version after the one from 2008 listed on the page you referenced. Our data starts with updates released in 2015, after Apple changed the deployment method in response to the Thunderstrike vulnerability and beyond. Because of this we had to consider yours and other models legacy as there is no data to track for them.
To add some more context, any response you see for this model will contain a warning which is an expected behavior: to alert on old unsupported Mac models that can not be trusted if your organization's threat model includes nation-state firmware attacks. It is up to the reader to determine whether this is the case for them or not.
As for the incorrect build being reported, we have made some tweaks to resolve this. The OS build data is up to date with 15G19009 as of January 23, 2018. If you could, please test against a supported Mac model running the latest OS X 10.11 security update (2018-001 El Capitan) and let us know the results. I will go ahead and close the issue once we hear back about this part.

[+] SUCCESS - You are running the latest build number (15G19009) of the OS version you have installed (10.11.6)
I get it now. I did test the build number, and it is fixed.

Thanks for confirming, I'll close the issue.


Edits to do

Clean static folder
Clean navbar from duplicates
Fix images bug in Cache/Overview

@noamsabb Is this done?

@noamsabb Is this done?

Yes


[#4367] feat(iceberg): add OSS support for Gravitino Iceberg REST server

What changes were proposed in this pull request?
add OSS support for Gravitino Iceberg REST server
Why are the changes needed?
Fix: #4367
Does this PR introduce any user-facing change?
yes, will add a separate PR to add document
How was this patch tested?

setup a Iceberg REST server will following configurations

gravitino.iceberg-rest.catalog-backend = jdbc
gravitino.iceberg-rest.jdbc-driver = org.postgresql.Driver
gravitino.iceberg-rest.uri = jdbc:postgresql://<IP_ADDRESS>:5432/postgres
gravitino.iceberg-rest.jdbc-user = postgres
gravitino.iceberg-rest.jdbc-password = xxx
gravitino.iceberg-rest.jdbc-initialize = true
# change to s3a://test/my/key/prefix for Hive catalog backend
gravitino.iceberg-rest.warehouse = oss://xxx/key/prefix
gravitino.iceberg-rest.io-impl= org.apache.iceberg.aliyun.oss.OSSFileIO
gravitino.iceberg-rest.oss-access-key-id = xx
gravitino.iceberg-rest.oss-access-key-secret = xx
gravitino.iceberg-rest.oss-endpoint = https://oss-cn-beijing.aliyuncs.com

test with Spark SQL

Iceberg doesn't provide a aliyun-bundle jar like S3 or GCS, some are some options:

add aliyun oss dependencies for Gravitino Iceberg REST server
let the user copy aliyun oss-related jars (3 jars for this version) to Gravitino Iceberg REST server classpath when using oss.
provide a Aliyun bundle jar for Iceberg, tracked int https://github.com/apache/iceberg/issues/10970

I prefer using Option2 for now and Option3 for long term.

@FANNG1 is it ready for review?

@FANNG1 is it ready for review?

yes


Add upsert option to upload/update

What kind of change does this PR introduce?
This is to address issue https://github.com/supabase-community/supabase-py/issues/693.  Documentation said we could use the upsert option, but user would have to specify x-upsert.
I also updated pre-commit hooks as the version of autoflake is old enough to still require distutils which no longer exists in python 3.12.
What is the current behavior?
Old behavior code:
with open(filepath, 'rb') as f:
  supabase.storage.from_("bucket_name").update(file=f, path=path_on_supastorage, file_options={"cache-control": "3600", "upsert": "true"})

would not pass along the upsert option to server side.
New behavior:
with open(filepath, 'rb') as f:
  supabase.storage.from_("bucket_name").update(file=f, path=path_on_supastorage, file_options={"cache-control": "3600", "upsert": "true"})

Is passed along as x-upsert header option.
Please link any relevant issues here.
https://github.com/supabase-community/supabase-py/issues/693
What is the new behavior?
See above
Feel free to include screenshots if it includes visual changes.
Additional context
This addresses the client side operation.
Add any other context or screenshots.

Codecov Report
Attention: Patch coverage is 33.33333% with 8 lines in your changes are missing coverage. Please review.

Project coverage is 83.08%. Comparing base (d3f7e8c) to head (84d6dce).
Report is 13 commits behind head on main.

Files
Patch %
Lines

storage3/_async/file_api.py
33.33%
4 Missing :warning:

storage3/_sync/file_api.py
33.33%
4 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #199      +/-   ##
==========================================
- Coverage   84.36%   83.08%   -1.28%
==========================================
  Files          12       12
  Lines         467      479      +12
==========================================
+ Hits          394      398       +4
- Misses         73       81       +8

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

The .pre-commit-config.yaml file has been modified but the formatting of this shouldn't have changed. Also did you use the make build_sync command to create the changes in the _sync/ directory? if not this is how you should do it and not make manual edits to the _sync/ directory.

Okay, I updated my editor to not autosave on yaml (that was the problem there) and ran make build_sync to copy over the changes.   I uploaded, but didn't squash.  Let me know if we need to squash as well for this type of work.
Let me know any additional feedback and if there is another issue to go after.

@clefelhocz2 thanks for this, I'm going to ask for one more update, can you change the upsert to match that of the JS library so it only does upsert on POST requests. I've opened an issue on the storage-js repo to ask about this behaviour but would prefer if we followed it for now so we can merge this in and then change if I get a reply that the behaviour in the JavaScript library is incorrect. https://github.com/supabase/storage-js/issues/194

Thanks.  Will start on updating the PR today and hopefully have it over soon.

@silentworks I was able to update per your guidance.  It looks different than the javascript code, but that's because the javascript code actually works differently.  The options are copied into headers based on different filetypes, duplex, etc.   It isn't a great implementation as adding a new default to DEFAULT_FILE_OPTIONS isn't going to add it to the headers based on my reading.
Let me know any feedback and/or new issue I could tackle.  I was looking at https://github.com/supabase-community/auth-py/issues/320, but it looks involved.

@clefelhocz2 thanks for this PR. The change will be out in the next release of the library. In regards to the other issue, lets move the conversation to the issue itself. It is involved but I can guide you through it a bit.


service: add state attr to Service class

Issue defined at https://github.com/3scale/apisonator/issues/34
Disable service:
require 'pisoni'

ENV['THREESCALE_CORE_INTERNAL_API'] = nil
ThreeScale::Core.url = "http://myhost:myport"
ThreeScale::Core.username = "username_key"
ThreeScale::Core.password = "password_key"

service_id = "SVC_A"
service = ThreeScale::Core::Service.load_by_id(service_id)

service.deactivate
service.save!

Activate service:
require 'pisoni'

ENV['THREESCALE_CORE_INTERNAL_API'] = nil
ThreeScale::Core.url = "http://myhost:myport"
ThreeScale::Core.username = "username_key"
ThreeScale::Core.password = "password_key"

service_id = "SVC_A"
service = ThreeScale::Core::Service.load_by_id(service_id)

service.activate
service.save!

Activation status can be queried
require 'pisoni'

ENV['THREESCALE_CORE_INTERNAL_API'] = nil
ThreeScale::Core.url = "http://myhost:myport"
ThreeScale::Core.username = "username_key"
ThreeScale::Core.password = "password_key"

service_id = "SVC_A"
service = ThreeScale::Core::Service.load_by_id(service_id)

puts service.active?

Requires apisonator >= 2.87.1

ready for new review

bors r+


Service starts and then stops

I setup my app (npm module) it runs fine from command line, but when run as service it gives this
2017-09-12 14:03:51,843 INFO  - Starting ServiceWrapper in the service mode
2017-09-12 14:03:51,890 INFO  - Starting json-server --watch C:\Development\DC\DC_Tasks\db.json --routes C:\Development\DC\DC_Tasks\routes.json
2017-09-12 14:03:51,890 INFO  - Starting json-server --watch C:\Development\DC\DC_Tasks\db.json --routes C:\Development\DC\DC_Tasks\routes.json
2017-09-12 14:03:51,906 DEBUG - Completed. Exit code is 0
any ideas?
thanks
Mark

Sorry, missed the notification. Is it still impacting you? If yes, anything else in wrapper/error logs?

Hint: cannot start service from command line or debugger. You must first install the windows service (using installutil.ext), and then start it with ServerExplorer, Windows service management tools, or the NET START command.
any ideas?
thanks

Probably there should be better executable existence diagnostics

Also beware that (in order to omit the full executable path) the executable has to be on the service's user's PATH, which isn't necessarily the same as your logged in user's. I got into trouble because I had an executable on the User PATH but not on the System PATH.

Probably there should be better executable existence diagnostics

Indeed 😉


Stalled: add explanation

Coverage remained the same at 25.796% when pulling 5b8cfb4fe1d9e173b5f9dfe183b05a1ada9ca259 on calixteman:explain_stalled into 1d69a7748a9dfa0744e93730aad89128f7398eb9 on mozilla:master.


Instanced geometry not importing correctly

When importing some H2 bsps (haven't tested all of them but district and sacred icon are affected), some of the instanced geo gets offset randomly.

I'm having this issue as well. Some of the geometry is offset at random or missing entirely. For isntance, the big spinning wheel on Zanzibar is placed on the beach instead of the middle of the map, and Zanzibar's building is now inside the big spinning wheel and rotated 90 degrees on its side.


Each puppet run will restart the xl release service

From @ltutar on May 7, 2015 19:43
Each puppet run will restart the xl release service.
otice: /Stage[main]/Xlrelease::Config/Ini_setting[xlrelease.admin.password]/value: value changed '{b64}CAIxCbqlmdK93f2p1dtBQw==' to 'xebia1'
Info: Class[Xlrelease::Config]: Scheduling refresh of Class[Xlrelease::Service]
Info: Class[Xlrelease::Service]: Scheduling refresh of Service[xl-release]
Notice: /Stage[main]/Xlrelease::Service/Service[xl-release]: Triggered 'refresh' from 1 events
Notice: unable to reach localhost:5516
Notice: unable to reach localhost:5516
Notice: unable to reach localhost:5516
The result of ps -ef | grep - jetty
501       5716  5707 16 21:36 ?        00:00:44 /usr/lib/jv
501       6165  6156 95 21:41 ?        00:00:11 /usr/li
Copied from original issue: WianVos/puppet-xlrelease#5

fixed it ..


Unit test for issue #4913

Added unit test that demonstrates issue:
http://www.prjhub.com/#/issues/4913

Merged manually against 2.1.x and majors, thanks.


One Pace Catalog

Addon Manifest URL
https://onepaceaddon-zoropogger.koyeb.app/manifest.json
Addon Description
One Pace!
Language of Content
None
Choose Labels

[X] anime
[ ] asian drama
[ ] bollywood
[ ] cartoons
[ ] debrid support
[ ] http streams
[ ] live tv
[ ] metadata
[ ] misc
[ ] movies
[ ] music
[ ] nsfw
[ ] podcasts
[ ] radios
[ ] subtitles
[ ] torrents
[ ] tv shows
[ ] video games

Thank you !!


Bottom Bar background colour doesn't change

Firstly, I just wanted to say thanks for this great project.
It has definitely helped me a lot but I am now trying to customize the bar to fit with my colour scheme.
I am using this code:
        mBottomBar.setItems(R.menu.bottombar_menu);         //Colours         //mBottomBar.mapColorForTab(0, ContextCompat.getColor(this, R.color.colorAccent));         mBottomBar.mapColorForTab(0, "#000000");         mBottomBar.mapColorForTab(1, "#000000");         mBottomBar.mapColorForTab(2, "#7B1FA2");         mBottomBar.mapColorForTab(3, "#FF5252");
for my four tab bar items. However nothing changes and the bar remains white. I have tried uncommenting the line for tab 0 but it causes an error. Anyone got any ideas?
Thanks

I'm having the same issue, though only in fixed mode it seems.

Yes i think it is just fixed mode.
I resorted to doing this, as I didn't need the tab bar colour to change depending on the tab:
mBottomBar.getBar().setBackgroundColor(getResources().getColor(R.color.colorPrimary));
and then used this to set the active tab colour:
mBottomBar.setActiveTabColor("#ffffff");
Thanks

Not sure, but you might be affected by the night mode setting. Try this out..
mBottomBar.ignoreNightMode();

I tuned on dark mode also which made inactive tabs a readable white colour


[FEATURE] TagPathParser

Is your feature request related to a problem? Please describe.
For one of the projects, TagPathParser is needed since the path of a tag is important for subsequent modifications of other references.
Describe the solution you'd like
Adding the TagPathParser function from https://files.inductiveautomation.com/sdk/javadoc/ignition81/8.1.21/com/inductiveautomation/ignition/common/tags/paths/parser/TagPathParser.html#<init>()

Hello, @cosminroger.
Thanks for submitting a feature request. I have added the TagPathParser class as well as supporting classes.
Expect a release by Friday.


Constant error 39508

I keep getting messages saying "Un problème est survenu", with error code 39508, when updating my risk status.
The message redirects to the FAQ but I see no mention of this issue there.
Exposure logging seems to work fine.
Android version: 8.1.0 (any other infos needed?)

This error message still refers to the german FAQ.
Here is the revelant info:
https://www.coronawarn.app/en/faq/#API39508

Thanks for the clarification!
I've waited 24h before restarting the app and didn't get the message this time.


Fix/dateline

Committed fixes to the LDT latlon domain code to allow for grid domains to cross the International Date Line (IDL).  Main code implemented in LDT but also tested LDT domain that crosses IDL with LIS test as well.
Test case files have been added to the LIS source code testcase directory.  Input and output target files are located here:
/discover/nobackup/projects/lis/Projects/LDT/LISF_Testcases/Cross_Dateline/Testcase/
LDT files: idlcrossdomain_ldt_v72.tar.gz
LIS input files:  idlcrossdomaintest_input.v72.tar.gz
LIS output files:  idlcrossdomaintest_output_v72.tar.gz
This addresses the ticket issue (bug) number #117 .

Thank you for testing the committed code and for the feedback on the code, with regards to areas that need to be cleaned up.  I will make sure to work on that and commit the updates by tomorrow.

All updated code should now be committed and pushed over.
Working on updating the testcase next, adding in 'Openwater' surface type.

Still TODO is adding this test case to the test case suite.


Search screen re-design

Small re-design of how the first screen should look like IMO.
Please take a look at the PR thoroughly and ask any questions regarding changes.

This PR contains all commits from the latest closed PR by @r-ss. Closed as duplication.


Can we extend Sparse4D to a new dataset?

https://github.com/linxuewu/Sparse4D/blob/c48ecd2b020d7770faaa865f577eed28406b75e9/projects/mmdet3d_plugin/models/blocks.py#L514
Assuming that we are using a new trainset composed by different cars and they have different projection_mats(lidar2img) to train a model, now we are trying to use this model in a new car without lidar. The question is whitch projection_mat should we choose?Thanks a lot!

Any coordinate system that is fixed relative to the ego vehicle is fine, and the commonly used one is the "vehicle coordinate system".

As far as I know the camera coordinate system and lidar coordinate system are not fixed relative to the ego vehicle for all nuscenes data. So will we futher improve the mAP by changing lidar coordinate system into CAM_FRONT coordinate system?Thanks!

The camera coordinate system and lidar coordinate system are fixed relative to the ego vehicle. The reason why ego2cam and ego2lidar are changing is to synchronize the time of multiple sensors.mAP does not vary with coordinate system changes.


starboard-octant-plugin: 0.10.0 -> 0.10.3

Motivation for this change
Bump starboard-octant-plugin to 0.10.3
0.10.1 & 0.10.2 don't exist
Things done

[X] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[X] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[X] Ensured that relevant documentation is up to date
[X] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 123539 run on x86_64-linux 1

1 package built:

starboard-octant-plugin


Fix add smime, isDefault can not be set on insert

From: https://developers.google.com/gmail/api/guides/smime_certs
Upload an S/MIME key
The S/MIME key should be present in the pkcs12 field in that format; no other fields should be set in the request.


[BUG] - WSDL containing multiple endpoints (wsdl:port) fails to deploy

Release version
v4.10.3
Describe the bug
If an APIM instance has multiple custom domain gateways defined the Specification.WSDL file that is extracted as part of the APIOps will contain multiple endpoints.  When the APIOps attempts to publish the API it fails with an error "Multiple service endpoints available, only one can be imported at a time".
Note: If I manually import that WSDL in the APIM console it will prompt me to select one of the service endpoints and then once selected it will successfully import.
If I remove the second endpoint (the wsdl:port) from the Specification.wsdl the publish will be successful.
This is a sample from the wsdl:
<wsdl:service name="Sample-Web-Service">
<wsdl:port name="Sample-Web-Service-1" binding="tns:Sample-Web-Service">

</wsdl:port>
<wsdl:port name="Sample-Web-Service-2" binding="tns:Sample-Web-Service">

</wsdl:port>
</wsdl:service>
Please see the Actual behavior section for the error that is generated.
Thanks very much for any assistance.
Expected behavior
The same specification.wsdl that is exported should successfully import.
Actual behavior
The publish using a specification.wsdl that contained multiple endpoints fails:
2023-11-27T16:43:33.3150772Z       System.Net.Http.HttpRequestException: HTTP request to URI
https://management.azure.com/subscriptions/xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/xx-xx-xxxxx-xx-xxxxxxxxx/providers/Microsoft.ApiManagement/service/xxxxxxxxxxx/apis/xxxxxxxxxxxx?api-version=2022-04-01-preview&import=true
failed with status code 400. Content is
'{"error":
{"code":"ValidationError","message":"One or more fields contain incorrect values:","details":
[
{"code":"ValidationError","target":"representation","message":"Parsing error(s): Multiple service endpoints available, only one can be imported at a time"},
{"code":"ValidationError","target":"representation","message":"Parsing error(s):
{"events":[{"level":"Informational","event":"ParsingXMLStarted","message":"Started parsing XML"},
{"level":"Informational","event":"ParsingXMLComplete","message":"Completed parsing XML"},
{"level":"Verbose","event":"WsdlImportRuleVerifyWadl11Schema","message":"WSDL validated against XML Schema"},
{"level":"Informational","event":"WsdlPrecheckComplete","message":"Completed WSDL verification. WSDL is considered valid."},
{"level":"Informational","event":"WsdlParsingStarted","message":"Service :  Endpoint : "},
{"level":"Informational","event":"WsdlIdentification","message":"WsdlVersion: 'Wsdl11' TargetNamespace: 'http://ACORD.org/Standards/Life/2'."},
{"level":"Informational","event":"LoadedSchema","message":"Target Namespace: 'http://ACORD.org/Standards/Life/2'."},
{"level":"Informational","event":"LoadedSchemas","message":"Loaded '1' schemas."},
{"level":"Informational","event":"LoadedTypes","message":"Loaded '2' types."},
{"level":"Informational","event":"LoadedMessages","message":"Loaded '2' messages."},
{"level":"Informational","event":"LoadedInterfaces","message":"Loaded '1' interfaces."},
{"level":"Informational","event":"LoadedBindings","message":"Loaded '1' bindings."},
{"level":"Informational","event":"LoadedServices","message":"Loaded '1' services."},
{"level":"Informational","event":"WsdlParsingComplete","message":"Parsed 1 operations and 2 messages "}]}"}]}}'.
Reproduction Steps
From an APIM instance that has multiple custom gateway domains configured export a soap API.  The exported WSDL will contain multiple wsdl:port instances.
Try to publish that same API to another APIM instance to generate the problem.

We currently don't have bandwidth to look into this issue. Anyway you can take a look at the extractor code dealing with this use case and submit a PR?

I'll take a look.  Thanks
From: Wael Kdouh @.>
Sent: Thursday, November 30, 2023 2:58 PM
To: Azure/apiops @.>
Cc: Ian Reid @.>; Author @.>
Subject: Re: [Azure/apiops] [BUG] - WSDL containing multiple endpoints (wsdl:port) fails to deploy (Issue #428)
You don't often get email from<EMAIL_ADDRESS>Learn why this is importanthttps://aka.ms/LearnAboutSenderIdentification
CAUTION: This email originated from outside of Equitable Life. Do not click links or open attachments unless you recognize the sender and know the content is safe.
We currently don't have bandwidth to look into this issue. Anyway you can take a look at the extractor code dealing with this use case and submit a PR?

Reply to this email directly, view it on GitHubhttps://github.com/Azure/apiops/issues/428#issuecomment-1834456332, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BELVFZUIMI7RYU6LC3NCNOLYHDQLPAVCNFSM6AAAAABABXKFASVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZUGQ2TMMZTGI.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>IMPORTANT NOTICE: Internet may be vulnerable to interception. Equitable Life cannot ensure privacy or authenticity of information sent via internet, and is not responsible for any damages you incur by sending confidential or personal information to us or by sending such information to you at your request. This e-mail and any attachments are confidential, may be covered by privilege or exempt from disclosure under applicable law, and are intended for the addressee only. If you are not the intended recipient, you must not disclose, copy, distribute or retain any or part of this e-mail and any attachments without written permission of The Equitable Life Insurance Company of Canada.
AVIS IMPORTANT : L'Internet est susceptible ? l'interception. L'Assurance vie ?quitable ne peut garantir le droit ? la vie priv?e et l'authenticit? des renseignements envoy?s par l'entremise de l'Internet, et elle n'est pas responsable de tout dommage que vous pourriez subir en nous envoyant vos renseignements personnels ou confidentiels ou si nous vous envoyons de tels renseignements ? la suite de votre demande. Ce courriel et ses pi?ces jointes sont confidentiels, et peuvent ?tre prot?g?s par une clause de non-divulgation selon la loi applicable, et s'adressent ? la destinataire ou au destinataire seulement. Si vous n'?tes pas le destinataire vis? de ce courriel et de ses pi?ces jointes, vous ne devez pas les divulguer, les copier, les distribuer ou les conserver en tout ou en partie sans la permission ?crite de L'?quitable, compagnie d'assurance vie du Canada.

This is an apim limitation. We can't do anything from the apiops side.

We ended up adding a step in the extract pipeline to remove N - 1 endpoints leaving only one endpoint within the wsdl before adding it to auto generated branch.
From: Wael Kdouh @.>
Sent: Friday, July 5, 2024 11:47 AM
To: Azure/apiops @.>
Cc: Ian Reid @.>; Author @.>
Subject: Re: [Azure/apiops] [BUG] - WSDL containing multiple endpoints (wsdl:port) fails to deploy (Issue #428)
You don't often get email from<EMAIL_ADDRESS>Learn why this is importanthttps://aka.ms/LearnAboutSenderIdentification
CAUTION: This email originated from outside of Equitable Life. Do not click links or open attachments unless you recognize the sender and know the content is safe.
This is an apim limitation. We can't do anything from the apiops side.

Reply to this email directly, view it on GitHubhttps://github.com/Azure/apiops/issues/428#issuecomment-2211094002, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BELVFZR2WBQWCLJAUV7RE7LZK25VXAVCNFSM6AAAAABKNNLY2SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMJRGA4TIMBQGI.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>IMPORTANT NOTICE: Internet may be vulnerable to interception. Equitable Life cannot ensure privacy or authenticity of information sent via internet, and is not responsible for any damages you incur by sending confidential or personal information to us or by sending such information to you at your request. This e-mail and any attachments are confidential, may be covered by privilege or exempt from disclosure under applicable law, and are intended for the addressee only. If you are not the intended recipient, you must not disclose, copy, distribute or retain any or part of this e-mail and any attachments without written permission of The Equitable Life Insurance Company of Canada.
AVIS IMPORTANT : L'Internet est susceptible ? l'interception. L'Assurance vie ?quitable ne peut garantir le droit ? la vie priv?e et l'authenticit? des renseignements envoy?s par l'entremise de l'Internet, et elle n'est pas responsable de tout dommage que vous pourriez subir en nous envoyant vos renseignements personnels ou confidentiels ou si nous vous envoyons de tels renseignements ? la suite de votre demande. Ce courriel et ses pi?ces jointes sont confidentiels, et peuvent ?tre prot?g?s par une clause de non-divulgation selon la loi applicable, et s'adressent ? la destinataire ou au destinataire seulement. Si vous n'?tes pas le destinataire vis? de ce courriel et de ses pi?ces jointes, vous ne devez pas les divulguer, les copier, les distribuer ou les conserver en tout ou en partie sans la permission ?crite de L'?quitable, compagnie d'assurance vie du Canada.


Update README.md

Change Summary
Overview
Added a repo for I-ALiRT
Updated Files

README.md

I-ALiRT repo name and description

And you should be fine to merge despite the pre-commit failure -- we haven't implemented pre-commit for this.


Phoenix 4.5 released

Cloudera Labs Phoenix is now two versions behind: it is on Phoenix 4.3, but Phoenix 4.5 has now been released.
Could the repo please be updated to add support for Phoenix 4.5?

Phoenix is incredibly useful, do you know when the version will be updated from 4.3 to 4.5? The use of WAL against HBase and SQL-like interpretation make this one of the most important Parcels CDH can offer my company.

Does Cloudera give up  on  this project ?

a CLABS update for phoenix 4.5 happened back in November:
http://blog.cloudera.com/blog/2015/11/new-apache-phoenix-4-5-2-package-from-cloudera-labs/


Building with rpaths on OSX High Sierra

With OSX, dynamic libraries can be referred to as (using the example of libboostfilesystem.dylib):

libboostfilesystem.dylib,
@rpath/libboostfilesystem.dylib, or
/absolute/path/to/libboostfilesystem.dylib.

Compilation against the first of these three causes linking errors on my version of OSX. This is possibly due to changes to SIP for High Sierra (or possible a problem specific to my machine).
Possible solutions include:

using install_name_tool to change the first option to the second or third, or
linking against static libraries.

Hopefully this issue will sort itself out over time.

@rijobro, I think this can be closed?

The option, SHARED_LIBS_ABS_PATH, means that all dynamic libraries compiled by CMake will have the full paths, and so the problem is resolved. However, for dependencies that are not compiled with CMake (e.g., Boost, which is compiled with boostrap), the problem remains.
A short-term fix is to use brew install boost (or brew upgrade boost, if they're not using the current version, 1.66).
As for long-term, I'm not so sure. Boost isn't compiled with CMake; what else isn't? We would have to find fixes for all of those...

I was wondering if this wouldn't be fixed if we'd let the SuperBuild use boost 1.66. I saw no reason not to, until I found that boost 1.66 doesn't compile on Travis OSX, and therefore presumably some versions of OSX. Sigh.
I therefore suggest that on OSX, we currently set the USE_SYSTEM_BOOST to ON by default, and document the brew stuff in our OSX installation instructions.


Clear the KeyStateTracker when the window is blurred

While working on #1621 we noticed that the KeyStateTracker can be incorrect when keys are pressed but focus moves out of the window. We should clear the KeyStateTracker (and emit correctly) when that happens.

This is now done as part of #1621, closing.


[Task]: Give execute permission on workflow as a side effect when execute permission is given on Trigger Workflow query

So, this needs to be handled at multiple places:

In GAC Screen, when we are giving execute query permission on Trigger Workflow query, we need to give the execute workflow permission to the underlying workflow.
When creating a Trigger Workflow query action, we need to give execute workflow permission to the underlying workflow.
Whenever the execute action permission is edited for a Trigger Workflow query, the same needs to be reflected in execute workflow permission on the underlying workflow

Here, instead of giving permissions to Workflows and related resources, we have decided to implement a way in which we can control the operation on Workflows and related resources.
The reason, why we are trying out this approach is because of the reason that taking away side effect permissions is non-deterministic, in a way that we are not completely sure, that we are going to take away the correct permissions from correct roles for correct resources. Also, this approach has a lot of edge cases, and we may end up missing them.

Hey team! Please add your planning poker estimate with Zenhub @srix @ayushpahwa @Parthvi12


No type error when setting an unknown property of React component using the spread operator

TypeScript Version:  2.5.3
Code
interface IFooProps {
  key: string
  value: number
}

interface IBarProps {
  key: string
  value: number
  something: any
}

const Foo = (props: IFooProps) => {
  return (
    <div />
  )
}

const barProps: IBarProps = {
  key: 'foo',
  value: 1,
  something: 'something'
}

const Test1 = () => {
  // I'm setting the `something` property which isn't defined in the `Foo` component (compile-time error). This is a correct behavior.
  return <Foo key={barProps.key} value={barProps.value} something={barProps.something} />
}

const Test2 = () => {
  // However, I can set the `something` property using the spread operator and the TypeScript compiler will not throw any error ---> Is this a bug?
  return <Foo {...barProps} />
}

Expected behavior:

Test1 throws the [ts] Property 'something' does not exist on type 'IntrinsicAttributes & IFooProps' error.
Test2 throws the [ts] Property 'something' does not exist on type 'IntrinsicAttributes & IFooProps' error.

Actual behavior:

Test1 throws the [ts] Property 'something' does not exist on type 'IntrinsicAttributes & IFooProps' error.
Test2 throws no error.

This is the intended behavior. It's pretty common to spread in a bunch of props into a child that only needs a properly-typed subset of them, and it's very annoying to "pick off" the extra properties when you know they're going to be safely ignored. However, if you manually specify an attribute that doesn't exist, it's almost certain that you're accidently misspelling a real attribute's name, which is certain to be an error.

Automatically closing this issue for housekeeping purposes. The issue labels indicate that it is unactionable at the moment or has already been addressed.

Hm, I thought the primary purpose of using Typescript is to have strict type checking. However, if this is an intended behavior, I can't rely on that anymore - because Typescript allows me to set some non-defined properties, which may introduce (and actually introduces) some bugs in my application.


Update aws-java-sdk-dynamodb to 1.12.128

Updates com.amazonaws:aws-java-sdk-dynamodb from 1.11.1034 to 1.12.128.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.amazonaws", artifactId = "aws-java-sdk-dynamodb" } ]

labels: library-update, early-semver-minor, semver-spec-minor

Superseded by #201.


CDDA rip with UTF-8 CDDB entry fails

So I tried to encode a few audio books from the kids, some worked, dome didn't.
Turns out the ones that failed had entries with "Umlaut" in their CDDB titles and that caused an issue - to the extent that there was never a file ... or so it seems, because due to the script naming them in a bad way SMB wouldn't show them over the network, so I had to SSH to the server and rename them.
Any way to bring UTF-8 support to the ripper?

https://github.com/rix1337/docker-ripper#how-do-i-set-ripper-to-do-something-else

Good issue, please send a pull request if you deem this mergable. I'll take a look if I find the time. 👍

Please send a pull request if you want something else done by ripper. ripper.sh is just a proof of concept.

So I tried to encode a few audio books from the kids, some worked, dome didn't.
Turns out the ones that failed had entries with "Umlaut" in their CDDB titles and that caused an issue - to the extent that there was never a file ... or so it seems, because due to the script naming them in a bad way SMB wouldn't show them over the network, so I had to SSH to the server and rename them.
Any way to bring UTF-8 support to the ripper?

Did you found a solution ?
For me the File 204 Der dunkle Wächter is named '204 Der Dunkle W'$'\220\303''Chter and i can not rename it via ssh because the i can not type the filename. How did you manage to rename the folders

Sorry, I missed this ... type "mv 204" and then use the Tab key to make the OS cycle through the available files, this should do the trick.

thx :)


[feature request] Support  group and pear modifiers.

Support for the following modifiers provided by TailwindCSS.

Styling based on parent state (group-{modifier})
Styling based on sibling state (peer-{modifier})

Are there any current plans to add this feature?

I plan for railwind to be a 1:1 with tailwind, if possible even better. For now I haven't looked much at the other features but the classes, but yes, I plan to have support for every feature that tailwind has.


Twitter Agent run schedule

Determine whether the agent runs every X time period or on every mention.  Implement this schedule or response.

Reading timelines by cron may be suboptimal due to low limits https://developer.x.com/en/docs/x-api/v1/tweets/timelines/overview
Twitter also supports streams from Pro plan ($5k per month) https://developer.x.com/en/docs/x-api/tweets/filtered-stream/introduction


[Feature Request] Google Gemini Vertex AI Function Calling

The new @google-cloud/vertexai library makes it very easy to use function calling with Google Gemini Models. It would be awesome to see more models other than OpenAI that supports Function Calling in Langchain js 👍

Working on it! (But always appreciate help.)
My biggest challenge is that I haven't used OpenAI's or LangChainJS' implementation, so I'm not entirely sure I'm capturing how it is used correctly. Some simple examples and test cases of what you're looking to see would be appreciated.
I won't be using the vertex library itself, but will be using REST. See the @langchain/google-common package to see the current base package.

@afirstenberg
Yea. I am also not familiar with Langchain JS implementation, but I'll share what I know. (If I'm wrong on some parts, I'm sorry 🙏)
Function Calling with Langchain JS
I believe there are two ways:

Pass the functions or tools as inputs for .invoke

const result = await model.invoke([new HumanMessage("What a beautiful day!")], {
  functions: [extractionFunctionSchema],
  function_call: { name: "extractor" },
});

Bind the functions or tools first using .bind and then use .invoke (.bind basically attaches any call arguments passed in to all future calls)

const model = new ChatOpenAI({ modelName: "gpt-4" }).bind({
  functions: [extractionFunctionSchema],
  function_call: { name: "extractor" },
});
const result = await model.invoke([new HumanMessage("What a beautiful day!")]);

The tools argument in defined in the ChatOpenAICallOptions So we can basically update GoogleVertexAIChatInput to have the tools argument with type of StructuredToolInterface[] | ChatVertexAITool[] also.
For reference: Langchain JS OpenAI Implementation:
https://github.com/langchain-ai/langchainjs/blob/5f5507ddab86b1ce5b4c6b558d7cc10264f81a01/libs/langchain-openai/src/chat_models.ts#L433-L435
Then we convert the Structured Tools to Gemini Function schema which uses OpenAPI Schema. Vertex AI Function Calling Document
We may need to create a .createTool method in BaseChatGoogleVertexAI and update .createInstance to be able to do Multi-turn function calling (sending back function output) and support the ToolMessage too, so it would be easier to create AgentExecutor in the future.
For reference: Gemini Request body API
Lastly we need to update the .convertPrediction and GoogleVertexAIChatMessage to support content with role of "functionCall" and "function"
I believe that in the OpenAI implementation, the function and tools calls are placed as kwargs for an AIMessage.
https://github.com/langchain-ai/langchainjs/blob/5f5507ddab86b1ce5b4c6b558d7cc10264f81a01/libs/langchain-openai/src/chat_models.ts#L101-L109
I hope this is useful 👍

@afirstenberg
Here is my first draft of ChatGoogleVertexAI 👍👍 It uses @google-cloud/vertexai version 0.3.0 (0.3.1 of the library seems to have a where the response returns a promise instead)
The tools along with streaming are working.
chat_google_vertex_ai.ts
import {
  type BaseChatModelParams,
  BaseChatModel,
} from "@langchain/core/language_models/chat_models";
import { CallbackManagerForLLMRun } from "@langchain/core/callbacks/manager";
import {
  AIMessageChunk,
  HumanMessage,
  type BaseMessage,
  AIMessage,
} from "@langchain/core/messages";
import { ChatGenerationChunk, type ChatResult } from "@langchain/core/outputs";
import * as GoogleVertexAI from "@google-cloud/vertexai";
import type { BaseLanguageModelCallOptions } from "@langchain/core/language_models/base";
import type { StructuredToolInterface } from "@langchain/core/tools";
import { zodToJsonSchema } from "zod-to-json-schema";

export interface ChatGoogleVertexAIInput
  extends BaseChatModelParams,
    GoogleVertexAI.VertexInit,
    GoogleVertexAI.GenerationConfig {
  model?: string;
  tools?: StructuredToolInterface[] | GoogleVertexAI.Tool[];
  safety_settings?: GoogleVertexAI.SafetySetting[];
  convert_system_message_to_human?: boolean;
}

export interface ChatGoogleVertexAICallOptions
  extends BaseLanguageModelCallOptions {
  tools: StructuredToolInterface[] | GoogleVertexAI.Tool[];
}

function convertToGoogleVertexAITools(
  tools: StructuredToolInterface[]
): GoogleVertexAI.Tool[] {
  return [
    {
      function_declarations: tools.map((tool) => {
        return {
          name: tool.name,
          description: tool.description,
          parameters: zodToJsonSchema(tool.schema) as any,
        };
      }),
    },
  ];
}

export class ChatGoogleVertexAI extends BaseChatModel<ChatGoogleVertexAICallOptions> {
  model: string;

  vertexAI: GoogleVertexAI.VertexAI;

  safety_settings?: GoogleVertexAI.SafetySetting[];

  generation_config?: GoogleVertexAI.GenerationConfig;

  convert_system_message_to_human?: boolean;

  constructor(fields: ChatGoogleVertexAIInput) {
    super(fields);
    this.model = fields.model || "gemini-pro";
    this.vertexAI = new GoogleVertexAI.VertexAI({
      project: fields.project,
      location: fields.location,
      apiEndpoint: fields.apiEndpoint,
      googleAuthOptions: fields.googleAuthOptions,
    });
    this.safety_settings = fields.safety_settings;
    this.convert_system_message_to_human =
      fields.convert_system_message_to_human;
  }

  _llmType() {
    return "chat_google_vertex_ai";
  }

  async _generate(
    messages: BaseMessage[],
    options: this["ParsedCallOptions"],
    runManager?: CallbackManagerForLLMRun | undefined
  ): Promise<ChatResult> {
    if (!messages.length) {
      throw new Error("No messages provided.");
    }
    if (typeof messages[0].content !== "string") {
      throw new Error("Multimodal messages are not supported.");
    }
    if (this.convert_system_message_to_human) {
      messages = this.convertSystemMessages(messages);
    }

    const generative_model = this.vertexAI.preview.getGenerativeModel({
      model: this.model,
      safety_settings: this.safety_settings,
      generation_config: this.generation_config,
      tools: this.isStructuredToolArray(options.tools)
        ? convertToGoogleVertexAITools(options.tools)
        : options.tools,
    });
    const vertexAI_content = this.mapChatMessagesToVertexAIContent(messages);
    const response = (await generative_model.generateContent(vertexAI_content))
      .response.candidates[0];
    if (response.functionCall) {
      // Is function Call
      return {
        generations: [
          {
            message: new AIMessage({
              content: "",
              additional_kwargs: {
                function_call: {
                  name: response.functionCall.name,
                  arguments: JSON.stringify(response.functionCall.args),
                },
              },
            }),
            text: "",
          },
        ],
      };
    } else {
      const response_text = response.content.parts[0].text || "";
      return {
        generations: [
          {
            message: new AIMessage(response_text),
            text: response_text,
          },
        ],
      };
    }
  }

  async *_streamResponseChunks(
    messages: BaseMessage[],
    options: this["ParsedCallOptions"],
    runManager?: CallbackManagerForLLMRun
  ): AsyncGenerator<ChatGenerationChunk> {
    if (!messages.length) {
      throw new Error("No messages provided.");
    }

    const generative_model = this.vertexAI.preview.getGenerativeModel({
      model: this.model,
      safety_settings: this.safety_settings,
      generation_config: this.generation_config,
      tools: this.isStructuredToolArray(options.tools)
        ? convertToGoogleVertexAITools(options.tools)
        : options.tools,
    });
    const vertexAI_content = this.mapChatMessagesToVertexAIContent(messages);
    const streamingResp = await generative_model.generateContentStream(
      vertexAI_content
    );
    for await (const item of streamingResp.stream) {
      const text_chunk = item.candidates[0].content.parts[0].text || "";
      yield new ChatGenerationChunk({
        message: new AIMessageChunk({
          content: text_chunk,
        }),
        text: text_chunk,
      });
      await runManager?.handleLLMNewToken(text_chunk);
    }
  }

  _combineLLMOutput() {
    return {};
  }

  private mapChatMessagesToVertexAIContent(
    messages: BaseMessage[]
  ): GoogleVertexAI.GenerateContentRequest {
    let vertexAIContent: GoogleVertexAI.Content[] = [];
    messages.forEach((message) => {
      vertexAIContent.push(this.mapMessageToVertexAIPart(message));
    });
    return { contents: vertexAIContent };
  }

  private mapMessageToVertexAIPart(
    message: BaseMessage
  ): GoogleVertexAI.Content {
    const type = message._getType();
    if (typeof message.content !== "string") {
      throw new Error("Multimodal messages are not supported yet.");
    }
    switch (type) {
      case "system":
        throw new Error(`Google Vertex AI does not support system message`);
      case "ai":
        const ai_part: GoogleVertexAI.TextPart = {
          text: message.content,
        };
        return {
          role: "model",
          parts: [ai_part],
        };
      case "human":
        const human_part: GoogleVertexAI.TextPart = {
          text: message.content,
        };
        return {
          role: "user",
          parts: [human_part],
        };
      case "function":
        if (!message.additional_kwargs.function_call) {
          throw new Error(`function call is not defined in FunctionMessage`);
        }
        const function_part: GoogleVertexAI.FunctionCallPart = {
          functionCall: {
            name: message.additional_kwargs.function_call.name,
            args: JSON.parse(message.additional_kwargs.function_call.arguments),
          },
        };
        return {
          role: "functionCall",
          parts: [function_part],
        };
      default:
        throw new Error(`Unknown message type: ${type}`);
    }
  }

  private convertSystemMessages(messages: BaseMessage[]): BaseMessage[] {
    const convertedMessages: BaseMessage[] = [];

    for (const message of messages) {
      if (message._getType() === "system") {
        if (typeof message.content !== "string") {
          throw new Error(`SystemMessage content is not a string`);
        }
        convertedMessages.push(new HumanMessage(message.content));
        convertedMessages.push(new AIMessage("Understood"));
      } else {
        convertedMessages.push(message);
      }
    }

    return convertedMessages;
  }

  private isStructuredToolArray(
    tools?: unknown[]
  ): tools is StructuredToolInterface[] {
    return (
      tools !== undefined &&
      tools.every((tool) =>
        Array.isArray((tool as StructuredToolInterface).lc_namespace)
      )
    );
  }
}

pnpm add langchain zod-to-json-schema<EMAIL_ADDRESS>Try testing it 👍
import { ChatGoogleVertexAI } from "./chat_google_vertex_ai";

import { WikipediaQueryRun } from "@langchain/community/tools/wikipedia_query_run";

// Note to developers: You need to authenticate first. Take a look at https://cloud.google.com/docs/authentication/client-libraries
// Or use the following instead (Create a service account and get PROJECT_ID PRIVATE_KEY CLIENT_EMAIL in service account json)
// const llm = new ChatGoogleVertexAI({
// 	project: FB_PROJECT_ID,
// 	location: "asia-southeast1",
// 	googleAuthOptions: {
// 		credentials: {
// 			project_id: PROJECT_ID,
// 			private_key: PRIVATE_KEY,
// 			client_email: CLIENT_EMAIL,
// 		},
// 	},
// });

const llm = new ChatGoogleVertexAI({
	project: FB_PROJECT_ID,
	location: "asia-southeast1",
});

export async function test(query: string) {
	const res1 = await llm.invoke(query);

	console.log(res1);

	const stream = await llm.stream(query);

	for await (const chunk of stream) {
		console.log(chunk);
	}

	const tools = [
		new WikipediaQueryRun({
			topKResults: 3,
			maxDocContentLength: 4000,
		}),
	]

	llm.bind({tools})

	const res3 = await llm.invoke("On Wikipedia search for Google Deepmind");

	console.log(res3);
}

Hello, I'm also looking to for this feature, wondering if there is any immediate plan for add this feature into main?

Right now I am working on rewriting Vercel AI RSC demo to use LangChain including function calling. It would be awesome to have this done with more than just ChatOpenAI. :) Looking forward to this PR!

Right now I am working on rewriting Vercel AI RSC demo to use LangChain including function calling. It would be awesome to have this done with more than just ChatOpenAI. :) Looking forward to this PR!

Good timing! See #4864 which I just submitted the PR for.

Awesome, I love that! So if I am correct this snippet should work when I swap ChatOpenAI for ChatGoogle or is there anything more to edit? :)
Maybe even createOpenAIFunctionsAgent? Because it is tied to OpenAI (at least by name).
      const llm = new ChatOpenAI({
        streaming: true,
        temperature: 0,
      })
      const prompt = await pull<ChatPromptTemplate>("hwchase17/openai-functions-agent")
      const tools: StructuredToolInterface[] = functions.map(
        (fn) =>
          new DynamicStructuredTool({
            name: fn.name,
            description: fn.description || "",
            schema: fn.parameters,
            func: (fn as any).func,
          }),
      )

      const agent = await createOpenAIFunctionsAgent({
        llm,
        tools,
        prompt,
      })

      const agentExecutor = new AgentExecutor({
        agent,
        tools,
      })

      const stream = await agentExecutor.stream({
        input: messages.at(-1)?.content,
        chatHistory: messages.slice(0, -1),
      })

I'll be honest, @BleedingDev - I'm kinda turning to you for the answer to your question. 😁
I've tried to make it compatible with the OpenAI way of doing things, but the OpenAI way is something of a mess, so I'm not sure if I've gotten it right.
I'm wondering if createOpenAIFunctionsAgent() was replaced with llm.withStructuredOutput(), but @jacoblee93 is working on adding that part and probably knows better than I do.

Oh, great. I will try to validate this. To be honest it is tough to operate in this field - it is still not well supported. But I am doing this for my client, so research is important. :)
Let's see what @jacoblee93 tells us about it. Would be great to be as universal as possible.

We're looking into a more general way of performing agent outputs that takes advantage of structured output but doesn't directly use tool/function calling, but it's not ready yet!
@afirstenberg I haven't looked over the PR yet but yeah you'd theoretically be able to drop in Gemini for OpenAI for the function calling agent if it supports the same interface.

Well, hopefully I supported the interface correctly then! {:
That was certainly my goal.
(As painful as it was.)
If not - let's identify where it fell short and fix it.

When are you planing to release that? does not seem to be part of the current version.

@SebastianStehle - This should be in the latest versions of @langchain/google-vertexai and @langchain/google-vertexai-web.
What are you seeing?


patch error when fetching openroad repository

When trying to build the following counter example project from the README.md:

WORKSPACE

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rules_python",
    url = "https://github.com/bazelbuild/rules_python/releases/download/0.5.0/rules_python-0.5.0.tar.gz",
    sha256 = "cd6730ed53a002c56ce4e2f396ba3b3be262fd7cb68339f0377a45e8227fe332",
)

http_archive(name = "rules_hdl",
    sha256 = "6b4f6797b73d8a9cae9999ce890635dc725e30c958c8dae2a84bbf762791c309",
    strip_prefix = "bazel_rules_hdl-main",
    urls = [
        "https://github.com/hdl/bazel_rules_hdl/archive/refs/heads/main.zip",
    ],
)

load("@rules_hdl//dependency_support:dependency_support.bzl", rules_hdl_dependency_support = "dependency_support")
rules_hdl_dependency_support()

load("@rules_hdl//:init.bzl", rules_hdl_init = "init")
rules_hdl_init()

BUILD

load("@rules_hdl//synthesis:build_defs.bzl", "synthesize_rtl")
load("@rules_hdl//place_and_route:build_defs.bzl", "place_and_route")
load("@rules_hdl//verilog:providers.bzl", "verilog_library")

place_and_route(
    name = "counter_place_and_route",
    clock_period = "10",
    core_padding_microns = 20,
    die_height_microns = 200,
    die_width_microns = 200,
    placement_density = "0.7",
    synthesized_rtl = ":verilog_counter_synth",
)

synthesize_rtl(
    name = "verilog_counter_synth",
    top_module = "counter",
    deps = [
        ":verilog_counter",
    ],
)

verilog_library(
    name = "verilog_counter",
    srcs = [
        "counter.v",
    ],
)

counter.v

module counter(
  input wire clk,
  input wire reset,
  output [127:0] out,
);
  reg [127:0] counter_tmp;
  always @ (posedge clk or posedge reset) begin
    if (reset) begin
      counter_tmp <= 0;
    end else begin
      counter_tmp <= counter_tmp + 1;
    end
  end

  assign out = counter_tmp;

endmodule

I get the following error:
Error in patch: Unable to load package for //dependency_support/org_theopenroadproject:0001-logging-change-to-support-silence.patch: BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package.

wrapping patches entries with  Label() invocation seems to fix the issue.

👺 bazel --version
bazel 4.2.1


Rule 933120 FP: Common Words

Description
rules/php-config-directives.data has this comment:
https://github.com/coreruleset/coreruleset/blob/9875b44c0b9d91144d02df78af8e056d96ce0ffb/rules/php-config-directives.data#L14
However, it does include "engine", "extension", and "precision". I'm guessing they were accidentally added in version 4.0.0.
Confirmation
[X] I have removed any personal data (email addresses, IP addresses,
passwords, domain names) from any logs posted.

I would not call it an accident, but definitely something that will lead to false positives.
Simply removing them - as you propose - is a straightforward solution, but we also loose detection capabilities that way.
We do not really have a regex for php configuration directives, but the @pmf approach is hard to fix.
Anybody has a proposal?

I can confirm they were re-added in CRSv4 as part of the bug bounty see: https://github.com/coreruleset/coreruleset/pull/3028/files#diff-e08f25b74f02a4dfc08f6dbba996d2d811c0d67324c72c63ea97dd57e10f5af5R106
It doesn't look like these words are causing many false positives. I can't find any reports of this causing false positives, and I haven't encountered any personally. An easy fix would be to move these to PL-2, or we could move the false positive prone words to another rule using a regular expression that does something like this engine(?:\s+)?=. I'm not sure how bypassable the latter would be.

Same thinking here.
I mean the rule looks for the keywords and then it goes on to match on =. So engine is not enough. The parameter also needs to contain = which reduces FPs quite a bit I suppose.
@ssigwart have you seen FPs in the wild, or did you just stumble over the keywords when looking at the rules (which would be a perfectly valid reason for a report, it's more a question of perspective).

Thanks, @dune73 and @EsadCetiner. I'm upgrading from version 3 to 4 now and was comparing them to thwart off potential FPs.
Even with the = sign, I thought these were highly likely to have an FP when used in text areas.  For example, here are a couple plausible FPs.
Our promise = No bioengineered ingredients.

For extension pricing, see below:
Option 1 = $10
Option 2 = $20

Tour Pricing Below:
Engine = $30
Cabin = $10
Caboose = $20

I don't know a ton about the vulnerability to know the impact, but I was thinking that requiring only whitespace after would cut down on a lot of FPs. It wouldn't stop the FP on Engine = $30 though.

That's a good price for the Cabin.
Thanks for the context. Yes, FPs like this are totally possible.
This rule does not necessarily look at an exploit but it searches requests for traces of PHP code.
Also: Your thinking how to reduce likeliness of FPs is good, but unfortunately, this is very hard to express in ModSecurity rule language. The problems is we have the entire parameter and the only thing we we know is that somewhere in that parameter, there is one of the keywords.

I'm not too familiar with the rule language, but would there be a way to change the chain rule to something like this:
SecRule MATCHED_VARS "@pm %{tx.933120_tx_0}=" \
        "capture,\
        t:removeWhitespace,\
        setvar:'tx.php_injection_score=+%{tx.critical_anomaly_score}',\
        setvar:'tx.inbound_anomaly_score_pl1=+%{tx.critical_anomaly_score}'"

I haven't gotten it to actually work, but if you could remove whitespace and then check for the matched word (e.g. "engine"), followed but an equal sign, that might solve this.

Nice try, but this is not going to work. For one, I think @pmf does not do macros and I do not think you can put the match in the tx variable like to you tried. Well maybe, but it would be a lot of playing around with this and I lack the time right now.

Yeah, I just got to the point where I noticed that macro expansion isn't supported for @pm. Thanks for your help with this. I'll see what other tricks I might try.

Maybe this, but I'm not sure if there's a way to escape the macro replacement for tx.933120_tx_0.
SecRule MATCHED_VARS "@rx \b%{tx.933120_tx_0}\s*=" \
    "capture,\
    setvar:'tx.php_injection_score=+%{tx.critical_anomaly_score}',\
    setvar:'tx.inbound_anomaly_score_pl1=+%{tx.critical_anomaly_score}'"

This seems pretty good if it's okay to require only whitespace before the =. The only other question I have is on the macro expansion.
SecRule REQUEST_COOKIES|!REQUEST_COOKIES:/__utm/|REQUEST_COOKIES_NAMES|ARGS_NAMES|ARGS|XML:/* "@pmFromFile php-config-directives.data" \
    "id:933120,\
    phase:2,\
    block,\
    capture,\
    t:none,t:normalisePath,\
    msg:'PHP Injection Attack: Configuration Directive Found',\
    logdata:'Matched Data: %{TX.933120_TX_0} found within %{MATCHED_VAR_NAME}',\
    tag:'application-multi',\
    tag:'language-php',\
    tag:'platform-multi',\
    tag:'attack-injection-php',\
    tag:'paranoia-level/1',\
    tag:'OWASP_CRS',\
    tag:'capec/1000/152/242',\
    ver:'OWASP_CRS/4.1.0',\
    severity:'CRITICAL',\
    setvar:'tx.933120_tx_0=%{tx.0}',\
    chain"
    SecRule MATCHED_VARS "@pm =" \
        "chain"
        SecRule MATCHED_VARS "@rx \b%{tx.933120_tx_0}\s*=" \
            "capture,\
            setvar:'tx.php_injection_score=+%{tx.critical_anomaly_score}',\
            setvar:'tx.inbound_anomaly_score_pl1=+%{tx.critical_anomaly_score}'"

From some testing, macro extension for @rx doesn't seem to escape it. Here's an alternative. What do you think? I can add it to a PR if you want.
SecRule REQUEST_COOKIES|!REQUEST_COOKIES:/__utm/|REQUEST_COOKIES_NAMES|ARGS_NAMES|ARGS|XML:/* "@pmFromFile php-config-directives.data" \
    "id:933120,\
    phase:2,\
    block,\
    capture,\
    t:none,t:normalisePath,\
    msg:'PHP Injection Attack: Configuration Directive Found',\
    logdata:'Matched Data: %{TX.933120_TX_0} found within %{MATCHED_VAR_NAME}: %{MATCHED_VAR}',\
    tag:'application-multi',\
    tag:'language-php',\
    tag:'platform-multi',\
    tag:'attack-injection-php',\
    tag:'paranoia-level/1',\
    tag:'OWASP_CRS',\
    tag:'capec/1000/152/242',\
    ver:'OWASP_CRS/4.1.0',\
    severity:'CRITICAL',\
    setvar:'tx.933120_tx_0=%{tx.0}',\
    chain"
    SecRule MATCHED_VARS "@rx \b([^\s]+)\s*=" \
        "capture,\
        setvar:'tx.933120_possible_directive=%{TX.1}',\
        chain"
        SecRule TX:933120_possible_directive "@pmFromFile php-config-directives.data" \
            "capture,\
            setvar:'tx.php_injection_score=+%{tx.critical_anomaly_score}',\
            setvar:'tx.inbound_anomaly_score_pl1=+%{tx.critical_anomaly_score}'"

@ssigwart that looks to work but it's overengineered imo, each chained rule added makes a rule harder to read and negatively effects performance. I'd remove the false positive words from 933120 and create another rule 933121 that uses the aforementioned regex to avoid the false positive.

Thanks, @dune73. Yes, I intentionally did the @rx in the chain thinking that it would only execute if it already thought it was a potential issue. I'll update the PR.

Hey 👋
I have stumbled over this issue because I had a lot of FPs in the logs. php-config-directives.datacontains smtp as a word. This character sequence is somewhat common to appear in Bearertokens/Cookies, which resulted in Users getting blocked. Could you change rules using this file to not trigger when the keyword is within 5 letters before the string and after the string?
So a;sljfgqegSMTPaksjföalj would not trigger the rule, but asljfgqe;gSMTPaksjföalj would trigger the rule? This way, there would not be as many FPs and you could still detect attempts to send SMTP as a string.

@niklasweimann There's already a PR open here that should resolve this issue, but I'm not sure of the status of it. https://github.com/coreruleset/coreruleset/pull/3638

I've added your proposal to the agenda for the 2nd May meeting of the team, @ssigwart. Thank you very much. Feel free to join on May 20.
https://github.com/coreruleset/coreruleset/issues/3694

Thanks, @dune73. The May 20th meeting is in the middle of my workday, but thanks for the invite.

I'm not sure if this is a case of 'works as designed', but I work a lot with SAML-related software and I see two clear cases for rule 933120 that to me are FPs:

URLs containing SAMLRequest=hVLLTiNBDPyVUd8n88**sMtP**...vBoUORLU%2BS%2F%2F%2Fd8h8%3 where the value is a base64 encoded string.

SAML also defines a HTTP param called RelayState, which contains an encoded url. One common SAML-product standardizes their hostname on engine, leading to a HTTP param like RelayState=https%3A%2F%2Fengine.example.org%2F

Is this expected to hit this rule, or have I just hit an edge case that is fixable?
One major drawback is that I now have to disable the rule completely for the specific HTTP params, or I have to edit the php-config-directives.data file on every update of this package.

This is potentially something we would want to cover, namely if it's a standard product. Please open a separate bug report and add as much details as possible, namely the name of said product.


Revert "buildcmake: Use -cpmod to copy Fortran .mod files (#3159)"

This reverts commit 3cd2c7b17bb4be260d9606ad65b6c9de2a6c3429, which did not actually work, and CI did not catch it.

should it be target_link_options?

The usage of the parameter is entirely wrong, you have to specify the path to copy to and the .M file as its own command.
I tried target_link_libraries just to be sure and it gave:
CMake Error at src/libs/ck-libs/ampi/CMakeLists.txt:479 (target_link_libraries):
  Object library target "ampimod" may not link to anything.

target_link_options was added in CMake 3.13.


Additional enemies

Adds the toast enemy which is fairly strong and at a speed of 0.8.
Adds a goo monster enemy which is super strong but very very slow.
Does a little level setting on level 2 so that you actually need the right combination of towers to beat it.

Looks Great!


Object-oriented refactor of Suggest Gems

@1337LutZ @TheGroxEmpire Finally finished the Suggest Gems refactor to make it object-oriented and extensible to different types of sims without copy-pasting code. Would appreciate if you can take a look before I merge.

Super nice work <3 Added some comments in regards of TS / gem orders.
Will build it locally and see if it behaves the same as the current one :)

Created a build and looks like it is gemming sometimes even better (esp prioing the tear socket) for most specs 👍


Typo "Azure SQL database"→”Azure SQL Database”

https://docs.microsoft.com/en-us/azure/azure-sql/migration-guides/database/access-to-sql-database-guide
#PingMSFTDocs

@hyoshioka0128 : Thanks for your contribution! The author(s) have been notified to review your proposed change.

Duplicate #74647
CLosed #76191


Objective: To drastically automate the documentation development process by templatizing a starter XD doc

Objective: To drastically automate the documentation development process by templatizing a starter XD doc
Feature request
Create an XD template system that contains all the various section patterns that can be:
turned on and off.
Stacked and repositioned while maintaining spacing
Next steps
Audit the current ux.redhat.com site
Meet with the design system team members to work out XD approach and implementation
Create a how-to-use document and  added to the team process documentation
UX Designer Deliverable:
Final  XD template & usage documentation
Originally posted by @ronniemcbride in https://github.com/RedHat-UX/red-hat-design-system/discussions/352

Thanks for opening this Ronnie
I'm anxious to ship design-tokens for consumption in XD, could this be a place to do that? e.g. rolling tokens into the templates? Yes, ideally this would be automated (occur every time tokens makes a release) and decoupled (tokens can update independent of templates and vice-versa), but so far in my cursory investigation into the xd ecosystem, there's no straightforward way to accomplish that

Sounds good!
From a development perspective, it would be fairly straightforward to replicate the demo page at red-hat-design-tokens.netlify.app anywhere in ux-dot. Likewise, it would be easy to pull in individual sections of that page anywhere.
The content for those sections is defined within the tokens source as markdown strings, so we can replicate that anywhere.


Fix naming

Release Notes Description (public)

Implementation details (internal)
Fix imagePullSecrets naming and missing "v" for node multi env url
Information for QA

[ ] Is QA testing required?
[ ] Does PR contain unit tests?
[ ] Should QA create E2E tests for the change?

Additional QA Procedures (Optional)

Screenshots (if appropriate):

Can we test it by switching to the branch in preprod instead of bumping helm charts?

yes


PolylineDecorator always top layer even when calling bringToBack()

Got an issue using polylineDecorator with arrowHead symbols as sometimes the arrows heads can overlap other layers on the map and prevent you from being able to hover them to read an associated tooltip. I've tried to use bringToBack() on the polylineDecorator layer but it just seems to do absolutely nothing, the arrow heads are always the very top layer.

Hello @TLDC  did you find a solution to this?  I am having the same issue.

@TLDC @stharvey
I think this had to do with the way the polyline decorator works. It appears to be drawing each pattern on a map level layer. So doing something like myDecorator.addTo(myFeatureGroup) is essentially doing nothing.
For my use case, I want the decorators at the bottom of the layer stack at all times so I was able to resolve this issue by overriding the "onAdd" function. More specifically, the this._map.on('moveend') event:

    const myDecorator = L.polylineDecorator(yourOptions);

    myDecorator.onAdd = function(map) {
      this._map = map;
      this._draw();
      // original line:
      // this._map.on('moveend', this.redraw, this)
      // new line:
      this._map.on('moveend', ()=> { this.redraw(); this.bringToBack(); }, this);
    }

    myDecorator.addTo(myLayer);

The polyline decorator attempts to redraw on pan/zoom which moves everything back to the top of the stack without this change.


Attachment actions inputs property incorrectly calling dictionary

Receiving a TypeError: 'collections.OrderedDict' object is not callable error when accessing .inputs on an AttachmentAction.
In mixins/attachment_action.py, on line 73:
return self._json_data('inputs') should be return self._json_data.get('inputs')

+1 ran into the same problem. Accessing the json dict directly works as a work-around
attachment = api.attachment_actions.get(attachment_id)
inputs = attachment._json_data.get('inputs')

Thank you for catching and fixing the bug.  🙂  I will merge the PR.