Invalid Cast Exception when running code interactive on Mono

Hi I don't know if this is the right place, but  I am going to post it here because I don't know where else to report it.
When I run the following code in FSharpi on Mono
open System
open System.Diagnostics

type Option<'a> =
    | None
    | Some of 'a

type Result<'TSuccess,'TFailure>=
    | Success of 'TSuccess
    | Failure of 'TFailure

let Trimchars = if Environment.OSVersion.Platform.Equals("Unix") then "\n" else "\r\n"

let RunCommand command parameters =
    let startInfo = new ProcessStartInfo()
    startInfo.CreateNoWindow <- true;
    startInfo.UseShellExecute <-false;
    startInfo.FileName <- command
    startInfo.Arguments <- parameters
    startInfo.RedirectStandardOutput <- true;
    use proc = Process.Start(startInfo);
    let output = proc.StandardOutput.ReadToEnd();
    proc.WaitForExit()
    output.TrimEnd(Trimchars.ToCharArray())

let GetZpoolNames() =
    let output = RunCommand "/sbin/zpool" "list"
                    |> (fun x -> x.Split(Trimchars.ToCharArray(),StringSplitOptions.RemoveEmptyEntries))
                    |> List.ofArray

    if  output.Head |> (fun x -> x.Equals("no pools available"))
    then None
    else output
         |> List.tail
         |> List.map(fun x -> x.Substring(0, x.IndexOf(" ")))
         |> (fun x -> Some(x))

When I run:
GetZpoolNames()

I get the following error:
Please report this so it can be fixed.ing list>
Trace: System.InvalidCastException: Cannot cast from source type to destination type.
  at (wrapper castclass) object:__castclass_with_cache (object,intptr,intptr)
  at Microsoft.FSharp.Compiler.Interactive.Shell+Utilities+AnyToLayoutSpecialization`1[FSI_0002+Option`1[Microsoft.FSharp.Collections.FSharpList`1[System.String]]].Microsoft-FSharp-Compiler-Interactive-Shell-Utilities-IAnyToLayoutCall-AnyToLayout (Internal.Utilities.StructuredFormat.FormatOptions options, System.Object o) [0x00000] in <filename unknown>:0
  at Microsoft.FSharp.Compiler.Interactive.Shell+FsiValuePrinter.PrintValue (Microsoft.FSharp.Compiler.Interactive.FsiValuePrinterMode printMode, Internal.Utilities.StructuredFormat.FormatOptions opts, System.Object x, System.Type ty) [0x00000] in <filename unknown>:0                                              val it : Option<string list> =

If I compile the program it runs fine.
output:
pool
storagepool

Am I doing something wrong?
Info:
Debian 8.1 Linux
Mono:

Mono JIT compiler version 4.0.1 (tarball Thu May 28 09:08:28 UTC 2015)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
        TLS:           __thread
        SIGSEGV:       altstack
        Notifications: epoll
        Architecture:  amd64
        Disabled:      none
        Misc:          softdebug
        LLVM:          supported, not enabled.
        GC:            sgen

Looks related to #426.  Could you add a Mono/Xamarin buzilla bug on this please?

Very likely fixed by https://github.com/mono/mono/pull/1654 but hard to verify without /sbin/zpool

I think I figured it out, after trying to recreate it with some standard output pasted as a variable it worked.
The self defined option type was tripping it up.
type Option<'a> =
| None
| Some of 'a
This trips up the runtime.
My apologies if this is normal behaviour.

@boeremak - do you have a standalone repro for how the self-defined option type causes problems for the runtime?  It's possible that using a self-defined option type exposed you to the bug fixed by mono/mono#1654 - but with that fix in place it there should be no problem with using a self-defined option type.

(closing this as we believe it fixed by mono/mono#1654)


BasicTracer package missing

The README.md mentions the BasicTracer included in this package, but there is no such package. Furthermore, https://github.com/opentracing/basictracer-csharp is an empty repository.
Is there another place hosting this sample code, or is this something that we should add? I believe @cwe1ss has some code on his repository that was looking for a home - perhaps this library should reference the basictracer-csharp one, and the code be hosted there?

@ndrwrbgs not sure if I should add this here, but I have a zipkin v2 tracer at Jasiri

There‘s a branch with some code in that repository but it’s just a prototype.

"Basic Tracer" is a topic that has been mentioned a few times on gitter and in the cross-language meeting. It exists in some languages and it doesn't exist in some others (e.g. Java).
I'm moving it to "Backlog" as I'd rather invest some time in getting a proper tracer library done ASAP.
Feel free to comment here if you want to work on this.

Just out of curiosity, why is the default implementation of span and/or span builder missing?

@cwe1ss would you be okay with me contributing an implementation?
Any 'base' implementation I would make would likely be close the interfaces (given I translated those from java 😜) but would provide a plug-in-able way to configure the SpanId's to be able to support https://github.com/Microsoft/CorrelationVector as a provider of SpanIds (but without taking a dependency on that package).
Does this sound okay or worthless or duplicate work form something existing, etc?
The main thing I want is something that can do traces in an easy way, provide context propagation, but not keep them all in memory like MockTracer does.

@ndrwrbgs sure, but I wonder if it's worth it, given that OpenTelemetry is making (slow) progress?! I haven't been involved in any tracing relevant things over the last year though (I'm renovating a house), so I don't really know anything about the OpenTelemetry timeline.

I feel I'm doing a disservice to the community - given the current situation is "the old thing is deprecated but the new thing isn't ready yet" - but I'm going to close this to shorten my backlog and be more realistic. Might open a new one later if it comes to it :-)


Fix duplicate AssemblyName instances

Fixes https://github.com/microsoft/vs-mef/issues/251

Codecov Report

Merging #252 (5759f82) into main (0fbe01a) will decrease coverage by 2.36%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #252      +/-   ##
==========================================
- Coverage   89.59%   87.23%   -2.37%
==========================================
  Files          87        3      -84
  Lines        6009       47    -5962
==========================================
- Hits         5384       41    -5343
+ Misses        625        6     -619

Impacted Files
Coverage Δ

...Studio.Composition/RuntimeExportProviderFactory.cs

...sualStudio.Composition/DelegatingExportProvider.cs

...rosoft.VisualStudio.Composition/ByValueEquality.cs

...dio.Composition/Configuration/CachedComposition.cs

...position/Configuration/SerializationContextBase.cs

...udio.Composition/ExportProvider+NonSharedExport.cs

...ft.VisualStudio.Composition/IndentingTextWriter.cs

...isualStudio.Composition/Reflection/ParameterRef.cs

...mposition.AppDomainTests/ExternalExportWithLazy.cs

...o.Composition/Reflection/StrongAssemblyIdentity.cs

... and 72 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0fbe01a...5759f82. Read the comment docs.

Just tested this PR locally, down to 152 AssemblyName instances (19kb)

@AArnott PTAL

@therzok Thanks again!


[SPARK-5519][MLLIB] add user guide with example code for fp-growth

Test build #27653 has started for   PR 4661 at commit 58ccc25.

This patch merges cleanly.

Test build #27653 has finished for   PR 4661 at commit 58ccc25.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
public class JavaFPGrowthExample

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27653/
Test PASSed.

Merged into master and branch-1.3.


Add building=riding_hall as new type

Description
Currently building=riding_hall can only be entered manually in the tags but not selected via the search. It would be more convenient if it would be available.
See https://wiki.openstreetmap.org/wiki/Tag:building%3Driding_hall
Screenshots
No response

Hi Simon - you'll need to file this request over on https://github.com/openstreetmap/id-tagging-schema - that's the repository where presets that show up in search are configured. I don't have permissions to just transfer it across, so I'm going to close it here, but encourage you to open a new issue there (sorry for the extra work)

See https://github.com/openstreetmap/id-tagging-schema/issues/1012


[SQL] JSON datatype return value and TEXT type workaround.

Hi,
Thanks for your hard work.
I am trying to use the JSON datatype in an SQLite database.
I opened a previous issue about this here https://github.com/tauri-apps/plugins-workspace/issues/93
Thanks to @FabianLars for suggesting to use a text field and preform parsing and deserialization manually.
Works just fine but trying to use the JSON datatype was still confusing.
SQLite supports JSON as a datatype although it is stored as text, listed in the docs here.
https://www.sqlite.org/json1.html
And when I attempted to use it with the SQL plugin I was able to store the information in the SQLite database but on retrieval a null value is returned.
If the plugin is not going to support deserialization then rather than returning NULL for JSON datatypes returning the string representation or simply an error would be better I believe.
I think that this would make it easier for a developer to apply this workaround.

I opened a previous issue about this here https://github.com/tauri-apps/plugins-workspace/issues/93

Next time please give me more than 5 minutes to react. 😂 I was in the process of writing down my thoughts and may have re-opened it.
So until now we tried to somewhat follow the actually supported datatypes (the 5 at the top here https://www.sqlite.org/datatype3.html) and i could write X paragraphs about how it should stay that way since JSON is not an actual datatype, and what about all the other non-existent datatypes that work in sqlite due to its typeless nature (take a look at 3.1.1. Affinity Name Examples in the link i send for example) blabla
But i do realize that JSON is a really "special" case here and we should at least handle it like a TEXT column (prior discussions/PRs always revolved around deserializing it via serde_json to have automatic nested json support which imo still should not be the plugin's responsibility, at least for sqlite)

Thanks so much!


Add Release Notes/Changelog

Could you please add release notes detailing what breaking changes came with version 1.0.0?
I've seen people add a CHANGELOG.md and/or add release notes here.

@dommmel any update on this?

Taking a quick look at the code:
1.1.0

Bugfix to preserve form data encoding and escape form field names. #42

1.0.0

Bugfix to escape parenthesis in form data. #40

Even though it has been a really long time since any action has been performed on this repo, I will look into doing this. Thanks!

Great! Thank you.

Just curious, are you still actively using this library? It may be time to 2.0.0 and add some quality of life features/fixes.

I'm not, but it looks like 1300 people a week are downloading it!
https://www.npmjs.com/package/fill-pdf

FYI a typical best-practice is to keep issues open until they are resolved.
Thanks for this library! It was helpful in the past and I was happy to contribute.


Home: Add index to optimize imported form queries

For the home page.

selfie

cuts query cost to 20% of what it was


docs: update storage package documentation

Description

This PR is part of a series of PRs aimed at improving the documentation of our publicly exported packages.
Once all exported structs, vars, funcs are documented we can proceed with enforcing these good practices through the golangci linter configuration by enabling the revive and stylecheck linters.
Notes
While improving the documentation and analyzing the storage packages I also encountered a few opportunities for a potential future vNext of the publicly exposed packages:

pkg/storage/test pkg should be removed and test functions migrated to _test.go files within the storage pkg.
Codebase should not have any un-exported returns.

For example: https://github.com/openfga/openfga/blob/d897d391831ee76cda1a7d818808a2ab590bc3da/pkg/storage/storagewrappers/boundedconcurrency.go#L42

exported: type name will be used as memory.MemoryBackend by other packages, and that stutters; consider calling this Backend

Perhaps we should rename:

memory.MemoryBackend to memory.Storage
mysql.MySQL to mysql.Storage
postgres.Postgres to postgres.Storage

Or alternatively keep everything within the storage pkg and then name the above as storage.MySQL, storage.Postgres, storage.Memory. This would be preferred.

Further good content on package design: https://dave.cheney.net/practical-go/presentations/gophercon-singapore-2019.html#_package_design
References

https://github.com/openfga/openfga/pull/1261
https://github.com/openfga/openfga/pull/1263
https://github.com/openfga/openfga/pull/1264

Review Checklist

[x] I have clicked on "allow edits by maintainers".
[x] I have added documentation for new/changed functionality in this PR or in a PR to openfga.dev [Provide a link to any relevant PRs in the references section above]
[x] The correct base branch is being used, if not main
[ ] I have added tests to validate that the change in functionality is working as expected

@sergiught

pkg/storage/test pkg should be removed and test functions migrated to _test.go files within the storage pkg.
This was actually on purpose as we want to have a public suite of storage conformance tests that developers can import if they implement the OpenFGADatastore interface with their own database choice. So we do want to keep this as a public export. For example, we use this storage conformance test suite in Auth0 FGA for the DynamoDB implementation that we implement.

@jon-whit

This was actually on purpose as we want to have a public suite of storage conformance tests that developers can import if they implement the OpenFGADatastore interface with their own database choice. So we do want to keep this as a public export. For example, we use this storage conformance test suite in Auth0 FGA for the DynamoDB implementation that we implement.

Oh I see! Smart! 👍🏻 Thanks for the clarity on that one.

What's wrong with that? You see that in the standard Go lib in various places. If you don't want a direct dependency on the struct but you want to encourage the usage of the interface it implements, then this is an idiomatic way of doing so. Alternatives I've seen such as os.File is that you export the struct but don't export any of its members.

This was actually coming directly from the revive linter and it can be problematic for a few reasons:

Usability: External code that calls this function can receive an instance of this type, but since the type is unexported, the calling code cannot directly use or manipulate the returned object in a strongly-typed manner. This limits how the object can be used and may force the caller to rely on interfaces or reflection.
Documentation and Tooling: Many IDEs and documentation tools in Go use type information to provide hints, autocompletion, or documentation. If the type is unexported, these tools cannot offer detailed information about the type, making it harder for developers to understand or use the returned object.

However the solution to fixing that specific linter error is already within your comment:) If we want to enforce interfaces, instead of returning *boundedConcurrencyTupleReader we simply need to return storage.RelationshipTupleReader (the interface) and that will cull the linter error.


can't get resource

I can't get resource at link example.com/v1/resource/abc.png
can you explain me how to fix it or what i forgot when config ? Thank

Should work. Want me to guess what you might have forgotten? ;)

Feel free to re-open this issue if you can provide a working example of this failing. I suspect the recent fix for #80 may have something to do with this as well, but cannot tell without more information.

Our rest api doesn't work anymore. Still got a "error: 404" message. The rest api framework can't find the @url /notifications/getAll path with the fix 4024098ed93a4
\Jacwright\RestServer\RestServer::getPath()
Line 368
Before ltrim()

notifications/getAll

After ltrim()

/notifications/getAll

Our system call the rest api on url <domain.tld>/cms-shop/rest/notifications/getAll and received this:
{
    "error": {
        "code": 404,
        "message": "Not Found"
    }
}

Is this a regression? If so please open a new bug. In the meantime, I'm looking into it.


Fix golint error and make it enable again in travis

Pull Request Test Coverage Report for Build 810

6 of 17 (35.29%)  changed or added relevant lines in 5 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage increased (+0.04%) to 54.851%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

types/conf.go
1
2
50.0%

kubeletclient/kubeletclient.go
1
3
33.33%

logging/logging.go
0
8
0.0%

Files with Coverage Reduction
New Missed Lines
%

logging/logging.go
1
39.51%

Totals

Change from base Build 809:
0.04%

Covered Lines:
701

Relevant Lines:
1278

💛  - Coveralls


Determine policy around MSRV and new MSRV

I suspect I'll be bumping the MSRV to at least 1.46 to take advantage of const fn, but more broadly, I'd like a policy here.
More broadly, I'd like to find a more tenable strategy for MSRV than either:

waiting for our deps to force us to bump it.
forcing latest stable.

My inclination here is to adopt a policy of stable - N (e.g. stable - 1 would mean we support stable and the previous release). I'm thinking N is between 1 and 3. (It seems like a good balance that's predictable, avoids the worst problems of "latest stable", while not forcing you to make the semver number useless).
The other option I'm fond of is to consider it a minor (but not major) bump, which is not really relevant here, but avoids the version becoming almost entirely meaningless.

I think the minor bump is the right call.
Most large projects will be using a fork, or a completely specified version, of this project anyway, so "surprising" someone with an update, when they aren't updated rustc, would surprise me.
I dislike keeping up with stable Rust since this forces significant amounts of updates just to keep to MSRV policy.

The issue with it being a minor bump is that prior to 1.0 (which I'm not ready for) is that it's equivalent to no policy at all.

the underlying idea here is that you intend to move the repo to be MSRV on the new update of Rust which is expected to have min_const_generics, right?

I meant the const fn changes of 1.46, but it's possible there's some array code that would benefit from min_const_generics.

For tools where they are likely to be packaged by distros, using a minimum version of "whatever is packaged in most distros" is a good approach to MSRV policy, however that probably isn't a major consideration for imgui-rs.
Otherwise I'd be happy with incrementing it "as needed" - e.g bumping the MSRV for a minor helper method which reduces 2 lines to 1 is probably not worthwhile, but for something like the const-generics, sure.
I don't have strong opinions on if bumping the MSRV being a major bump versus minor etc. The thing I do have strong opinions on is: bump the version intentionally, and note the change in the changelog
So on these lines, I've made a PR to test the current minimum version in CI:
https://github.com/imgui-rs/imgui-rs/pull/512
That way if a PR adds usage of something added after 1.48 it'll be easy to spot - and either the version can be bumped or the PR be altered

Okay -- as of commit 04af6ab69af61246a2d152a3e2755d02cdabb7a4, I've committed our MSRV to 1.54 (updating our CI to reflect this) and noted it in the readme. For only this release, I've added a feature gate around min-const-generics (which is used in the tables-api a bit, and I'd like to use in the string api too) and added it as a default feature. In 0.9.0, that'll be removed and all msrv jumps will have the minor break in them, but since this is the first intentional one, it felt fair to give a cushion.
I'm going to close this issue, but if anyone would like to debate or change this, feel free to comment and I'll reopen


Support Mixtral

Related?
https://huggingface.co/casperhansen/mixtral-instruct-awq-it1/tree/main
https://github.com/casper-hansen/AutoAWQ/commit/9c3dfa078f82b164d886c1378784e8d1510a2a97
Also see:
https://huggingface.co/ybelkada/Mixtral-8x7B-Instruct-v0.1-AWQ
https://github.com/huggingface/transformers/pull/27950
Right now get:
  File "/home/jon/miniconda3/envs/h2ogpt/lib/python3.10/site-packages/awq/models/auto.py", line 50, in from_quantized
    model_type = check_and_get_model_type(quant_path, trust_remote_code)
  File "/home/jon/miniconda3/envs/h2ogpt/lib/python3.10/site-packages/awq/models/auto.py", line 25, in check_and_get_model_type
    raise TypeError(f"{config.model_type} isn't supported yet.")
TypeError: mixtral isn't supported yet.

Or is this something that has to be only done in transformers?  Sorry I get confused between what is in transformers vs. here.

Hi @pseudotensor, support is coming! #251 Before the model is supported, in figuring out an effective scaling of its layers.

Great, looking forward.  Love AWQ stuff, always works better than llama.cpp, esp. for heavy use in vLLM.

That’s why I keep working on it! Admittedly, this is a very time consuming task:

Make tweak to code
Quantize model (wait 40 minutes)
Measure perplexity
Repeat

Mixtral support is on main now.

Howdy @casper-hansen Curious what you did with your AWQ process for Mixtral.  Do you have your steps documented/coded?  For https://huggingface.co/casperhansen/mixtral-instruct-awq.
I ask because we checked out many AWQ Mixtrals, e.g.

TheBloke/dolphin-2.7-mixtral-8x7b-AWQ : Bad repetition
TheBloke/mixtral-8x7b-v0.1-AWQ : Doesn't even start to generate, as others have complained
ybelkada/Mixtral-8x7B-Instruct-v0.1-AWQ : Bad repetition


Remove circe-jackson dependency

Fixes #891.

Codecov Report

Merging #916 into master will decrease coverage by 0.06%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #916      +/-   ##
==========================================
- Coverage   82.79%   82.73%   -0.07%
==========================================
  Files          52       52
  Lines         779      776       -3
  Branches       37       36       -1
==========================================
- Hits          645      642       -3
  Misses        134      134

Impacted Files
Coverage Δ

circe/src/main/scala/io/finch/circe/package.scala
100% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c33218d...bad9258. Read the comment docs.


Multiline-Symbol literals do not increase the line counter

Bug Report
When using symbols which span accross multiple lines, the line counter is not increased.
For example this code:
symbol = :"M
U
L
T
I
L
I
N
E"
puts __LINE__

It prints 2 to the terminal, while it is actually on line 10.

Should multi-line symbols be allowed by the compiler? They would be very inconvenient to use.

I think they could be useful for macros.

The idea of quoted symbol identifiers is to allow almost any character as part of the symbol name. If that really needs to include LF and I don't know what kind of control characters...? 🤷 But technically it's not a big issue, there's just a minor bug in the symbol parsing logic which doesn't count the lines.
So while the use case is questionable it's probably easiest to just fix this bug and allow the usage of LF in quoted symbol identifiers. There's no real harm from this.

it also doesn't compile with CRLF

That's because CRLF are two characters and a char literal contains exactly one character.


Chat Push Notification

I created a new branch for Chat Push Notification.
This branch is based on Chat 1.1.0 GA release and contains all the code changes in the old PN PR (https://github.com/Azure/azure-sdk-for-ios/pull/1154).
APIView: https://apiview.dev/Assemblies/Revisions/993b3da970c24d24a05f035d674b7525

/azp run ios - client - ci


context class loader during object deserialization problem?

// use context class loader during object deserialization.
// thanks Daniel Peters!
sessionFactory.setClassLoader(getContext().getClassLoader());
log.info("use context class loader for session deserializer.");
// FIXME: is there any safe way to refer context's class loader?
// getContext().getClassLoader() may raise SecurityException.
// this will be determine by policy configuration of JRE.
About FIXME, I think you should try set Thread.currentThread().getContextClassLoader() again,otherwise, the kryo doesn't work without the security policy.

Please send me back some PR if you still have this problem.


Update Gateway status on ReferenceGrant changes

What this PR does / why we need it:
Reconciles a Gateway if the controller sees a ReferenceGrant that has a From clause for the Gateway.
Update Gateway statuses on every reconcile if they have changed, not just on creation.
Changes Gateway(Class) deploy test helpers to delete existing resources before creating new ones, to ease cluster re-use.
Which issue this PR fixes:
fix #2785
fix #2786
Special notes for your reviewer:
Contrary to #2785, this does not involve #2694, as the ReferenceGrant Condition is part of Listener status, not the Gateway status. Although Listener statuses are part of the Gateway status, they have their own Condition sets. I don't believe we needed anything to make Listener status conditions a snapshot other than updating them on each reconcile (not doing this earlier was a bug), since we generate them from scratch each time (we do use the previous conditions to make decisions about how to generate the new set, but we do not append them).
PR Readiness Checklist:
Complete these before marking the PR as ready to review:

[x] the CHANGELOG.md release notes have been updated to reflect any significant (and particularly user-facing) changes introduced by this PR

Just lists the status fix; the rest is tests or part of the ReferenceGrant changes that are already staged for 2.6.

@mlavacca can you approve as well? Apparently GH is stuck on the old review--not really sure, but I can't see any reason it's indicating your change request is still active.


Tutorial03 (rllib) fails to run due to Exception: Unknown config parameter sgd_batchsize

Choose an option
This is a:

[ ] Question
[ ] Feature request
[X] Bug report

Describe the issue
I wanted to run the rllib tutorial from https://github.com/flow-project/flow/blob/master/tutorials/tutorial03_rllib.ipynb . However, it fails with the following error message (in 4.4 Running Experiments)
== Status ==
Using FIFO scheduling algorithm.
Resources requested: 0/2 CPUs, 0/0 GPUs
Memory usage on this node: 2.1/4.1 GB
Created LogSyncer for /home/andreas/ray_results/ring_stabilize/PPO_WaveAttenuationPOEnv-v0_0_2019-02-03_14-36-014h1eblcc ->
== Status ==
Using FIFO scheduling algorithm.
Resources requested: 2/2 CPUs, 0/0 GPUs
Memory usage on this node: 2.1/4.1 GB
Result logdir: /home/andreas/ray_results/ring_stabilize
RUNNING trials:

PPO_WaveAttenuationPOEnv-v0_0:	RUNNING

Error processing event.
Traceback (most recent call last):
File "/home/andreas/ray/python/ray/tune/trial_runner.py", line 261, in _process_events
result = self.trial_executor.fetch_result(trial)
File "/home/andreas/ray/python/ray/tune/ray_trial_executor.py", line 211, in fetch_result
result = ray.get(trial_future[0])
File "/home/andreas/ray/python/ray/worker.py", line 2386, in get
raise value
ray.worker.RayTaskError: ray_PPOAgent:train() (pid=9977, host=andreas-kubuntu1804)
File "/home/andreas/ray/python/ray/rllib/agents/agent.py", line 244, in init
Trainable.init(self, config, logger_creator)
File "/home/andreas/ray/python/ray/tune/trainable.py", line 87, in init
self._setup(copy.deepcopy(self.config))
File "/home/andreas/ray/python/ray/rllib/agents/agent.py", line 304, in _setup
self._allow_unknown_subkeys)
File "/home/andreas/ray/python/ray/tune/util.py", line 63, in deep_update
raise Exception("Unknown config parameter {} ".format(k))
Exception: Unknown config parameter sgd_batchsize
Replacing
config["sgd_batchsize"] = min(16 * 1024, config["timesteps_per_batch"])  # stochastic gradient descent
by
config["train_batch_size"] = 128
and removing
config["timesteps_per_batch"] = HORIZON * N_ROLLOUTS  # batch size
I receive another error message as follows:
== Status ==
Using FIFO scheduling algorithm.
Resources requested: 0/2 CPUs, 0/0 GPUs
Memory usage on this node: 2.4/4.1 GB
Created LogSyncer for /home/andreas/ray_results/ring_stabilize/PPO_WaveAttenuationPOEnv-v0_0_2019-02-03_14-52-21k54fo275 ->
== Status ==
Using FIFO scheduling algorithm.
Resources requested: 2/2 CPUs, 0/0 GPUs
Memory usage on this node: 2.4/4.1 GB
Result logdir: /home/andreas/ray_results/ring_stabilize
RUNNING trials:

PPO_WaveAttenuationPOEnv-v0_0:	RUNNING

Error processing event.
Traceback (most recent call last):
File "/home/andreas/ray/python/ray/tune/trial_runner.py", line 261, in _process_events
result = self.trial_executor.fetch_result(trial)
File "/home/andreas/ray/python/ray/tune/ray_trial_executor.py", line 211, in fetch_result
result = ray.get(trial_future[0])
File "/home/andreas/ray/python/ray/worker.py", line 2386, in get
raise value
ray.worker.RayTaskError: ray_PPOAgent:train() (pid=10279, host=andreas-kubuntu1804)
File "/home/andreas/anaconda3/envs/flow/lib/python3.5/site-packages/gym/envs/registration.py", line 143, in spec
return self.env_specs[id]
KeyError: 'WaveAttenuationPOEnv-v0'
During handling of the above exception, another exception occurred:
ray_PPOAgent:train() (pid=10279, host=andreas-kubuntu1804)
File "/home/andreas/ray/python/ray/rllib/agents/agent.py", line 244, in init
Trainable.init(self, config, logger_creator)
File "/home/andreas/ray/python/ray/tune/trainable.py", line 87, in init
self._setup(copy.deepcopy(self.config))
File "/home/andreas/ray/python/ray/rllib/agents/agent.py", line 312, in _setup
self._init()
File "/home/andreas/ray/python/ray/rllib/agents/ppo/ppo.py", line 75, in _init
self.env_creator, self._policy_graph)
File "/home/andreas/ray/python/ray/rllib/agents/agent.py", line 413, in make_local_evaluator
config["local_evaluator_tf_session_args"]
File "/home/andreas/ray/python/ray/rllib/agents/agent.py", line 553, in _make_evaluator
output_creator=output_creator)
File "/home/andreas/ray/python/ray/rllib/evaluation/policy_evaluator.py", line 217, in init
self.env = env_creator(env_context)
File "/home/andreas/flow/flow/utils/registry.py", line 106, in create_env
return gym.envs.make(env_name)
File "/home/andreas/anaconda3/envs/flow/lib/python3.5/site-packages/gym/envs/registration.py", line 167, in make
return registry.make(id)
File "/home/andreas/anaconda3/envs/flow/lib/python3.5/site-packages/gym/envs/registration.py", line 118, in make
spec = self.spec(id)
File "/home/andreas/anaconda3/envs/flow/lib/python3.5/site-packages/gym/envs/registration.py", line 153, in spec
raise error.UnregisteredEnv('No registered env with id: {}'.format(id))
gym.error.UnregisteredEnv: No registered env with id: WaveAttenuationPOEnv-v0
Could it be that the installation instructions point to a wrong version of rllib or that the assignment of env_names and gym_names is not correct in the notebook?
To Reproduce
After installing flow and rllib acording to the installation manual on https://flow.readthedocs.io/en/latest/flow_setup.html#installing-flow-and-sumo and https://flow.readthedocs.io/en/latest/flow_setup.html#optional-install-ray-rllib, I wanted to run the rllib tutorial https://github.com/flow-project/flow/blob/master/tutorials/tutorial03_rllib.ipynb .
Leave empty if not applicable.

File in which error occurred.

Exact command that was run that caused the error to occur.

Please attach the full error output log.
tutorial03_rllib.zip

Expected behavior
A clear and concise description of what you expected to happen.
The tutorial should run through without errors when following the installation instructions.
Screenshots
If applicable, add screenshots to help explain your problem.
Leave empty if not applicable.
Desktop (please complete the following information):

OS: kubuntu 18.04 with anaconda
Commit number of flow: 1067b8c690c42d4326bb960ec67e2483535b35e1
Version: 0.3.0
Leave empty if not applicable.

Additional context
Add any other context about the problem here.

hey @ae137 thank you for bring this up. The tutorial is out-of-date with respect to the current version of ray we are using (0.6.1). There is a new PR up to resolve this. Feel free to use it in the meantime! (see #408)

Hello @AboudyKreidieh .. Thank you for mentioning the change. It was very useful for me as well. I have tried to run the example with the new code lines but still I couldn't run it. It is giving the similar error saying "NotJSONError('Notebook does not appear to be JSON: '{\n "cells": [\n {\n "cell_type": "m...',) ".  Is there something else which I have to do it apart from these changes?

the fix has been merged into master, so closing. If you have any other questions, feel free to start another issue or reopen this one

Thanks for fixing!


chore(eslint): add eslint configuration

:tada: This PR is included in version 0.2.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


quick keys

It would be nice to have a little number/character next to each item in the list (1-9,a-z) that would let you quickly switch to that tab by hitting a modifier+key.  I find that I have a lot of tabs that have similar names and I end up having to type quite a bit of the name and then arrow up/down to get to the one I want. Just a thought.
Amazing extension, makes living the browser so much easier.

This actually already works natively in chrome in chrome.

Ctrl+1 through Ctrl+8	Switches to the tab at the specified position number on the tab strip.
Ctrl+9	Switches to the last tab.


UnicodeDecodeError caused by setup.py on Python 3

When installing dateparser on a system which has no or non-utf8 locale variables set (e.g. RTD’s servers), installation fails because you’re making use of the open() command in setup.py to read things like README.rst and HISTORY.rst. This happens because Python 3 tries loading those files with the default encoding (which happens to be ASCII), and then fails with an UnicodeDecodeError.
You could use io.open(), which is available for both Python 2 and 3 and specify the encoding of said files explicitly.
One could argue that this problem is client-sided, but I don’t see any real drawbacks fixing this on your side.
Thanks.

Fixed by PR #176


Add ability to filter capcodes and source from main index

This will require the below to be added to the users config.json file as per the changes to default.json.
Haven't quite got a grip on how to handle that yet.
This partially fixes https://github.com/davidmckenzie/pagermon/issues/38 these are still visible in the api...
"HideCapcode": false,
"HideSource": false
Thanks to @damiankw for assisting with this

This :+1:


[Bug]: Unhandled exception thrown in ~cudnn_context()

What Operating System(s) are you seeing this problem on?
Windows
dlib version
14ba5572e7b1ce4c5eb5ca690484583a4bc4717f
Python version
N/A
Compiler
MSVC 14 (Visual Studio 16.11.34)
Expected Behavior
Programs that correctly train neural networks and reach the end of main() without error should terminate successfully.
Current Behavior
Either during the exit of main() or afterwards, an access violation occurs in ~cudnn_context(). The exact error is
Exception thrown at <address> (nvcuda64.dll) in test.exe:
0xC0000005: Access violation reading location <address>

and appears to be a problem when calling cudnnDestroy  here
https://github.com/davisking/dlib/blob/14ba5572e7b1ce4c5eb5ca690484583a4bc4717f/dlib/cuda/cudnn_dlibapi.cpp#L91
Steps to Reproduce
Building and running dnn_introduction_ex.cpp will demonstrate the error. I believe running any dnn training example will.
Here is a minimal example though
#include <dlib/dnn.h>

using namespace dlib;

int main(void)
{
    using net_type = loss_binary_log<fc<1, input<matrix<float>>>>;

    std::vector<matrix<float>> mini_batch_samples = { {1.0f, 1.0f, 1.0f, 1.0f} };
    std::vector<float> mini_batch_labels = { 1.0f };

    net_type net;
    dnn_trainer<net_type> trainer(net);
    trainer.train_one_step(mini_batch_samples, mini_batch_labels);

    return 0;
}

Anything else?
I tested this with different versions of CUDA and cuDNN. This error occurs with CUDA/cuDNN versions 12.4/9.0 and also 11.8/8.9.7. I have two GTX 1080 Ti GPUs with driver version 551.86.
It seems similar to https://github.com/davisking/dlib/issues/2186.
Additionally, I have written some small programs with the cuDNN api directly (no dlib), and I have not encountered the error when calling cudnnDestroy is those cases.
This occurs both with Debug and Release builds.

Is it somehow related to using the trainer and not just exercising the network object in a way that calls cudnn?  This really ought to work fine, unless there is some new change in visual studio+cudnn that makes thread_local not work here.

Exercising the network does not reproduce the error, but using the trainer does.
When I created this issue, I did not realize that training is performed in a separate thread even with a single device. So, I tested the effect of the thread_local specifier, and found that this code
#include <iostream>
#include <thread>
#include <cudnn.h>

class cudnn_context
{
public:
    cudnn_context(const cudnn_context&) = delete;
    cudnn_context& operator=(const cudnn_context&) = delete;

    cudnn_context() : handle(nullptr) {}
    ~cudnn_context()
    {
        if (handle)
            cudnnDestroy(handle);
    }

    cudnnHandle_t get_handle()
    {
        if (!handle)
            cudnnCreate(&handle);
        return handle;
    }

private:
    cudnnHandle_t handle;
};

static cudnnHandle_t context()
{
    thread_local cudnn_context c;
    return c.get_handle();
}

void fn()
{
    std::cout << context() << "\n";
    return;
}

int main(void)
{
    std::thread t(fn);
    t.join();
    return 0;
}

prints the handle address value and then throws the access violation.
However, this example
#include <iostream>
#include <thread>
#include <cudnn.h>

void fn()
{
    cudnnHandle_t handle;
    cudnnCreate(&handle);
    std::cout << handle << "\n";
    cudnnDestroy(handle);
    return;
}

int main(void)
{
    std::thread t(fn);
    t.join();
    return 0;
}

prints the address and terminates successfully.
I would think that thread_local basically does the same thing as the second example, but maybe there is something else happening to the CUDA context.
Thanks for the help!

Yeah there seems to be a bug in cuda or that version of visual studio maybe.  Try this kind of thing
The little crashing program you have there but replace thread_local with static.  Does that crash?  Maybe it's purely about cudnnDestroy() being called after main() has terminated.  Which really ought to be fine.

Changing the specifier to static eliminates the exception. Also, calling cudnnDestroy from main() after the thread is finished works successfully.

Maybe it's purely about cudnnDestroy() being called after main() has terminated

That does not appear to be the issue though. I verified that the thread_local destructor is called before all the global/static object destructors. Additionally, putting the code in a scope block like this
int main(void)
{
    {
        std::thread t(fn);
        t.join();
    } // exception thrown
    return 0; // program never arrives here
}

still results in an exception even though main() has not returned.
I did a little more investigating and it appears to be an issue elsewhere in PyTorch https://github.com/pytorch/pytorch/issues/17658, tvm https://github.com/apache/tvm/pull/8267, and Caffe2 https://github.com/pytorch/pytorch/pull/95382. It is mostly (but not exclusively) happening on Windows, and appears to be a problem with cuDNN/CUDA.
Anyways, I made some local changes to the cudnn_context to implement a static handle pool that reuses handles when a thread is done and cleanups on program exit (PyTorch does something similar). This works because cuDNN handles are tied to a device context, not a thread. The only requirement is that multiple threads do not use a handle simultaneously. It fixed the problem for me, and has a added benefit of not having to re-initialize a handle for every new thread.
Let me know if you are interested in this kind of change. I am happy to submit a PR.

Yeah, a PR that works around the problem would be cool.  It's got to work in all contexts though. Not sure if you were saying it wasn't totally thread safe.
So what's the exact bug?  Just that you can't call cudnnDestroy in code executed by thread_local?  It might also be that you can't call cudnnCreate from one thread and then cudnnDestroy from a different thread.  I've no idea how visual studio executes thread_local. Maybe it doesn't execute the thread_local destruction on the same thread that created the thread local object?  I'm just suspicious that the problem isn't purely about thread_local, since that seems like something surprising to have cudnn somehow mess up.  I wouldn't be surprised if there was a "oh but cudnnDestroy can't run on a different thread from the one that created the handle bug" though.

Well, in any case, post your change and we can look at it :)

Not sure if you were saying it wasn't totally thread safe.

I believe a reusable handle pool should be thread safe, at least based on this statement from the cudnn documentation on thread safety

The cuDNN library is thread-safe. Its functions can be called from multiple host threads, so long as the threads do not share the same cuDNN handle simultaneously.

which supports the idea that changing ownership is acceptable.

So what's the exact bug?

Honestly, I don't know. It makes no sense that the thread_local destructor could be an issue. I verified it works properly with several MSVC versions.
However, after testing several different drivers, CUDA/cuDNN versions, and MSVC toolkits I finally found that the 537.83 NVIDIA driver works. So, it seems this is somehow a driver bug that is present in possibly all of the 54*.** and 55*.** Windows drivers. I filed a bug report with NVIDIA.
Thanks for your help @davisking!

Sweet, yeah, glad it's working now and we don't have to hack around it :D


xlsm support?

Hello.
I am unable to open .xlsm files through Onlyoffice. Is this on purpose?
As per the specifications here : "Currently the following document formats can be edited with this app: DOCX, XLSX, PPTX, TXT, CSV" https://api.onlyoffice.com/editors/owncloud. However, here in the config api, we see that Onlyoffice can open .xlsm https://api.onlyoffice.com/editors/config/ . Is this limitation on purpose or did someone just forget to include .xlsm files in the config file somewhere on the application server or the onlyoffice nextcloud app?
Thanks!

Just noticed this : it does not open .xls files either...

add macro-enabled and template formats
fa1907abcf386e652e673609af506e81a7baa26f

I just overwrote the lib/appconfig.php file with the new version and it still downloads the .xls files instead of opening them in the onlyoffice suite.

Just pulled the whole ONLYOFFICE/onlyoffice-owncloud app from git and installed it on my Nextcloud, same issue...

Hello @stanelie,
Please note that the file could not be opened from the action menu, it is being sent for conversion.

Hello.
I am not sure I understand. When I clic on a .xls file within Nextcloud, I expect it to be opened with the OnlyOffice online editor, as is the case with .xlsm files, or with Collabora, for that matter. However, this is not the case, I get the dialog from my browser prompting me to download the file instead.

Hello @stanelie,
Please open the administrator section of NC and choose ONLYOFFICE settings, there will be settings for default opening where you can choose formats. See the attached screenshot.

As for the formats not stated in the list they need to be conversed at first, that is why it is necessary to click on the action menu and choose "Open in ONLYOFFICE" option. The file will be converted and copied in the same folder.

Hello @stanelie,
Please open a new issue or describe the problem in similar issues related to the conversion.
Please do the following: click on the Save button in ONLYOFFICE address settings and attach the logs of Nextcloud after that.

Hello.
Does this limitation of needing to convert .xls files to .xlsm files first before being able to edit them in OnlyOffice still exist?
Thanks!

@stanelie
Hello, yes, .xls files are opened in viewer only. In order to be able to edit them, you need to convert them to .xlsx.

Are there plans to have this fixed?
To me, this is still a bug, since it is not the expected behavior of an Office suite. I expect an Office suite to be able to open "standard" (yeah, I know, locked into microsoft formats) office files.
With about 50 000 .xls files on our file server, converting them all is not something I look forward to.

@stanelie
This is not a bug. ONLYOFFICE editors work only with Open XML formats (docx/xlsx/pptx), so .xls files have to be converted.

Alguna alternativa para formatos .xlsm en el trabajo usan los archivos con macros y no puedo modificarlos.


Forked Docusaurus project does not install dependencies after creating GitHub repository

Describe the bug
I forked the StackBlitz Docusaurus template and it worked perfectly fine. I then connected my GitHub account, gave the GitHub app permissions needed, and then created a new repository for my newly forked project. After being redirected to the "new IDE experience", my dependencies are not getting installed.
Link to the blitz that caused the error
https://stackblitz.com/~/github.com/abdallahshaban557/testing-stackblitz
Steps to reproduce

Fork Docusaurus template.
Provide GitHub app permissions to your GH account.
Create GH repository
Try to access the project
Dependencies will not be installed.

Expected behavior
Dependencies should be installed normally for projects that have a GH repo connected to them.
Parity with Local

[X] I have run the project in my local machine and I could not reproduce the issue.

Screenshots

Platform
Version = 1.84.0
Hash = a33a469ce26ab066f66abb377bcc8f68bdae5fc8
WebContainer = a9c8a5a3

Browser name  = Chrome
Full version  = <IP_ADDRESS>
Major version = 121
navigator.appName = Netscape
navigator.userAgent = Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36
performance.memory = {
  "totalJSHeapSize": 158209553,
  "usedJSHeapSize": 140761425,
  "jsHeapSizeLimit":<PHONE_NUMBER>
}

Additional context
No response

Hi! Try to remove package-lock.json from your new repo...


Getting Internal Server Error 500 When Creating some PRs

Description
When I try to merge certain branches to our dev branch in a private repo, I get the 500 error with the following logs:
2023/03/26 20:47:25 router: completed POST /Dikurium/consulting-tool-backend/compare/dev...user-story/732-submit-goal-categories for <IP_ADDRESS>:46804, 500 Internal Server Error in 81.4ms @ repo/pull.go:1143(repo.CompareAndPullRequestPost)

Gitea Version
1.18.5
Can you reproduce the bug on the Gitea demo site?
No
Log Gist
No response
Screenshots
No response
Git Version
No response
Operating System
Linux / Kubernetes (K3s)
How are you running Gitea?
Gitea is running as a StatefulSet in K3s. The deployment is based on the provided Helm chart, which I rewrote to a Terraform module.
Database
PostgreSQL

Are there any more detail logs?

No more feedback, I think this issue can be closed, feel free to reopen if there are more clues.
Gitea has new stale releases, please try the latest one.


Project Board Automation(s)

Description
In order to make working with the new Project Board a bit easier / keep it up-to-date in an automatic fashion, we should have some automation in place which:

automatically assigns new issues to the Project Board
adds PRs which are good to be reviewed by others to the Project Board with status "in Review"

adds PRs which are good to be reviewed by others to the Project Board with status "in Review"

what does it mean "good to be reviewed" ? How can github determine what is a "good to be reviewed" state?

"Good to be reviewed" is to be determined by the developer creating the PR. He/she is the one moving a PR from a possible "draft" to a "can be reviewed" phase. So possibly, as soon as someone is getting assigned to do a PR Review, the move to the "in review" column could happen ?

I'm going to do a couple things:

update some workflows in the project board ( I think I have admin access )
create some automated workflows settings status periodically on PRs that have no action on them for a little while

when done, we need to add this to template repo.

Feedback:

rename review to status
look into creating timeline events instead of comments on the pr
update readme
create issue from pull request
add possibility to define multiple project numbers for an issue when it's created

These issues have been addressed. Next up is creating some actions for the ocm-controller project.


add pacmod3 depend for clang_tidy_pr

Signed-off-by: kosuke murakami<EMAIL_ADDRESS>Description
fix clang_tidy_pr
https://github.com/tier4/AutowareArchitectureProposal.iv/runs/4259999021?check_suite_focus=true

Review Procedure

Remarks

Pre-Review Checklist for the PR Author
PR Author should check the checkboxes below when creating the PR.

[ ] Read pull request guidelines
[ ] Code follows coding guidelines
[ ] Assign PR to reviewer

Checklist for the PR Reviewer
Reviewers should check the checkboxes below before approval.

[ ] Commits are properly organized and messages are according to the guideline
[ ] Code follows coding guidelines
[ ] (Optional) Unit tests have been written for new behavior
[ ] PR title describes the changes

Post-Review Checklist for the PR Author
PR Author should check the checkboxes below before merging.

[ ] All open points are addressed and tracked via issues or tickets
[ ] Write release notes

CI Checks

Build and test for PR / build-and-test-pr: Required to pass before the merge.
Build and test for PR / clang-tidy-pr: NOT required to pass before the merge. It is up to the reviewer(s). Found false positives? See the guidelines.
Check spelling: NOT required to pass before the merge. It is up to the reviewer(s). See here if you want to add some words to the spell check dictionary.

FYI: @KeisukeShima Clang-Tidy in metrics-reporter may also need these changes (in the future?).

I'd like to put these into https://github.com/tier4/autoware-github-actions (like setup-autoware-depends.yaml) instead of setting in each workflow/job.
@KeisukeShima @kmiya How do you feel?


Support extraEnv value setting for all flux controllers

What this PR does / why we need it:
Add extraEnv helm value setting for all flux controllers. source-controller already has this. this change exposes the same across all controllers consistently.
In case of using kubeconfig reference in HelmReleases or Kustomizations to reconcile on remote clusters, and in such setups where Kubernetes API endpoints of those remote clusters could require going through an egress proxy for reachability.
Checklist

[x] DCO signed
[x] Chart Version bumped
[x] helm-docs are updated
[x] Helm chart is tested
[x] Update artifacthub.io/changes in Chart.yaml
[x] Run make reviewable

@stefanprodan @dwerder  PTAL 🙏

@stefanprodan  if you don't mind closing above linked PRs and issues, which are all addressed by this PR. I missed linking the issue here. Thanks.

While I appreciate this finally being merged upstream, I will admit it's a little irritating that my PR from 7 months ago which is near identical never went anywhere.


Optional HeaderParams do not work as expected

If I have:
interface FooService {
  void doFoo(@HeaderParam("On-Behalf-Of") @Safe Optional<String> onBehalfOf);
}

And I call it as such:
myFooService.doFoo(Optional.empty());

Then in my witchcraft my FooResource get's onBehalfOff as Optional.of("") instead of Optional.empty().

Want to take a shot at this, Jared?

This seems like a very similar issue to https://github.com/palantir/conjure-java-runtime/issues/384


Missing Operations

When I look in an exported file, I see Operations that are not listed as audited events, specifically for the Exchange workload
New-AuthenticationPolicy
Set-OrganizationConfig
Set-AuthenticationPolicy
Set-HostedContentFilterPolicy
New-ReportSubmissionPolicy
are in my file but are not described in the documentation. Please update the documentation to ensure that it accurately describes all of the events that are audited.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3494b782-92ad-e365-3398-bd2d760d10c6
Version Independent ID: fa4bca98-3975-23dd-dfac-f23049c71d01
Content: Search the audit log in the Security & Compliance Center - Microsoft 365 Compliance
Content Source: microsoft-365/compliance/search-the-audit-log-in-security-and-compliance.md
Service: o365-seccomp
GitHub Login: @markjjo
Microsoft Alias: markjjo

@DeanGross It's not really feasible to document all Exchange admin audit operations in the context of auditing. This topic addresses this by citing the Exchange admin audit log and that these operations correspond to cmdlets in EXO, which are documented in cmdlet reference topics. Please see: https://docs.microsoft.com/en-us/microsoft-365/compliance/search-the-audit-log-in-security-and-compliance?view=o365-worldwide#exchange-admin-audit-log

Thanks, i totally missed that explanation


Simple fix for duplicate results with multiple flavours with same SPARQL endpoint

Just putting this up as a suggestion to solve this problem in the short term, as this issue affects multiple IDN tools, including Prez UI in general.
At lines 42 & 108 of routers/sparql.py:
endpoints = {p: settings.sparql_creds[p]["endpoint"] for p in settings.enabled_prezs}
unique_endpoints = {}
for p, endpoint in endpoints.items():
    if endpoint not in unique_endpoints.values():
        unique_endpoints[p] = endpoint
results = await asyncio.gather(
    *[sparql_query(query, p) for p in unique_endpoints.keys()]
)

I know this solution doesn't accurately show which subsystem returned the results, but the results would come from the same endpoint anyway, and fixing the duplicate results I think is much more important than correctly returning the endpoint info.
Happy to discuss further and make a PR if preferred.

Test comment


Update _getExperiment.mdx

🚀 Deployed on https://656f79f9eb8faf2aeeef87f6--cozy-fox-0defec.netlify.app


switch to diff format from REST API

resolves #10 (again)
partially resolves #4 (in a compromising way)
Doc updates

cli_arg_parser object moved to its own module.
slightly changed how the CLI options are documented (using std:option built into Sphinx).
added docs for the new git.py module
tweaked some existing docstrings as some info was missing from recent updates

API changes
All diff parsing happens in get_list_of_changed_files() which uses functions from the new git.py module. This means that filter_out_non_src_files() doesn't need to check for the status or patch fields that were used in the JSON format of REST API calls.
If executed from a local developer's environment, get_list_of_changed_files() will use the output provided by git status -v (with the summary info prelude stripped away). This should allow using options like lines-changed-only and files-changed-only without needing a CI environment.
unit test updates
It was getting pretty messy to test multiple commits from different repos that were defined in a hacky event_files.json file. We now have separate test resources to individually test multiple repos for multiple commits.
Furthermore, all tested repos' files are downloaded to the temporary test folder that is managed by pytest (as opposed to gitignoring the downloaded files in our tests/capture_tools_output folder). To retain test performance, I employed a caching tactic to avoid re-downloading the same files for similar tests.

Codecov Report

Merging #26 (a0c38ec) into main (53e4755) will decrease coverage by 1.45%.
The diff coverage is 87.61%.

@@            Coverage Diff             @@
##             main      #26      +/-   ##
==========================================
- Coverage   81.97%   80.51%   -1.46%
==========================================
  Files           6        8       +2
  Lines         649      693      +44
==========================================
+ Hits          532      558      +26
- Misses        117      135      +18

Impacted Files
Coverage Δ

cpp_linter/thread_comments.py
62.03% <ø> (-7.41%)
:arrow_down:

cpp_linter/__init__.py
92.30% <80.00%> (+0.24%)
:arrow_up:

cpp_linter/git.py
84.37% <84.37%> (ø)

cpp_linter/run.py
76.05% <85.00%> (-3.11%)
:arrow_down:

cpp_linter/cli.py
100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

I was giving you time to review because it is a kinda big change, but it is well covered in the unit tests. Also, the news coming out of China sounds pretty serious, so take your time and try to stay safe.

"xfail" is short for "expected failure". So, yes "xfail" results are fine.

refactor run.py for easy reading in the future.

Yeah, I think some of the functions that use the REST API could go into its own rest.py module as well. I have no problem renaming run.py as main.py. Remember back when it was just a shell script named runchecks.sh?

reported with sonar, it seems to complain the code may not easy to understand.

It complains about a nest inline condition in cli.py. I chose to ignore it since it is used in a lamdba statement.

  0 Code Smells

Fixed it.

This CI hit the REST API rate limit for unauthenticated requests. https://github.com/cpp-linter/cpp-linter/actions/runs/3663491831/jobs/6193307913#step:8:251
This is because the test_get_changed_files() test actually requests data from the REST API.

yeah, I can refactor that logic. I pushed rather hastily...

yeah I think another CI run on this branch will fail. I need to monkeypatch the test.

Ok. I fixed the code smell issue by factoring out the logic to a private function. I also adjusted some unit tests that were directly using the REST API, so we shouldn't get those CI failures when the API rate limit is hit.


[DATA-605] Orbslam homebrew formula

A couple questions:

After running brew install, I still had to manually link everything from /opt/homebrew/Cellar/orb-grpc-server/HEAD-af2dec0/lib/ into /usr/local/lib/. Otherwise, I got a runtime error:
dyld[13090]: Library not loaded: /tmp/orb-grpc-server-20221026-10229-f8sjnr/slam-libraries/viam-orb-slam3/ORB_SLAM3/Thirdparty/DBoW2/lib/libDBoW2.dylib Referenced from: /opt/homebrew/Cellar/orb-grpc-server/HEAD-af2dec0/bin/orb_grpc_server Reason: tried: '/tmp/orb-grpc-server-20221026-10229-f8sjnr/slam-libraries/viam-orb-slam3/ORB_SLAM3/Thirdparty/DBoW2/lib/libDBoW2.dylib' (no such file), '/usr/local/lib/libDBoW2.dylib' (no such file), '/usr/lib/libDBoW2.dylib' (no such file)
I'm not sure if this is a problem with HEAD installs, or a problem with the brew formula that I wrote. Any advice?
EDIT: Now that I recall, I had the same problem when I installed pangolin using the brew package.
I tested on my M1. Do we have access to an intel mac to test on?
UPDATE: Kat and I tested on her intel mac, and the behavior was mostly the same as on my M1 mac. Yay! The only difference was that on my mac, the binary was installed to /opt/homebrew/bin, so which orb_grpc_server returned something. On Kat's mac, the binary wasn't linked to any location that we found, so which orb_grpc_server returned nothing. We tested by running the binary in the Cellar directly.

Sorry, I should have said, this requires the changes from https://github.com/viamrobotics/slam/pull/95. Can you try change branch to main (since the PR is now merged, and I deleted the branch) and installing with --HEAD?

Now that https://github.com/viamrobotics/slam/pull/95 is merged, I changed branch to main. For now, this will only work with --HEAD. Once I tag a version that includes https://github.com/viamrobotics/slam/pull/95, I'll update url to point to that tagged version.

I've been looking into the issue you call out in top comment/body of this PR, about the libraries not being in the correct locations.
During the build this comes up:
  Policy CMP0042 is not set: MACOSX_RPATH is enabled by default.  Run "cmake
  --help-policy CMP0042" for policy details.  Use the cmake_policy command to
  set the policy and suppress this warning.

  MACOSX_RPATH is not specified for the following targets:

   ORB_SLAM3

The RPATH stuff is key here I think. The "proper" way to do this with cmake is to actually setup install targets and handle the "install" from within cmake itself, with rpath stuff enabled. Cmake does a lot of "magic" during the install step to swap paths around in built binaries. Then, in brew, call cmake directly with *std_cmake_args, which will handle setting up all the paths and details for the build in brew, and use cmake's install rather than directly picking out the libraries and main executable. There's an entire article on the cmake wiki about dealing with RPATH issues if you wanted to go that route. https://gitlab.kitware.com/cmake/community/-/wikis/doc/cmake/RPATH-handling
That's a lot of work though, even though it'd be the "correct" way of dealing with things. With that in mind, I found a simpler (but uglier) approach that still works. Basically, just swaping out the broken build-time library paths with their post-install paths. OSX has a tool for this called "install_name_tool." Adding the following to the formula (after the build but before the install steps) appears to fix it for me.
      if OS.mac?
        system "install_name_tool", "-change", buildpath.to_s.delete_prefix("/private") + "/slam-libraries/viam-orb-slam3/ORB_SLAM3/Thirdparty/DBoW2/lib/libDBoW2.dylib", "#{lib}/libDBoW2.dylib", "viam-orb-slam3/ORB_SLAM3/lib/libORB_SLAM3.dylib"
        system "install_name_tool", "-change", buildpath.to_s.delete_prefix("/private") + "/slam-libraries/viam-orb-slam3/ORB_SLAM3/Thirdparty/g2o/lib/libg2o.dylib", "#{lib}/libg2o.dylib", "viam-orb-slam3/ORB_SLAM3/lib/libORB_SLAM3.dylib"
        system "install_name_tool", "-change", buildpath.to_s.delete_prefix("/private") + "/slam-libraries/viam-orb-slam3/ORB_SLAM3/Thirdparty/DBoW2/lib/libDBoW2.dylib", "#{lib}/libDBoW2.dylib", "viam-orb-slam3/bin/orb_grpc_server"
        system "install_name_tool", "-change", buildpath.to_s.delete_prefix("/private") + "/slam-libraries/viam-orb-slam3/ORB_SLAM3/Thirdparty/g2o/lib/libg2o.dylib", "#{lib}/libg2o.dylib", "viam-orb-slam3/bin/orb_grpc_server"
        system "install_name_tool", "-change", buildpath.to_s.delete_prefix("/private") + "/slam-libraries/viam-orb-slam3/ORB_SLAM3/lib/libORB_SLAM3.dylib", "#{lib}/libORB_SLAM3.dylib", "viam-orb-slam3/bin/orb_grpc_server"
      end

I tested on linux and an M1 mac. Would be good to test on an intel mac as well, make sure it works there too.

Thank you, @Otterverse, both for the longer explanation and the easier solution. I had started reading about RPATH when trying to fix openssl for the macOS CI job, and that does seem like it would take me awhile to get right, so I'm using the solution you gave.
I also updated the tag (and switched the download to a git clone, so that submodules are included), so it's now possible to test installing without --HEAD.
@kkufieta, would you be willing to test the formula again with these changes? You should now be able to install it with brew install --build-from-source --verbose --debug orb-grpc-server. I'm interested in whether we see the Library not loaded error when we try to run orb_grpc_server.


please add links for play

For Commercial, it is https://apps.powerapps.com/
For GCC: https://apps.gov.powerapps.us/
Not certain for GCC High, but I think its https://apps.high.gov.powerapps.us/

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ea524f2a-1e2a-6ade-8e2b-bc61173107c5
Version Independent ID: 5990c060-8907-62b4-a8b3-f7ed8339a70e
Content: Microsoft Power Apps US Government - Power Platform Admin center
Content Source: power-platform/admin/powerapps-us-government.md
Service: power-platform
GitHub Login: @jimholtz
Microsoft Alias: jimholtz

Duplicate of 213.


ImportError: cannot import name 'AsyncCallbackManager' from 'langchain.callbacks.base' (/usr/local/lib/python3.9/dist-

Traceback (most recent call last):
File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.9/dist-packages/uvicorn/_subprocess.py", line 76, in subprocess_started
target(sockets=sockets)
File "/usr/local/lib/python3.9/dist-packages/uvicorn/server.py", line 61, in run
return asyncio.run(self.serve(sockets=sockets))
File "/usr/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/usr/local/lib/python3.9/dist-packages/uvicorn/server.py", line 68, in serve
config.load()
File "/usr/local/lib/python3.9/dist-packages/uvicorn/config.py", line 473, in load
self.loaded_app = import_from_string(self.app)
File "/usr/local/lib/python3.9/dist-packages/uvicorn/importer.py", line 24, in import_from_string
raise exc from None
File "/usr/local/lib/python3.9/dist-packages/uvicorn/importer.py", line 21, in import_from_string
module = importlib.import_module(module_str)
File "/usr/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 790, in exec_module
File "", line 228, in _call_with_frames_removed
File "/home/chaseryang/langchain/chat-langchain/main.py", line 12, in
from query_data import get_chain
File "/home/chaseryang/langchain/chat-langchain/query_data.py", line 2, in
from langchain.callbacks.base import AsyncCallbackManager
ImportError: cannot import name 'AsyncCallbackManager' from 'langchain.callbacks.base' (/usr/local/lib/python3.9/dist-packages/langchain/callbacks/base.py)

use from langchain.callbacks.manager import AsyncCallbackManager

@yudexiaozhou Works!

Is there an open PR for this issue?

I opened a PR to solve this issue #91

use from langchain.callbacks.manager import AsyncCallbackManager

it works


displays square instead of thumbs up

Challenge Waypoint: Add Font Awesome Icons to all of our Buttons has an issue.
User Agent is: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36.
Please describe how to reproduce this issue, and include links to screenshots if possible.
My code:
<link href="http://fonts.googleapis.com/css?family=Lobster" rel="stylesheet" type="text/css">

CatPhotoApp

      <button class="btn btn-block btn-primary"> Like</button>

      <button class="btn btn-block btn-info">Info</button>

      <button class="btn btn-block btn-danger">Delete</button>

Things cats love:

cat nip

laser pointers

lasagna

Top 3 things cats hate:

flea treatment

thunder

other cats

  <form action="/submit-cat-photo">
    <input type="radio" name="indoor-outdoor"> Indoor
    <input type="radio" name="indoor-outdoor"> Outdoor
    <input type="checkbox" name="personality"> Loving
    <input type="checkbox" name="personality"> Lazy
    <input type="checkbox" name="personality"> Crazy
    <input type="text" placeholder="cat photo URL" required>
    <button type="submit">Submit</button>
  </form>

http://i.imgur.com/k9RZOd5.png

I can not repro this issue. If this is happening for your it means the font-awesome file is not loading correctly on your system. I would try disabling browser extensions and clearing your cache in an effort to troubleshoot this. Also, check out your console and see if you are getting any errors on page load or at the time your solution runs.

Hello,
The classes "fa" and "fa-thumps-up" are not loading properly.
has anybody faced the same issue?

Like

I ended my i class after Like and it worked.  Just a beginner also.
Like

very strange but I am having the exact issue now. Going to try the clear the cache, and disable extensions

Like
I think this should work.

//Like

##Like


Add decoding attribute

Arf, not pushed with the right user 😅

Thanks! That would be a good addition.
The file you edited is generated – see the comment on top – please contribute this upstream to Scala DOM Types.
See https://github.com/raquo/scala-dom-types?tab=readme-ov-file#contributing  This attribute should go into the "reflected" trait, and I think the docs should link to https://developer.mozilla.org/en-US/docs/Web/HTML/Element/img#decoding as the more intuitive starting point.


WWW-Authenticate header is not parsed properly

Parser can not parse a header with merely comma separators rather than comma+space and parameters with empty values within double quotes.

Digest realm="Please log in with a valid username",nonce="752a62306daf32b401a41004555c7663",opaque="",stale=FALSE,algorithm=MD5

No luck for the example :(

Hello, the library has evolved a lot since may, the string you posted is now fully supported and has also been added to the test suite.


⚠ Bofrost.be has degraded performance

In 66333b7, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 2331 ms

Resolved: Bofrost.be performance has improved in 871ff09.


fix: synchronize session on all deletes

Note that this does have some performance implications [1]:

The 'fetch' strategy results in an additional SELECT statement emitted and will significantly reduce performance.

But some overhead in performance when running clear and clear_all() seems worth it if it eliminates the sqlalchemy error.
See #213.

Does this fix the #213 ?

Yes, it does for me.

👍


Fix delivering response promises in actor state destructors

@lava and I went over the changes together in a call to address the review feedback.


Add e2e test for Multi-cluster Service Endpoints changes

Add a new e2e test to validate Endpoint change event to make sure event filter works as expected.
This fix needs to be back-ported to v1.8 with a minor change to remove GenerationChangedPredicate
in ServiceExport controller.
Signed-off-by: Lan Luo<EMAIL_ADDRESS>

/test-multicluster-e2e

The Endpoint doesn't have Generation field which means all Endpoints change will be ignored after controller start.

So, it happens only after controller restart? If controller does not restart, it can still receive the update if Endpoints changes?

The Endpoint doesn't have Generation field which means all Endpoints change will be ignored after controller start.

So, it happens only after controller restart? If controller does not restart, it can still receive the update if Endpoints changes?

This bug happens after controller restart. the Endpoint event can be processed properly when controller start up, but after that, no change event can be received.

/test-multicluster-e2e

/test-multicluster-e2e

/test-multicluster-e2e

Hi @jianjuns  Could you help to review this issue again? thanks. I added a new e2e to make sure Endpoint event can be handled correctly. The mc e2e is expected to fail due to PR #4211 [Update ResourceExport's json tag as lowerCamelCase] updated the json tag. I submitted a PR #4232 to fix this issue, but not included the yaml file change in this PR considering this need to be back-ported to v1.8.

I think let us merge #4232 first to pass the tests.

/test-multicluster-e2e

@jianjuns  I have re-based to latest codes, the mc e2e is passed, could you help take a look again? thanks.

@jianjuns  the change is also included in PR #4198 and it's already merged, should I just rename the title of this PR?

Yes, please rename and update the description.
A reminder for the earlier comment too, in case you missed it:

Could we improve e2e test to cover Endpoints changes? In general, I feel we got quite some bugs in Multi-cluster. Could we define a strategy to prevent such missed corner cases - with UT, e2e, or even manual tests for new features/changes?

yes, I noticed this comment, beside adding new e2e test in this PR, I will make sure there will be more tests for corner cases in the future.

/test-multicluster-e2e

/skip-all
It is a multi-cluster tests change.

/skip-all
It is a multi-cluster test change.

/skip-all


Properties list in keyword_list.json could be improved

It looks like the list is being compiled manually right now while it's possible to do this in an automated fashion, e.g.
jq -n \
    --arg ver "$(cmake --version | head -n1 | awk '{print $NF}')" \
    --arg props "$(cmake --help-property-list | sort -fu | sed 's|<[^>]*>|.+|g')" \
    '{cmake_version: $ver, properties: $props | split("\n")}'

Since each element in properties is then a valid regex, we could combine them into a single long regex and use that for matching. This also makes it unnecessary to split the properties into multuple groups (properties_full_names, properties_starting_with, properties_ending_with), and improves matching accuracy for properties that don't belong to any of those groups (e.g. VS_DOTNET_REFERENCEPROP_.+_TAG_.+).

Thank you for pointing that out. :+1:


Fix Google Calendar link in support doc

typo in the target so the link is not clickable.

Thanks @lesteve!


Make PNG editor configurable

A way to configure which app to open PNG files in when the edit button is clicked. Because PNG files are so commonly used for so many other purposes, most operating systems correctly open a viewer, not an editor, for this file type. Changing this setting system-wide is possible, but then leads to many problems since most of the time a viewer is what you want.
This setting could be controlled by the user by adding a field in the settings window to set a path to an app for editing PNG files, rather than the system default.
Another option would be to follow the system's preference on a per-file basis - which at least macOS supports.

This is a good idea, I've just implemented it in #672 :-)


Improve DataFusion scalability as more cores are added

Is your feature request related to a problem or challenge?
I ran some benchmarks in constrained Docker containers and found that DataFusion is pretty close to DuckDB speed when running on a single core but does not scale as well as DuckDB when more cores are added.
At 16 cores, DuckDB was ~2.9x faster than DataFusion for this particular test.

Cores
DuckDB 0.7.1
DataFusion Python 21.0.0
DuckDB x Faster

1
315,630.9
318,037.8
1.0

2
155,034.5
197,257.3
1.3

4
75,110.0
111,243.1
1.5

8
37,584.9
73,142.3
1.9

16
18,880.3
55,071.8
2.9

Describe the solution you'd like
I would like DataFusion to scale as well as DuckDB as more cores are added.
Describe alternatives you've considered
No response
Additional context
Instructions for creating the Docker images can be found at https://github.com/sql-benchmarks/sqlbench-runners

Possibly related: https://github.com/apache/arrow-datafusion/issues/5995

Suggestion I posted in slack:
I wonder whether there might be some regressions here wrt scalability with the nr of cores - https://github.com/apache/arrow-datafusion/pull/4219 comes to mind which makes building the hashmap for the build-side done in a single thread for smaller tables
might be checked by changing datafusion.optimizer.hash_join_single_partition_threshold

For q1 tpch-h the metrics are as follows for the complete plan are as follows (only including ):
SortExec: [l_returnflag@0 ASC NULLS LAST,l_linestatus@1 ASC NULLS LAST], metrics=[output_rows=4, elapsed_compute=25.042µs, spill_count=0, spilled_bytes=0]
  CoalescePartitionsExec, metrics=[output_rows=4, elapsed_compute=3.167µs, spill_count=0, spilled_bytes=0, mem_used=0]
    ProjectionExec: expr=[l_returnflag@0 as l_returnflag, l_linestatus@1 as l_linestatus, SUM(lineitem.l_quantity)@2 as sum_qty, SUM(lineitem.l_extendedprice)@3 as sum_base_price, SUM(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount)@4 as sum_disc_price, SUM(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount * Int64(1) + lineitem.l_tax)@5 as sum_charge, AVG(lineitem.l_quantity)@6 as avg_qty, AVG(lineitem.l_extendedprice)@7 as avg_price, AVG(lineitem.l_discount)@8 as avg_disc, COUNT(UInt8(1))@9 as count_order], metrics=[output_rows=4, elapsed_compute=8.376µs, spill_count=0, spilled_bytes=0, mem_used=0]
      AggregateExec: mode=FinalPartitioned, gby=[l_returnflag@0 as l_returnflag, l_linestatus@1 as l_linestatus], aggr=[SUM(lineitem.l_quantity), SUM(lineitem.l_extendedprice), SUM(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount), SUM(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount * Int64(1) + lineitem.l_tax), AVG(lineitem.l_quantity), AVG(lineitem.l_extendedprice), AVG(lineitem.l_discount), COUNT(UInt8(1))], metrics=[output_rows=4, elapsed_compute=181.588µs, spill_count=0, spilled_bytes=0, mem_used=0]
        CoalesceBatchesExec: target_batch_size=8192, metrics=[output_rows=64, elapsed_compute=350.731µs, spill_count=0, spilled_bytes=0, mem_used=0]
          RepartitionExec: partitioning=Hash([Column { name: "l_returnflag", index: 0 }, Column { name: "l_linestatus", index: 1 }], 16), input_partitions=16, metrics=[repart_time=296.707µs, fetch_time=1.652179502s, send_time=110.333µs]
            AggregateExec: mode=Partial, gby=[l_returnflag@5 as l_returnflag, l_linestatus@6 as l_linestatus], aggr=[SUM(lineitem.l_quantity), SUM(lineitem.l_extendedprice), SUM(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount), SUM(lineitem.l_extendedprice * Int64(1) - lineitem.l_discount * Int64(1) + lineitem.l_tax), AVG(lineitem.l_quantity), AVG(lineitem.l_extendedprice), AVG(lineitem.l_discount), COUNT(UInt8(1))], metrics=[output_rows=64, elapsed_compute=670.144742ms, spill_count=0, spilled_bytes=0, mem_used=0]
              ProjectionExec: expr=[CAST(l_extendedprice@1 AS Decimal128(38, 4)) * CAST(Some(100),23,2 - CAST(l_discount@2 AS Decimal128(23, 2)) AS Decimal128(38, 4)) as CAST(lineitem.l_extendedprice AS Decimal128(38, 4)) * CAST(Decimal128(Some(100),23,2) - CAST(lineitem.l_discount AS Decimal128(23, 2)) AS Decimal128(38, 4))CAST(Decimal128(Some(100),23,2) - CAST(lineitem.l_discount AS Decimal128(23, 2)) AS Decimal128(38, 4))Decimal128(Some(100),23,2) - CAST(lineitem.l_discount AS Decimal128(23, 2))CAST(lineitem.l_discount AS Decimal128(23, 2))lineitem.l_discountDecimal128(Some(100),23,2)CAST(lineitem.l_extendedprice AS Decimal128(38, 4))lineitem.l_extendedprice, l_quantity@0 as l_quantity, l_extendedprice@1 as l_extendedprice, l_discount@2 as l_discount, l_tax@3 as l_tax, l_returnflag@4 as l_returnflag, l_linestatus@5 as l_linestatus], metrics=[output_rows=5916591, elapsed_compute=162.828798ms, spill_count=0, spilled_bytes=0, mem_used=0]
                CoalesceBatchesExec: target_batch_size=8192, metrics=[output_rows=5916591, elapsed_compute=29.656446ms, spill_count=0, spilled_bytes=0, mem_used=0]
                  FilterExec: l_shipdate@6 <= 10471, metrics=[output_rows=5916591, elapsed_compute=54.707373ms, spill_count=0, spilled_bytes=0, mem_used=0]
                    ParquetExec: limit=None, partitions={16 groups: [[...]]}, predicate=l_shipdate <= Date32("10471"), pruning_predicate=l_shipdate_min@0 <= 10471, projection=[l_quantity, l_extendedprice, l_discount, l_tax, l_returnflag, l_linestatus, l_shipdate], metrics=[output_rows=6001215, elapsed_compute=16ns, spill_count=0, spilled_bytes=0, mem_used=0, row_groups_pruned=0, predicate_evaluation_errors=0, bytes_scanned=44132502, num_predicate_creation_errors=0, pushdown_rows_filtered=0, page_index_rows_filtered=0, pushdown_eval_time=32ns, time_elapsed_scanning_until_data=493.033123ms, time_elapsed_scanning_total=1.643740915s, time_elapsed_processing=290.136543ms, time_elapsed_opening=6.494334ms, page_index_eval_time=32ns]

Relevant metrics:

ParquetExec time_elapsed_scanning_until_data=493.033123ms
ParquetExec time_elapsed_scanning_total=1.643740915s
ParquetExec time_elapsed_processing=290.136543ms
ParquetExec time_elapsed_opening=6.494334ms
FilterExec elapsed_compute=54.707373ms
CoalesceBatchesExec elapsed_compute=29.656446ms
ProjectionExec elapsed_compute=162.828798ms
RepartitionExec elapsed_compute=670.144742ms
AggregateExec elapsed_compute=670.144742ms

I ran the benchmarks with a simpler version of q1 with just the table scan and filter:
select
    *
from
	lineitem
where
	l_shipdate <= date '1998-12-01' - interval '113 days';

Here are the results:

Cores
DataFusion Python 21.0.0
DuckDB
DuckDB x Times Faster

1
14376
15000.9
0.96

2
7547.9
7915.7
0.95

4
4243.9
3850.7
1.10

8
2655.8
1869.3
1.42

16
2096.6
954.9
2.20

32
2252.2
531.7
4.24

Here are the results for just the table scan:
select
    *
from
	lineitem;

Cores
DataFusion Python 22.0.0
DuckDB
DuckDB x Times Faster

1
14501.3
14751
0.98

2
7562.5
8135.8
0.93

4
4508.7
3618.6
1.25

8
2363.2
1837.5
1.29

16
2135.4
888
2.40

32
2355
511.4
4.61

This screenshot shows the DataFusion run on the left and the DuckDB run on the right, both using 32 cores.

Here is the directory listing of my lineitem directory:
$ ls -l /mnt/bigdata/tpch/sf10-parquet/lineitem.parquet/
total 2134532
-rw-rw-r-- 1 andy andy 91100218 Feb  3 08:45 part-0.parquet
-rw-rw-r-- 1 andy andy 90998179 Feb  3 08:46 part-10.parquet
-rw-rw-r-- 1 andy andy 91113498 Feb  3 08:46 part-11.parquet
-rw-rw-r-- 1 andy andy 90979631 Feb  3 08:46 part-12.parquet
-rw-rw-r-- 1 andy andy 91042031 Feb  3 08:46 part-13.parquet
-rw-rw-r-- 1 andy andy 91013004 Feb  3 08:46 part-14.parquet
-rw-rw-r-- 1 andy andy 91102199 Feb  3 08:46 part-15.parquet
-rw-rw-r-- 1 andy andy 91113429 Feb  3 08:46 part-16.parquet
-rw-rw-r-- 1 andy andy 91057291 Feb  3 08:46 part-17.parquet
-rw-rw-r-- 1 andy andy 90968073 Feb  3 08:46 part-18.parquet
-rw-rw-r-- 1 andy andy 91085729 Feb  3 08:47 part-19.parquet
-rw-rw-r-- 1 andy andy 91065849 Feb  3 08:45 part-1.parquet
-rw-rw-r-- 1 andy andy 90977725 Feb  3 08:47 part-20.parquet
-rw-rw-r-- 1 andy andy 91092444 Feb  3 08:47 part-21.parquet
-rw-rw-r-- 1 andy andy 91114853 Feb  3 08:47 part-22.parquet
-rw-rw-r-- 1 andy andy 91029929 Feb  3 08:47 part-23.parquet
-rw-rw-r-- 1 andy andy 91139350 Feb  3 08:46 part-2.parquet
-rw-rw-r-- 1 andy andy 91167180 Feb  3 08:46 part-3.parquet
-rw-rw-r-- 1 andy andy 91083996 Feb  3 08:46 part-4.parquet
-rw-rw-r-- 1 andy andy 91024700 Feb  3 08:46 part-5.parquet
-rw-rw-r-- 1 andy andy 91022201 Feb  3 08:46 part-6.parquet
-rw-rw-r-- 1 andy andy 91101885 Feb  3 08:46 part-7.parquet
-rw-rw-r-- 1 andy andy 91198702 Feb  3 08:46 part-8.parquet
-rw-rw-r-- 1 andy andy 91120030 Feb  3 08:46 part-9.parquet

@Dandandan I added more benchmark results. The issues appear to be related to the ParquetExec.

I could also replicate the issue of non-perfect scaling with loading the tables in memory.
One thing I noticed is that the current round-robin RepartitionExec doesn't spread the bathes evenly over the number of output channels, which can already be seen in MemoryExec itself:
MemoryExec: partitions=32, partition_sizes=[32, 32, 32, 32, 32, 32, 32, 32, 26, 26, 26, 25, 25, 25, 25, 25, 25, 25, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16], metrics=[]
It has a bias for the starting partitions as the outputs always go first to those channels.

Also, I wonder if DistributionSender / DistributionReceiver didn't introduce any regressions here FYI @crepererum

Also, I wonder if DistributionSender / DistributionReceiver didn't introduce any regressions here FYI @crepererum

Definitely possible, even if it's only due to changes it triggers in tokio's scheduler. On the other hand, tokio's scheduler isn't really optimized for the DataFusion use case in the first place (as @tustvold probably pointed out before).

There is a small application written in Rust called odbc2parquet that reads data from databases and exports parquet files.
For testing on dummy data I think it is useful.
https://github.com/pacman82/odbc2parquet

Interestingly, the speedup seems bigger for fewer cores.
There have been quite a few improvement for Q1 between 21 and 23 for aggregates, that might explain most of the difference.

I believe https://github.com/apache/arrow-datafusion/pull/6929 from @YjyJeff will perhaps help with this

Latest results with DF 33

I believe that https://github.com/apache/arrow-datafusion/issues/6937 (basically that the fact we are doing much more work as the cores go up (as each group gets processed in each core) contributes to this problem

I produced the chart using scripts at https://github.com/sql-benchmarks/sqlbench-runners/blob/main/scripts

@sunchao @viirya cc ^^^^

The regression between 32 and 33 seems to be caused by my "improvement" to filter statistics, although we saw that others saw big improvements with 33 (such as https://x.com/mim_djo/status/1725510410567307702?s=46&t=8JleG5FR5SAVU1vl35CPLg) so maybe this is environment specific. I will continue to debug this in the next few days.
$ git bisect bad
6fe00ce2e30d2af2b64d3a97b877a96109c215f9 is the first bad commit
commit 6fe00ce2e30d2af2b64d3a97b877a96109c215f9
Author: Andy Grove<EMAIL_ADDRESS>Date:   Sun Nov 12 01:41:15 2023 -0700

    Fix join order for TPCH Q17 & Q18 by improving FilterExec statistics (#8126)

I wonder if this is just bad configuration?
I created a PR to update the DF configurations to use the defaults.
https://github.com/sql-benchmarks/sqlbench-runners/pull/31

@Dandandan was correct. It was the configs that were the issue. Here is a run of just versions 32 and 33 using default configs.


Add PWA Functionality

I would like to propose adding Progressive Web App (PWA) functionality to the htmgo framework. PWAs offer enhanced user experiences through features such as offline capabilities, improved performance, and the ability to install the app on devices.

Yes that's ok


Split execution fallback nodes selection step on Windows

Put it before installing Teku so it can be remembered easily if the user stop the installation and resume it later.

Completed with 6a0f674ffb34ecdeb1a0e0c5f2366da81004f225


Replace pylint, black and isort with ruff

Describe the enhancement you'd like
Ruff seems to be way faster (100x) and can autofix common issues: https://github.com/charliermarsh/ruff
Describe why this will benefit the LibrePhotos
A faster build pipeline will be helpful when reviewing code or implementing new features.

you can assign me this one i used ruff on my local seems to be working fine i only have to change the pre-commit.yaml file right ?

Yes, pre-commit and documentation need to be changing :)


Add support for py3.12

Looks like pycln needs an update first.

Looks like pycln needs an update first. I opened an issue there. hadialqattan/pycln#219

Hi, kindly, upgrade to pycln v2.3.0; it contains the fix.

Thanks @ml31415 and @hadialqattan !

@hadialqattan any ideas why this might be happening?
https://github.com/limx0/betfair_parser/actions/runs/6529781721/job/17727966092

I'm a bit surprised that it worked for you in 3.12. I suppose it's another issue with pycln and 3.12, see here: https://github.com/hadialqattan/pycln/issues/219#issuecomment-1763450260

I think its because my precommit install is a 3.10 env, but my local virtualenv is 3.12.

@limx0 how about relying on ruff for unused imports? It's currently checked twice anyways I suppose.
https://docs.astral.sh/ruff/rules/unused-import/


RPC retries (second PR)

Since the repo's layout has changed significantly since the last PR, I've opened a new PR.  I've attempted to address the comments from the last go-around - please have a look.

@calpeyser The cover environment is also failing because several lines do not have tests exercising them / some branches may not have all paths covered:

google/cloud/bigtable/retry.py:    60-61, 78-120, 123, 126, 175, 45->exit, 172->175
google/cloud/bigtable/row_data.py: 279, 277->279

Supersedes #3279.

Hi @dhermes, can you help take a look at this?

@lukesneeringer @garye - I've attempted to address the comments above.  Thanks for bearing with me!

@calpeyser -- looks like you have some conflicts with the base branch, please resolve them.
@lukesneeringer, @garye -- please take a look at the PR, it would be great to merge this functionality soon.
Thanks everyone!

LGTM from a bigtable perspective

Conflict resolved, tests are passing.  @lukesneeringer, what do you think?

Reading now

@lukesneeringer - can you merge it? Neither I nor @garye have write permissions.

:tada:

Thank you @calpeyser for your work on this and thanks to @garye and @lukesneeringer for the reviews!

This caused a system test breakage, I'm very tempted to roll it back and iterate in a PR / branch until the system tests are passing


Input truncation in collator

Hi! I had a few doubts regarding the truncation being done in the data collator.

For instance, in collators/llava_onevision.py:

https://github.com/zjysteven/lmms-finetune/blob/b3a68751d4631e7de5441f1c81cde982119991a4/collators/llava_onevision.py#L133-L141
What issues were faced when truncation was set to True?

As a follow-up, in the manual truncation:
https://github.com/zjysteven/lmms-finetune/blob/b3a68751d4631e7de5441f1c81cde982119991a4/collators/llava_onevision.py#L186-L189

When you directly truncate the tensor, isn't it possible that if there's an image present on the right side, it could get truncated unevenly?
For instance, let's say there's an image that extends from index 1000 to 2000, and you truncate it till index 1500. Wouldn't such an instance result in an error?
Thanks in advance!

What issues were faced when truncation was set to True?

We rely on return_assistant_tokens_mask of apply_chat_template to automatically slice assistant tokens, which is useful for constructing labels. As indicated in the comment, when truncation=True in apply_chat_template, the returned assistant tokens mask is somehow wrong when I tried earlier. You can try it yourself to confirm.

an image present on the right side, it could get truncated unevenly?

This is indeed possible, but 1) it wouldn't cause a technical error that would fail the training, and 2) this behavior (some image tokens getting truncated) simply cannot be avoided if it really happens.
Let me know if this makes sense.

Hi, that clears some of my doubts, but I have a related query.
Let's say my prompt is something like this: <image><image>\nWhat is the difference between the two images?, and the two images are a.jpg and b.jpg.
Now say I'm doing left truncation, and the image tokens corresponding to the first <image> tag get truncated. Could the second <image> tag accidentally get replaced by the tensor of a.jpg, instead of b.jpg?
https://github.com/zjysteven/lmms-finetune/blob/86895101a7f794c47cb3acc1061d0a148bc0b1df/collators/llava_onevision.py#L225-L230
I can see that the truncated prompt (input_ids) and the image tensors (vision_inputs) are returned separately. How are they processed while training?

This is a very specific (corner) case which I don't know for sure either. You can look at the source code of transformers to see how vision embeddings are processed during training https://github.com/huggingface/transformers/blob/f73f5e62e2383c1cb6975fca70082d6dc51ec6f2/src/transformers/models/llava_onevision/modeling_llava_onevision.py#L667-L693


Extract symbol and indices from varname string

@trappmartin suggested it would be nice to have the following in Turing IO utilities.
function extract_inds(inds_str)
    String[m.captures[1] for m in eachmatch(r"(\d+|Colon\(\))", inds_str)]
end
function unvarname(varname_str)
    sym = match(r"^([^\[]]*)\[", varname_str).captures[1]
    inds = [extract_inds(m.captures[1]) for m in eachmatch(r"(\[[^\]]+\])", varname_str)]
    return sym, inds
end

varname_str = "x[1,1][2,Colon()]"
unvarname(varname_str)

Related: https://github.com/TuringLang/Turing.jl/pull/742/files#diff-ae63d4eaa9cb3f08f2b99ebf9749af70R112


Add HotwordsFile and HotwordsScore fields to OnlineRecognizerConfig in C# API

Thank you for your first-time contribution!


[REQ] Chat Privacy / Unseen

Can you implement "unseen feature" ? (Block any "seen" and "is typing" footprint)
It would be great, thanks :)
Like these extensions for Chrome:
Chat Privacy - https://chrome.google.com/webstore/detail/facebook-chat-privacy/gfpgaanechfneiboempkfjghninbibjn
Facebook Unseen - https://chrome.google.com/webstore/detail/facebook-unseen/iicapmagmhahddefgokbabbgieiogjop

We will continue the discussion here https://github.com/Aluxian/Facebook-Messenger-Desktop/issues/359


Eclipse formatting rules should be provided for consistent source code formatting

Eclipse formatting rules should be provided for consistent source code formatting
Feature request
Expected Behavior - What where you expecting to happen?
There should be a file formatting_rules.xml (or similar) that can be imported into Eclipse.
It should contain formatting rules that are mandatory for the project.
Current Behavior - What happens?
Since default formatting in Eclipse is different from the current source formatting, if a reformatting of source code is done in Eclipse the resulting formatting is different. That leads to confusing diffs that hide the real changes.
Possible Solution
Provide a settings file with mandatory formatting rules.
Context
Notes and further information

Should we use the Google Style Guide (https://google.github.io/styleguide/javaguide.html) on this project? They provide the eclipse formatting rule (and also for other IDEs), which can be imported easily (https://github.com/google/styleguide/blob/gh-pages/eclipse-java-google-style.xml).
Also, we should consider some maven plugin to check the styleguide on Travis.

We should force the usage of a standardized formater. We need to have a look on impacts and changes to the default (Eclipse) formatting.


copy of shop-product element gives error when trying to print button below show-item

Sorta like this problem
In a derivation from pwa-starter-kit, I am getting the following error from trying to open the bean-counter nutrients of interest page :
VM4530 async.js:44 Uncaught TypeError: Cannot read property 'value' of null
at new Template (VM4510 lit-html.js:292)
at TemplateInstance.result [as _getTemplate] (VM4489 shady-render.js:43)
at NodePart._setTemplateResult (VM4510 lit-html.js:545)
at NodePart.setValue (VM4510 lit-html.js:498)
at NodePart._setIterable (VM4510 lit-html.js:603)
at NodePart.setValue (VM4510 lit-html.js:500)
at TemplateInstance.update (VM4510 lit-html.js:662)
at render (VM4489 shady-render.js:126)
at HTMLElement._applyRender (VM4470 lit-element.js:230)
at HTMLElement._propertiesChanged (VM4470 lit-element.js:168)
Now I saw in the pwa-starter-kit README.md that in node 10 I might get some issue like this, but I am using version 9.5...so?

Now I basically I am trying to initially create a clone of the "shop" page (page 3, my-view3.js) called bc-nutrientsofinterest.js.  Where my-view3.js calls shop-products I call nutrientsofinterest-nutrients.  Where these elements call shop-item, I call nutrientsofinterest-item.
All this works fine but when I try to put the  element below nutrientsofinterest-item like there is in shop-products element for the button below shop-item, I get this error.
I am trying to cobble a presentation for Denver Google Development Group meetup in early November...I am running out of time to get done what I wanna get done by the time I gotta get in front of all those people and say I know all about Polymer 3...so if you could help me I would really appreciate it!!

Always helps to put out a CodePen or jsbin.

apologies...github source is not good enough I guess.

Sorry:  how about my site itself
the error occurs when the page renders.  I have turned off minification and bundling.
This page is simply trying to clone my-view3.js only with different names.

The error occurs in nutrientsofinterest-nutrients.js in _render() when I try to add in the button using the same js values that just worked.  If you remove the  element it works...
(If this example is not good enough, I will work on getting a code pen for you...never done that...so it will be a good learning experience for me.)

It looks like you have an old version of lit-html, and this has been fixed.

yayy!!!   sorry to bug you  !!

hm...I changed package.json to ^0.6.1 for lit-element, ran npm i, and on local and in deployed firebase hosted version the issue seems to persist.

how do I tell if anybody is looking at my problem or not?

You'll have more chance of someone looking if you create a clear cut-down example of the issue. Right now you're effectively just asking people to debug your project for you. The shop codebase shows a working example and you're the person most familiar with what has been changed from that.

ok, fair enough!  I will do that.  Thank you @CaptainCodeman

well I figured out I had commented out some markup in render() which gives lit-element heartburn.