cmd/compile: use hashing for switch statements

Currently we implement switch statements with binary search. This works well for values that fit into a register, but for strings it might be better to compute a hash function.
Some thoughts:

Even in the simple case of using a full hash function, exactly two passes over the switched value is probably better than O(log N) possible passes over it.

Since we have the expected keys statically, we can use a simple, cheap universal hash (e.g., FNV? djb2?) and keep trying different seeds until we get one without any collisions.

Instead of hashing all of the input bytes, we can hash just enough of them to avoid collisions.

We can try using a minimal (or near minimal) perfect hash to emit a jump table instead of binary search. (This might even be useful for sparse integer values too.)

If someone can help figure out code that can take a []string and figures out an appropriate and cheap hash function to use, I can help wire it into swt.go.

Drive-by comment from phone: It wouldn’t surprise me if “use the first word of the string data” turns out to in practice to be a decent first-cut hash, and it is certainly cheap (can be generated inline, should be just a few instructions).

After reading minimal perfect hash function which does have an implementation go-mph, maybe that can just be used as is?

@mdempsky @josharian @pierrec @toothrot Has anyone started on this issue? It seems interesting.

@lranjbar I don't think anyone has started on it yet. Happy to answer any quick questions to get you started. (I probably won't be able to give any more serious guidance/feedback for another week though.)

@mdempsky I okay with waiting a little bit for more guidance. (I do actually have a pretty busy week myself.) I can still dig into it a bit more on my own, I was just curious before I started. :)

About the jump table, we probably prefer a near minimal perfect hash function. A strict minimal perfect hash function would always lead to a collision for the default case of the switch. One option is to extend the size of the jump table from minimal to the next power of 2, and jump to the default case for indexes that don't collide with a pre-computed hash. At the same time, that would allow us to use cheaper bitwise operators instead of the more expensive % operator when calculating the hash/index for the jump table.
A question is how to evaluate the performance of binary search vs jump table. Constructing the near minimal hash probably requires some additional cycles, compared to a cheap hash (or the even cheaper "use the first word of the string data", as mentioned by @josharian). I suppose that the performance benefits of the jump table comes with large n. For small n, the difference might be less clear?

I've implemented a hash function using fnv, based on string length and minimal unique prefix (@lranjbar I hope you don't mind). Also have a working implementation of a near minimal perfect hash function for a jump table.
This finds a unique hash reasonably fast (typically within the first 1-2 seeds) and hash/jumptable benchmarks indicate performance close to maphash on linux/amd64. (I've used all string-typed switch cases from the golang/go repo as testdata.)
@mdempsky how/where should I submit this code?

@jupj Great! Thanks for working on this. Do you have the code available somewhere that I can take a look?
What would probably be best is if you can make a standalone Go package that we can include in the standard repo as something like "cmd/compile/internal/phash" (or ".../perfect hash" or whatever; open to suggestions).
With that done, I can help wire it into gc/swt.go. Or give you pointers on doing this yourself if you'd like.

@mdempsky please find the code here: https://github.com/jupj/go-issue-34381
I'll modify this to a standalone package that could be included in the standard repo.
I'm new to compiler work, so I appreciate any help and/or pointers to wire it into gc/swt.go!

@jupj No worries. It's been almost a year and clearly I didn't have time outside of work to focus on this. 😅 Good luck with your submission.

I think we're on a good path for this, but one thing to double-check as we integrate is that if a giant string arrives for a switch statement only composed of small strings, we don't end up doing O(giant) work to calculate a hash. This protection could have many forms (hash function design, checking max length, checking exact length before hashing, etc.), but we should make sure it's there.

I think we're on a good path for this, but one thing to double-check as we integrate is that if a giant string arrives for a switch statement only composed of small strings, we don't end up doing O(giant) work to calculate a hash. This protection could have many forms (hash function design, checking max length, checking exact length before hashing, etc.), but we should make sure it's there.

Thanks, I updated the CL mentioned above, re-added the check for the minimum unique prefix length. This limits the work to O(prefixLen), with the maximum case string length in the switch statement as the upper bound of prefixLen. Is this sufficient to protect from giant strings?

Any tips for how to wire the perfect hashes into swt.go? I think this could be the next step, to validate the perfect hash functionality. I have two options in mind:

For each switch statement, generate an AST that implements the ad-hoc hash as an anonymous function. So switch s { will be rewritten to switch func(s string) uint32 { /* generated AST */ }(s) {, where the AST implements the selected perfect hash function with given seed and prefix length.

Generate functions (go source code) for all different combinations of cheap hashes and include them in the runtime library as runtime.phash_Len, runtime.phash_First, runtime.phash_LenFirst, etc. The compiler will rewrite the switch statements to use these, eg. statement switch s { will be rewritten to switch runtime.phash_LenFirst(s) {. The compiler will include seed and prefix parameters to the hash function where needed.

I don't have any experience generating AST, so I feel the latter option would be more approachable for me. There are 31 different combinations (fnv fallback included), so I think it would make sense to generate the functions into the runtime library, at least for programs with more than 31 switch cases.
Comments?

We should generate AST.
Or leave it alone in swt.go and generate SSA in gc/ssa.go. I find generating SSA easier than generating AST, but since this would be the first switch-to-SSA direct lowering there's no sample code to work from and you're more likely to hit unexpected stumbling blocks.

As an update here: On Sunday, I played with integrating phash into the Go compiler and got it mostly working. CL is 260597 above. (If you're really curious, I streamed the compiler development process on Twitch at https://www.twitch.tv/videos/761008858; string hashing stuff starts at 37m20s and runs to the end of the stream. Beware Twitch only stores videos for 14 days.)
You can build targets with -gcflags=all=-d=swthash to enable the optimization. I haven't done any performance/size benchmarking yet.

Thanks for the stream, very insightful to watch!

Update: I've updated phash to generate ir for computing the hash (https://golang.org/cl/256898), and updated mdempsky's proof-of-concept to use the new ir nodes in https://golang.org/cl/307191
Benchmarked with compilecmp:
compilecmp master -> HEAD
master (c40dc677be): go/doc: avoid panic on references to functions with no body
HEAD (49958c0094): swthash always on

benchstat -geomean  /tmp/951139350 /tmp/784215165

completed 100 of 100, estimated time remaining 0s (ETA 8:37PM)
name                      old time/op       new time/op       delta
Template                        327ms ± 4%        326ms ± 3%    ~     (p=0.957 n=98+100)
Unicode                         141ms ± 9%        141ms ± 9%    ~     (p=0.474 n=98+100)
GoTypes                         1.78s ± 2%        1.78s ± 2%    ~     (p=0.329 n=99+100)
Compiler                        153ms ± 9%        152ms ± 8%    ~     (p=0.114 n=97+95)
SSA                             12.9s ± 1%        12.8s ± 2%  -0.27%  (p=0.002 n=100+100)
Flate                           205ms ± 5%        205ms ± 5%    ~     (p=0.366 n=100+98)
GoParser                        306ms ± 4%        305ms ± 5%    ~     (p=0.207 n=92+99)
Reflect                         723ms ± 2%        723ms ± 2%    ~     (p=0.428 n=99+97)
Tar                             285ms ± 5%        285ms ± 6%    ~     (p=0.876 n=99+100)
XML                             368ms ± 3%        369ms ± 4%    ~     (p=0.179 n=98+98)
LinkCompiler                    628ms ± 3%        630ms ± 3%    ~     (p=0.124 n=97+95)
ExternalLinkCompiler            1.72s ± 2%        1.72s ± 2%    ~     (p=0.733 n=94+96)
LinkWithoutDebugCompiler        394ms ± 4%        395ms ± 4%    ~     (p=0.592 n=99+100)
[Geo mean]                      539ms             539ms       -0.02%

name                      old user-time/op  new user-time/op  delta
Template                        409ms ± 6%        408ms ± 8%    ~     (p=0.515 n=99+100)
Unicode                         186ms ±14%        185ms ±12%    ~     (p=0.752 n=99+100)
GoTypes                         2.19s ± 5%        2.19s ± 5%    ~     (p=0.415 n=100+100)
Compiler                        186ms ±12%        184ms ±14%    ~     (p=0.220 n=100+100)
SSA                             17.4s ± 4%        17.2s ± 5%  -0.90%  (p=0.002 n=100+100)
Flate                           249ms ± 9%        247ms ±10%    ~     (p=0.380 n=98+99)
GoParser                        359ms ± 9%        358ms ± 9%    ~     (p=0.200 n=96+99)
Reflect                         914ms ± 4%        914ms ± 4%    ~     (p=0.948 n=97+99)
Tar                             345ms ± 9%        347ms ±11%    ~     (p=0.430 n=99+100)
XML                             449ms ± 7%        449ms ± 6%    ~     (p=0.904 n=100+100)
LinkCompiler                    1.09s ± 5%        1.10s ± 4%  +0.63%  (p=0.032 n=100+100)
ExternalLinkCompiler            2.09s ± 3%        2.09s ± 4%    ~     (p=0.220 n=100+100)
LinkWithoutDebugCompiler        511ms ± 8%        515ms ± 6%  +0.92%  (p=0.043 n=99+97)
[Geo mean]                      689ms             689ms       -0.06%

name                      old text-bytes    new text-bytes    delta
HelloSize                       814kB ± 0%        813kB ± 0%  -0.09%  (p=0.000 n=100+100)

name                      old data-bytes    new data-bytes    delta
HelloSize                      13.1kB ± 0%       13.1kB ± 0%    ~     (all equal)

name                      old bss-bytes     new bss-bytes     delta
HelloSize                       206kB ± 0%        206kB ± 0%    ~     (all equal)

name                      old exe-bytes     new exe-bytes     delta
HelloSize                      1.21MB ± 0%       1.21MB ± 0%  -0.03%  (p=0.000 n=100+100)

file      before    after     Δ       %
addr2line 3696741   3696773   +32     +0.001%
api       5166358   5170862   +4504   +0.087%
asm       4734867   4738515   +3648   +0.077%
buildid   2445407   2445631   +224    +0.009%
cgo       4453043   4453171   +128    +0.003%
compile   23377375  23428593  +51218  +0.219%
cover     4565624   4566136   +512    +0.011%
dist      3288992   3295920   +6928   +0.211%
doc       3742131   3745099   +2968   +0.079%
fix       3154314   3159106   +4792   +0.152%
link      6561029   6562917   +1888   +0.029%
nm        3666294   3669582   +3288   +0.090%
objdump   4093865   4095577   +1712   +0.042%
pack      2218057   2217913   -144    -0.006%
pprof     13600084  13606156  +6072   +0.045%
test2json 2521324   2521068   -256    -0.010%
trace     10223351  10227271  +3920   +0.038%
vet       7033010   7040450   +7440   +0.106%
total     108541866 108640740 +98874  +0.091%

I streamed the compiler development process on Twitch at https://www.twitch.tv/videos/761008858; string hashing stuff starts at 37m20s and runs to the end of the stream. Beware Twitch only stores videos for 14 days.)

Could this content have been stored on another service (eg. Youtube/Vimeo/etc) because, 8 months later and I'm reading this issue gaining some fantastic learnings, and am not able to see that stream, which has obviously been helpful to the creation of the code


Loading grafana config whenever updated

What is missing?
Whenever config object for grafana is updated, grafana should load new changes.
Why do we need it?
On update of grafana config object in k8s, grafana should also get updated with new config so as to avoid manual intervention.
Environment
All

kube-prometheus version: release-0.6
ea7a834

Anything else we need to know?:

This is possible to implement but not trivial. As far as I can see there are two ways:

Automate recreation of grafana pod on config change
Use grafana admin API and a sidecar container

Personally, I think using API is a much cleaner way and we can use https://github.com/jimmidyson/configmap-reload but we need to handle authentication to admin endpoints.
We are open to external contributors creating PR with implementation, but we probably won't be actively trying to implement it right now.

@paulfantom Cool, I can pick this up. Looks like you have provided enough info, is there anything else I need know before picking this up.

Few things:

everything grafana-related should be in https://github.com/prometheus-operator/kube-prometheus/blob/main/jsonnet/kube-prometheus/components/grafana.libsonnet
grafana configuration is optional (as seen in https://github.com/prometheus-operator/kube-prometheus/blob/main/jsonnet/kube-prometheus/components/grafana.libsonnet#L69), so reloader sidecar should be too.

I would suggest experimenting with config reloader sidecar handling only reloading of main grafana config. Next, we can figure out if reloading of dashboards and datasources is needed.

Thanks @paulfantom for the info, I'll pick this up in coming weeks and will post any queries when I start working on this.

Few things:

everything grafana-related should be in https://github.com/prometheus-operator/kube-prometheus/blob/main/jsonnet/kube-prometheus/components/grafana.libsonnet
grafana configuration is optional (as seen in https://github.com/prometheus-operator/kube-prometheus/blob/main/jsonnet/kube-prometheus/components/grafana.libsonnet#L69), so reloader sidecar should be too.

I would suggest experimenting with config reloader sidecar handling only reloading of main grafana config. Next, we can figure out if reloading of dashboards and datasources is needed.

So I dont know if someone is working on it but for us a scenario where we need to reload datasources is when using vault for the datasource's credentials. So we template the datasource configuration with vault and would like to use dynamic database credentials. This requires the datasource.yaml to be reloaded on changes though. For now we are stuck with static credentials that get loaded on startup.


Add property based test to check parse reverses stringify correctly

What's the goal of this? What is the benefit over the existing tests?

The idea of this test is to cover (and maybe discover) edge-cases of the query-string package we have not think about in the already existing unit-tests.
Limitation of query-string I found with this technic
For instance, in the documentation nothing states precisely that not all the strings are valid as keys and/or values. Using this approach, I caught the following corner-case:
const qs = require('query-string');
qs.stringify(data);

https://runkit.com/dubzzz/bug-query-string
Either a bug or an expected case (surrogates characters) but I have seen nothing about it in the documentation. Note that those values of string are valid for JSON.stringify for instance.
The test itself
Basically the test just generates a valid entry for stringify of query-string, stringify it and check that it can decode it back to its original value. It covers the range of all possible combinations and can in some cases find edge-cases difficult to diagnose and think about.
Property based testing in a nutshell
Property based testing is a testing approach which has appeared mostly in functional world through QuickCheck framework (Haskell). The idea behind this approach is that it is difficult to cover the range of all possible failures and corner-cases using unit-tests which are mostly example based tests in the industry.
A property is of the form: Given any entry satisfying requirements (here: an object with string keys and string values) the following statement holds (here: I can stringify it and read it back).
Thsi approach is quite useful as it makes very few assumptions on the inputs. For instance it does not state that the string should be X char longs or that the object should have X keys. Moreover whener it fails it shrink its failing input to the minimal failing one (cf. runkit link where the framework give back an example with a sign key object and a string of length one).

Except the issue with half surrogate pairs, I have not found any other issues.
I will cancel this pull request as it was not able to discover cases not handled by classical unit-tests.


Optionally require forwardable

Newer versions of Ruby moved Forwardable
into a forwardable module.
Hey @justincampbell if you have a change to take a peek please do. If not I'll probably go ahead and merge in the next day or so.
Fixes #29

This makes sense! But I don't understand the if RUBY_DESCRIPTION part. Is that supposed to be matched against something?

RUBY_DESCRIPTION was added in 2.3 for the full Ruby version. I thought that we'd have errors in < 2.3 with the require. TIL it doesn't!

Cool, thanks for updating!


Swapper create config from command line

Prompt to add folders:

src
edits
renames

To find these pathnames:
Mac - pwd
Windows - control shift right click, copy AS path
Script should convert str path -> list, str folders

Swapper should make a .gitkeep for the newly made config

Swapper deleted


obs-studio: Update to version 28.1.2

What type of Apps related PR is this? (check all applicable)

[ ] ✨ Add Manifest
[ ] ♻ Refactor
[x] 🐛 Bug Fix
[ ] 👷 Optimization
[ ] 🚩 Other

Description
Remove the OBS-Studio 32bits Url for the new version, which isn't available anymore
Related Tickets & Documents
[optional] What gif/emoji best describes this PR or how it makes you feel?
:alien: :truck: :bug: :unamused:
Did you verify the followings
Add/Modify a description in the manifest?

[x] 👍 yes
[ ] 🙋 yes, but I need help
[ ] 🙅 no, because they aren't needed

Check URLs?

[x] 👍 yes
[ ] 🙋 yes, but I need help
[ ] 🙅 no, because they aren't needed

Update to the latest version?

[x] 👍 yes
[ ] 🙋 yes, but I need help
[ ] 🙅 no, because they aren't needed

Update Hashes?

[x] 👍 yes
[ ] 🙋 yes, but I need help
[ ] 🙅 no, because they aren't needed

Format the json manifest?

[x] 👍 yes
[ ] 🙋 yes, but I need help
[ ] 🙅 no, because they aren't needed

Did you identify extra steps
Notes in the manifest?

[x] 👍 yes
[ ] 🙋 yes, but I need help
[ ] 🙅 no, because they aren't needed

Application Dependencies?

[ ] 👍 yes
[ ] 🙋 yes, but I need help
[x] 🙅 no, because they aren't needed

Application Suggestions?

[ ] 👍 yes
[ ] 🙋 yes, but I need help
[x] 🙅 no, because they aren't needed

Pre-installation steps to perform?

[x] 👍 yes
[ ] 🙋 yes, but I need help
[ ] 🙅 no, because they aren't needed

Post-installation steps to perform?

[ ] 👍 yes
[ ] 🙋 yes, but I need help
[x] 🙅 no, because they aren't needed

File persistence to perform?

[x] 👍 yes
[ ] 🙋 yes, but I need help
[ ] 🙅 no, because they aren't needed

All changes look good.
Wait for review from human collaborators.
obs-studio

[x] Description
[x] License
[x] Hashes
[x] Checkver
[x] Autoupdate


The days of first page of monthDays are not highligted

Version i am using:
The latest one
My problem
I am starting a ViewController, then I make an api call and it returns me some dates. These date must me highligted.
What I've done
I have followed the wiki section and i have created a custom cell.
The dates apart from the first month are showed perfectly but the first month is not.
The peculiar behavior
When I swipe left and then right the dates of the first month are highlighted..
Some pictures
What I see when I fresh start the ViewController at the first month:
https://prnt.sc/oix479
What I see when I swip left and the right:
https://prnt.sc/oix4ct
I guess this happens because the calendar draw itself faster than my response from the webservice. Is there any way to solve this issue?

have you tried calendarView.ReloadData() after your API returns with the date information?

Thank you a lot I feel so dump for not finding that useful method.
Have a great day @patchthecode

Its ok.
havea  good one.


WIP: fixed upload file kabuto link, halfway fixed download file, still nee…

…d to add delay to await link

See #351


Make tolerations configurable in clustermesh-apiserver certgen job

A certgen job can be used to generate the TLS certificates required both for hubble and clustermesh. Yet, the clustermesh one lacks the possibility of configuring additional tolerations, which are instead configurable in the hubble case. This is possibly useful for instance to allow running the job on control-plane nodes. Hence, let's fix this divergence.
Related: https://github.com/cilium/cilium/issues/28167

Make tolerations configurable in clustermesh-apiserver certgen job

/test

Should we backport this to v1.14? Considering it seems to be a blocker for some users (ref https://github.com/cilium/cilium/issues/28167)

Should we backport this to v1.14? Considering it seems to be a blocker for some users (ref #28167)

Makes sense to me, marking for backport to v1.14.


Feature/569 otc routes

Changed routing so we can have multiple routes:

/swap/:token1/:token2
/my-orders
/make
/orders/:orderString

Other things:

Refactored Page component. It's now a wrapper for every page.
Added an Interface context so user interactions are saved among pages (like an opened transactions tab)
Moved SwapInputs and TokenSelect to components directory.

https://user-images.githubusercontent.com/86911296/177037451-d92b535a-450a-4a5e-a85e-b2183af387a4.mov

Still need to rebase. Will do tomorrow!

Lots of nice refactoring here, looks easier to manage for sure!
Haven't fully tested yet, will do so when I get a chance, but in the meantime I've reviewed the code and can't see any glaring errors.
Thought it might be worth merging sooner rather than later to ensure we can progress with the rest of the OTC flow without mass conflicts down the line?
If you want to wait for a more thorough review and fine tooth comb test then it will probably have to be Monday from me.


LuckPerms permissions inconsistent across 3 different viewing mediums

Description

LuckPerms is showing 3 different permission sets:
Yaml file

Editor

Command

I've run /lp sync but nothing changed. I'm confident that the storage type is truly YAML.

if you could provide a pastebin of the yaml file and an editor link instead of screenshots, we will be able to place the relation each has in context with the others. Thanks!

Editor: https://luckperms.net/editor/fcHndu47IJ
Yaml file: https://paste.bnn.gg/operator.yml

Thanks, could you please send a screenshot of the output of /lp info as well as the rest of the information originally requested in the issue template? The reason it is requested is because it allows us to help you quickly, and so we do not have to ask again. If you cannot remember what was originally asked,
 is a screenshot of the prompts.
Thank you!

I'm not really clear what the issue is? From your screenshots, it looks like everything is working properly.
The template is not optional, please don't ignore it. It makes it so much more painful for us to help you. If you'd still like our help, please open a new issue and fill out the template properly.


[SMALL] Improve HasKey on derived type exception message

Resolve #3863
A common cause of this exception is unintentionally including a base
type in the model. Adding info about how to stop this happening to the
exception message.
Old Message: The derived type 'Address' cannot have keys other than
those declared on the root type 'AddressBase'.
New Message: A key cannot be configured on 'Address' because it is a
derived type. The key must be configured on the root type 'AddressBase'.
If you did not intend for 'AddressBase' to be included in the model,
ensure that it is not included in a DbSet property on your context,
referenced in a configuration call to ModelBuilder, or referenced from a
navigation property on a type that is included in the model.

@divega can you take a look at the wording

IMO the part about "referenced in a configuration call to ModelBuilder" isn't very clear but I cannot think of a better way other than maybe mentioning the Entity method.
I think this is a big improvement so if you cannot think of anything else :shipit:

Looks good to me.

Yeah I agree it is not super clear... but I also could not come up with anything better :cry:

Merged https://github.com/aspnet/EntityFramework/commit/7fc5cd8c9bc6469de831f25ac48b99ff201d29a5


CLP Setup issue

I'm able to exxcute start-clp and compress scripts, I have imported the sample Hive Data set, when I try to search for following sbin % ./search "Task *" query, I'm getting below error:
My understanding: as per https://github.com/y-scope/clp/releases, clp-package-ubuntu-focal-x86_64-v0.0.2.tar.gz is built for ubuntu-focal.  Do I need to have EC2 instance with ubuntu-focal only ? can you guide me to resolve the below error:
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Traceback (most recent call last):
File "/opt/clp/sbin/native/search", line 261, in
sys.exit(main(sys.argv))
File "/opt/clp/sbin/native/search", line 248, in main
for ip in set(socket.gethostbyname_ex(socket.gethostname())[2]):
socket.gaierror: [Errno -2] Name or service not known
Traceback (most recent call last):
File "./search", line 135, in
sys.exit(main(sys.argv))
File "./search", line 126, in main
subprocess.run(cmd, check=True)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'run', '-i', '--rm', '--network', 'host', '-w', '/opt/clp', '-u', '504:20', '--name', 'clp-search-61c9', '--mount', 'type=bind,src=/Users/rerajesh/Downloads/clp-package-ubuntu-focal-x86_64-v0.0.2,dst=/opt/clp', '--mount', 'type=bind,src=/Users/rerajesh/Downloads/clp-package-ubuntu-focal-x86_64-v0.0.2/var/log/bcd07464dac2,dst=/opt/clp/var/log', '--mount', 'type=bind,src=/Users/rerajesh/Downloads/clp-package-ubuntu-focal-x86_64-v0.0.2/var/data/archives,dst=/mnt/archive-output', 'ghcr.io/y-scope/clp/clp-execution-x86-ubuntu-focal:main', '/opt/clp/sbin/native/search', '--config', '/opt/clp/var/log/.clp-search-61c9-config.yml', 'Task * deprecation']' returned non-zero exit status 1.

Hey @RajeshAccount,
Running on Ubuntu Focal isn't a requirement but the container image it's using is built for x86_64 whereas it looks like you're using an ARM EC2 instance:

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

We'll add some ARM container images but before that, the fastest work around would be to use an x86 EC2 instance. In case that's not possible, we will also write some instructions on how to build and use a local container image.

Thanks for quick response, mean while I tried with x86 EC2 instance, now I'm getting below error while executing start-clp
ubuntu@ip----:~/clp/clp-package-ubuntu-focal-x86_64-v0.0.2$ sudo ./sbin/start-clp
2023-12-21 16:06:59,383 [INFO] [clp] Starting database...

Traceback (most recent call last):
File "/home/ubuntu/clp/clp-package-ubuntu-focal-x86_64-v0.0.2/./sbin/start-clp", line 462, in main
start_queue(instance_id, clp_config)
File "/home/ubuntu/clp/clp-package-ubuntu-focal-x86_64-v0.0.2/./sbin/start-clp", line 271, in start_queue
subprocess.run(cmd, stdout=subprocess.DEVNULL, check=True)
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['docker', 'exec', '-it', 'clp-queue-589a', 'rabbitmqctl', 'wait', '/tmp/rabbitmq.pid']' returned non-zero exit status 137.

looks like its related to OOM, let me try with higher EC2 instance. Let me know if you have any other suggestions.

no files present under /var/log// Can i only see two empty directory (/var/log//db and /var/log//queue).

Are you running as a normal user or as root? Also what EC2 instance type are you using? I will try and replicate your setup to debug.

EC2 -> c5.9xlarge
normal user

I'm able to run the v0.0.2 release successfully on a c5.9xlarge instance. If you want to compare, here are the commands I ran on the machine after logging in:

Install Docker
sudo snap install docker
sudo addgroup --system docker
sudo adduser $USER docker
sudo snap disable docker
sudo snap enable docker

Log out and log back in

Extract & start clp:
curl -fsSLO https://github.com/y-scope/clp/releases/download/v0.0.2/clp-package-ubuntu-focal-x86_64-v0.0.2.tar.gz
tar xzf clp-package-ubuntu-focal-x86_64-v0.0.2.tar.gz
cd clp-package-ubuntu-focal-x86_64-v0.0.2
sbin/start-clp

Test compression and search on /var/log/dpkg.log
sbin/compress /var/log/dpkg.log
sbin/search purge

Here are some troubleshooting steps you could try:

Run docker ps and then docker stop <container-name> on any containers with the prefix clp-.
sudo rm -r <package>/var/log <package>/var/data
Then try starting the package again

I'm able to successfully import hive data set, I installed docker-ce-24.0.7-1 latest version of docker earlier I had 24.0.5

kirkrodrigues Thanks for quick response and helping me setting up the clp.
I have queries on the supported search patterns, request to share any documentation on the search query pattern. Patterns I'm looking for are date range search (searching specific date range instead of entire time range), AND, OR search support.

Hey @RajeshAccount, glad you got it working.
CLP currently supports wildcard queries where:

* matches 0 or more characters, and
? matches any single character

We do support filtering by timestamp in the clp binary but we haven't exposed that yet in the package version of CLP. We'll look into adding that quickly so that you can test with it; we actually have a rewrite of the distributed compression and search code (that improves search performance and adds some of these missing features) coming in the 1st/2nd week of January.
If you just want to test single-threaded compression and search performance, you can use the clp binaries directly instead of the package:

Start a container containing the clp binaries:
docker run \
  -u$(id -u):$(id -g) \
  -v<dir-containing-logs>:/mnt/logs \
  -v<output-dir>:/mnt/output \
  --rm \
  -it \
  clp/clp-core-x86-ubuntu-focal /bin/bash

Change <dir-containing-logs> and <output-dir> to the relevant paths on your system.

Then follow the usage instructions here.

You should set archives-dir to a path inside /mnt/output/
Your input logs should be a path inside /mnt/logs
clg supports time-range filtering using a syntax like ./clg --tge<PHONE_NUMBER>000 --tlt<PHONE_NUMBER>999 my-archives-dir which means search for logs which have a millisecond epoch timestamp in the range [1703242608000,<PHONE_NUMBER>999). You can see complete usage instructions with ./clg --help.

CLP's search doesn't currently support boolean operators (AND, OR, etc.). This is on our roadmap, but we don't have an exact date for it yet. That said, we are open-sourcing a new storage engine for semi-structured logs (JSON, etc.) in the next few weeks and that supports a KQL query language (which itself supports boolean operators, range filtering, etc.). We'll ping you when that's ready to try.

Hi @RajeshAccount,
We've added support for filtering by timestamps to the package. To try it, you can build the package from main by using the packager tool. Let us know if you have any trouble building the package.
Once built, the search command (<package>/sbin/search.sh) will show new options for filtering by timestamp (--begin-time and --end-time). Note that currently, timestamps must be specified as milliseconds since the UNIX epoch.


Website: Fixing Icons on Genesis page

Fixes icons for social and member name on Safari

This seems to just move the discord icon down by a pixel on chrome and safari?

This seems to just move the discord icon down by a pixel on chrome and safari?

hmm im trying to reproduce the issue on prod but i am not getting it anymore. this was going to fix aligning the icons and the text, and carey and i worked on it together on hangouts on friday. can someone take a look with me today? we can also ask marketing to reproduce the error, since i am not seeing it anymore..?

closing this one to make new pr with other fixes


Picarto update

"No stream found" since last update (26/11)

It seems like this project has been abandoned by the original developer.
streamlink is a fork of this project with multiple active developers. They also already fixed the issue with picarto.


[Backport release-21.11] llvmPackages_git: fix darwin build

Motivation for this change
Backport #147668. Same fix as #147514, but for llvmPackages_git instead of llvmPackages_13
ZHF: #144627
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[X] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[X] Fits CONTRIBUTING.md.

We usually wait for the master PR to get merged then use the backport label for this, that way the commits end up with the right links to the final commits from which they were cherry-picked.

I marked this as draft so that I can update this PR when #147668 gets updated.
I believe the backport label can't be used here because of the additional changes here.

:+1: cunning

Marked as ready for review since https://github.com/NixOS/nixpkgs/pull/147668 has been merged.


problem

Description
What steps will reproduce the problem?

What is the expected output? What do you see instead?
Please provide any additional information below
OSError: Stream is closed
Version and main components

Spyder Version: 3.2.3
Python Version: 3.5.4
Qt Versions: 5.6.2, PyQt5 5.6 on Windows

Dependencies
pyflakes >=0.6.0 :  1.6.0 (OK)
pycodestyle >=2.3:  2.3.1 (OK)
pygments >=2.0   :  2.2.0 (OK)
pandas >=0.13.1  :  None (NOK)
numpy >=1.7      :  1.13.1 (OK)
sphinx >=0.6.6   :  1.6.3 (OK)
rope >=0.9.4     :  0.9.4-1 (OK)
jedi >=0.9.0     :  0.10.2 (OK)
nbconvert >=4.0  :  5.2.1 (OK)
sympy >=0.7.3    :  None (NOK)
cython >=0.21    :  None (NOK)
qtconsole >=4.2.0:  4.3.1 (OK)
IPython >=4.0    :  6.1.0 (OK)
pylint >=0.25    :  1.7.2 (OK)

Please describe step by step how you got this error so we can reproduce it in our side. If you fail to do that, we'll close this issue in a week from now.
Also, please read our
https://github.com/spyder-ide/spyder/wiki/Troubleshooting-Guide-and-FAQ
to see if you are able to fix this problem by following it.

Thanks for reporting. Please try the steps listed in the Spyder Troubleshooting Guide, specifically the Basic First Aid and Emergency CPR sections, which resolve the great majority of Spyder installation issues.
If it still isn't working, please describe in detail the circumstances in which the error occurred, and post the information listed in relevant subsection of the Calling for Help section as a comment to this issue, and we'll try to help if we can. Otherwise, your issue will likely have to be closed after one week (7 days). Thanks, and best of luck!

Closed due to lack of response. If you can provide more information, please re-open and do so. Thanks.


Upgrade to react v18

Currently blocked by stipsan/react-spring-bottom-sheet#227

@aeharding is this still blocked? Looks like you're on 18.3.1
PS cool project!

Yes thanks, this is fixed now! I think new versions of safari don't have the race condition mentioned in attached issue.


Lake IDs at low zooms

As part of #227, I added lake OSM IDs but only for lakes from OSM data (z6 and above).  Merging with natural earth data is a bit tricky because we process all of natural earth first, then all of openstreetmap.
A few options for how to get OSM IDs on low-zoom lakes:

store all of the natural earth lakes features in-memory in water profile, add additional info to them while processing OSM  elements, then write them out after OSM is finished (like we do with the boundary layer)
Use OSM lakes at lower zoom-levels instead of natural earth.  We could add VW simplification with options like effective/weighted area to planetiler to produce a similar visual effect that natural earth gets with mapshaper
Extract OSM IDs as part of wikidata pre-processing, then go from wikidata ID to OSM ID when processing natural earth elements

Originally posted by @msbarry in https://github.com/onthegomap/planetiler/issues/227#issuecomment-1134455423

Transferred to https://github.com/openmaptiles/planetiler-openmaptiles/issues/15


[cxx-interop] Bail on functions that use unimportable types.

We already fixed this for "global" functions. This is a more generic solution that works for "nested" functions as well. (Members or functions in a namespace.)

@swift-ci please smoke test.

@swift-ci please smoke test OS X.

@swift-ci please test Windows.

@swift-ci please smoke test OS X.


Add doc string for resource.register()

Hi,
I haven't seen any docs on this. I would like to add odo support to pandasdmx. It should be about 3 lines. But without documenting the register API even the simplest task turns into an uphill battle. In particular, what
are the kwargs good for? Can I pass args to the registered callable? By accident I found out that the kwargs contain a dshape item...

does the docstring in #413 help?

I must admit that a quick look does not entirely answer all my
questions. But I cannot take more time to do it thoroughly.
In the main, I'd like to understand what's passed to myfunc:
@odo.resource.register('.*.mysuffix')
def myfunc(uri, **kwargs):
# do the conversion. To what format? csv? PD.DataFrame? jpeg? mp3?
Why is DataFrame better than # mp3? How do the kwargs passed help? Cann
the caller controll the conversion such as in  odo('somefile.mysuffix', 'targetfile.csv', careful=True, lazy=False)
return SomeType # which ones are good?
It would be good to read that kwargs contains dshape (and what else?),
and that the conversion function should ideally return the data in a
format understood by odo such as PD.DataFrame or whatever. The example
on rtd with pickle is not so inspiring because it does not talk about
what a good return type might be, and what the **kwargs are good for.
I do understand that the regex is meant to capture the source file
extension although this is nowhere documented either. It is likewise
unclear whether the passed regex must end with '$' to make sure that
'.sdmx' and '.sdmxml' are not treated the same. But it looks like
normalisation does just that...
Sorry for bothering. I'm an odo newbie who does not want to become an
odo pro. I am just a pandasdmx pro who heard about odo, thinks it is
great and wants to spend a few minutes writing 3 lines to register a
conversion...
Am 26.01.2016 um 21:25 schrieb Joe Jevnik:

does the docstring in #413 https://github.com/blaze/odo/pull/413 help?
—
Reply to this email directly or view it on GitHub
https://github.com/blaze/odo/issues/412#issuecomment-175210053.

do the conversion. To what format? csv? PD.DataFrame? jpeg? mp3?

This is not actually correct. resource does not do any conversions, that is convert's job.
resource is only used to return the objects that should appear in the odo graph. This allows us to turn str into sa.Table so that we can turn a table into a dataframe, or give the path to a csv on disc or s3. The kwargs are what ever you happen to pass to resource, so there is no spec for it. The user is the one who calls it. The positional args are the uri, and then maybe a field specified by ::<name> like postgresql://localhost/db::table.
The three key functions are:

resource :: str -> any : parse a string into some python object for use in the odo graph
convert :: A -> B : convert one structure into another structure.
append :: B, A -> B Append an A to a B, returning the structure that was added to.

Not all of these are needed, for example, in memory structures do not need a resource and there is no way to append to an immutable type.
convert and append will both be passed at least the dshape and any kwargs that were passed to odo. Hopefully this helps understand the odo registration process.

Thanks for this clarification. It helps me up to a point. However, I
guess my remaining questions are as follows:
In the odo docs there is an example on registering a pickle file which
is then returned unpickled. I do not quite see what odo does to it. I
guess it tries to retrieve the type in the graph and from there a path
to the target format. I understand that odo comes with a built-in graph
which, say, pandaSDMX users may extend at runtime by calling the below
function (I have decided not to run it automatically on each import as
it would take too long). Building on this, I have tried this for pandaSDMX:
@odo.resource.register('.*\.sdmx')
def resource_sdmx(uri, **kwargs):
    return Request().get(fromfile=uri).write()

'Request.get().write()' is how pandaSDMX converts SDMX files to
PD.DataFrame.
Now when I execute something like odo('myfile.scmx', 'target.csv'),
odo does an impeccable conversion job.

But how come? I haven't registered any 'convert' function as you
describe below.

how can I pass kwargs to my write method to control the conversion
process? - It seems the kwargs passed to resource() contain a dshape
parameter by default. This needs to be documented as I'd have to dismiss
it because pandaSDMX cannot do anything about it. Based on your
explanation I understand that any other kwargs will be passed on, so I
could write:

odo('myfile.sdmx', 'targetfile.csv', weather='rainy', food='cheese')
and these kwargs could be used to customize conversion from sdmx to
PD... Right?
All you need is a simple "doc string of the day" app that delivers one
of those super-diligently crafted, inspiring, say, pandas or SQLAlchemy
doc strings right onto your desk, and such tedious conversations will be
history.
Am 02.02.2016 um 17:25 schrieb Joe Jevnik:

do the conversion. To what format? csv? PD.DataFrame? jpeg? mp3?

This is not actually correct. |resource| does not do any conversions,
that is |convert|'s job.
|resource| is only used to return the objects that should appear in
the odo graph. This allows us to turn |str| into |sa.Table| so that we
can turn a table into a dataframe, or give the path to a csv on disc
or s3. The kwargs are what ever you happen to pass to |resource|, so
there is no spec for it. The user is the one who calls it. The
positional args are the uri, and then maybe a field specified by
|::| like |postgresql://localhost/db::table|.
The three key functions are:

|resource :: str -> any| : parse a string into some python object
for use in the odo graph
|convert :: A -> B| : convert one structure into another structure.
|append :: B, A -> B| Append an |A| to a |B|, returning the
structure that was added to.

Not all of these are needed, for example, in memory structures do not
need a |resource| and there is no way to append to an immutable type.
|convert| and |append| will both be passed at least the |dshape| and
any kwargs that were passed to |odo|. Hopefully this helps understand
the odo registration process.
—
Reply to this email directly or view it on GitHub
https://github.com/blaze/odo/issues/412#issuecomment-178670719.

The issue is that you are using resource to convert from your file format into a pd.DataFrame. Most on disk formats have a wrapper type for this reason. resource for csv files return a CSV object, not a dataframe. The reason that it is able to convert your file to csv is that odo always sees it as a dataframe and there is a dataframe -> csv edge. What I would suggest is making a class like:
class PandasSDMX(object):
    def __init__(self, uri):
        self.uri = uri

Then your resource function should return an instance of this class. You will also want a discover function for this type to check the datashape of the data.
@resource.register(r'*\.sdmx')
def _(uri, **kwargs):
    return PandasSDMX(uri)

@discover.register(PandasSDMX)
def _(sdmx):
    # is there some metadata for the dtype?
    # if not just load the first couple of lines like a csv
    raise NotImplementedError('idk how to discover PandasSDMX')

Now you can trivially define a convert function like:
from toolz import keyfilter
import toolz.curried.operator as op

from odo.utils import keywords

@convert.register(pd.DataFrame, PandasSDMX)
def _(sdmx, **kwargs):
    write = Request().get(fromfile=sdmx.uri).write
    return write(**keyfilter(op.contains(keywords(write)), kwargs))

Now you can do things like:
odo('a.sdmx', np.ndarray, a=1, b=2)

and a and b will go to your write method.
I would recomend looking at odo/backends/csv.py or odo/backends/json.py as they do very similar things.

Wow!
So I need this wrapper type really just to store the uri. OK, no
problem. But how about append etc.?
I think I now get a grasp of how convert works. Only the comments in
'discover' do not ring a bell yet. What is the meta-data format? Is it
just a dshape string? So do I have to write a function that reads an
sdmx file and returns a dshape for it? -
I think I can implement basic odo support for the next pandasdmx release.
That said, can I suggest that you take your last msg, add some stuff on
append and discover, and add it to the odo docs as a tutorial?
Leo
Am 04.02.2016 um 19:33 schrieb Joe Jevnik:

The issue is that you are using |resource| to convert from your file
format into a |pd.DataFrame|. Most on disk formats have a wrapper type
for this reason. |resource| for csv files return a |CSV| object, not a
dataframe. The reason that it is able to convert your file to csv is
that odo always sees it as a dataframe and there is a dataframe -> csv
edge. What I would suggest is making a class like:
class PandasSDMX(object):
def init(self, uri):
self.uri = uri
Then your resource function should return an instance of this class.
You will also want a |discover| function for this type to check the
datashape of the data.
@resource.register(r'*.sdmx')
def _(uri, **kwargs):
return PandasSDMX(uri)
@discover.register(PandasSDMX)
def _(sdmx):
# is there some metadata for the dtype?
# if not just load the first couple of lines like a csv
raise NotImplementedError('idk how to discover PandasSDMX')
Now you can trivially define a convert function like:
from toolz import keyfilter
import toolz.curried.operator as op
from odo.utils import keywords
@convert.register(pd.DataFrame, PandasSDMX)
def _(sdmx, **kwargs):
write = Request().get(fromfile=sdmx.uri).write
return write(**keyfilter(op.contains(keywords(write)), kwargs))
Now you can do things like:
odo('a.sdmx', np.ndarray, a=1, b=2)
and |a| and |b| will go to your |write| method.
I would recomend looking at |odo/backends/csv.py| or
|odo/backends/json.py| as they do very similar things.
—
Reply to this email directly or view it on GitHub
https://github.com/blaze/odo/issues/412#issuecomment-179987015.

The box type is needed because we use the type of objects in the graph. It is sort of unfortunate but luckily it doesn't contribute meaningfully to the runtime. Odo is not designed to be used in an inner loop situation.
You don't need append from sdmx to another type because you have already hooked into the graph by defining the single edge. For example, let's walk through this call:
odo('myfile.sdmx', 'postgresl://localhost/db::some_table')

The first thing odo will do is call resource on the strings to figure out what it is working with. This should give us a PandasSDMX object and a sqlalchemy.Table object respectivly.
Next, odo says, "is this a conversion or an append". It sees that the rhs is not a type object but an instance so it knows that this is an append operation.
Then it asks, "can we append a PandasSDMX into an sa.Table. Since it cannot do this directly it looks to find a path through the graph. Luckily we have already defined an edge from PandasSDMX to pd.DataFrame and odo comes with an edge from pd.DataFrame to sa.Table.
We then perform the needed conversions so that we can append the data.
discover should return a datashape object, not a string. This can be parsed with datashape.dshape. I am not sure how to get the shape information from one of the response objects you have showed but I imagine there is some way. If not, just convert to a dataframe and then discover the dataframe.
You might be interested in a notebook I did for a short odo tutorial at strata: https://github.com/cpcloud/strata-nyc-2015/blob/master/Extending Odo.ipynb. There is no prose as it was live demo; but, I think it will help show an end to end implementation for a trivial type. I will try to pair up some examples and prose from this discussion and turn it into docs for odo.

Hi,
sorry for the late reply. I have been very busy with other matters.
Based on your message below I have added basic odo support to pandaSDMX.
Thanks.
Leo
Am 04.02.2016 um 21:34 schrieb Joe Jevnik:

The box type is needed because we use the type of objects in the
graph. It is sort of unfortunate but luckily it doesn't contribute
meaningfully to the runtime. Odo is not designed to be used in an
inner loop situation.
You don't need append from sdmx to another type because you have
already hooked into the graph by defining the single edge. For
example, let's walk through this call:
odo('myfile.sdmx', 'postgresl://localhost/db::some_table')
The first thing odo will do is call |resource| on the strings to
figure out what it is working with. This should give us a |PandasSDMX|
object and a |sqlalchemy.Table| object respectivly.
Next, odo says, "is this a conversion or an append". It sees that the
rhs is not a type object but an instance so it knows that this is an
append operation.
Then it asks, "can we append a |PandasSDMX| into an |sa.Table|. Since
it cannot do this directly it looks to find a path through the graph.
Luckily we have already defined an edge from |PandasSDMX| to
|pd.DataFrame| and odo comes with an edge from |pd.DataFrame| to
|sa.Table|.
We then perform the needed conversions so that we can append the data.
|discover| should return a datashape object, not a string. This can be
parsed with |datashape.dshape|. I am not sure how to get the shape
information from one of the response objects you have showed but I
imagine there is some way. If not, just convert to a dataframe and
then discover the dataframe.
You might be interested in a notebook I did for a short odo tutorial
at strata:
https://github.com/cpcloud/strata-nyc-2015/blob/master/Extending Odo.ipynb.
There is no prose as it was live demo; but, I think it will help show
an end to end implementation for a trivial type. I will try to pair up
some examples and prose from this discussion and turn it into docs for
odo.
—
Reply to this email directly or view it on GitHub
https://github.com/blaze/odo/issues/412#issuecomment-180034872.


AURO MIGRATION: Update node to v22

Resolves AlaskaAirlines/auro-templates#6
Summary by Sourcery
Update Node to v22 and revise contributing guidelines.
Build:

Update the build step to include PostCSS and update Node to v22

CI:

Update CI to test on Node 20 and 22

Documentation:

Revise contributing guidelines, update issue templates, and add settings YAML file.

:tada: This PR is included in version 4.1.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Trivial: Update the documentation for the output param

I updated the description for the output parameter to indicate that it can be a directory
Currently it only shows it can be the file name
Original PR to make the code change is #66

Thanks!


Slow stop services

This PR is to address the issues we have seen in slow start of autoscaled nodes.There are 2 parts in this PR (and I can split it if preferred):

don't try to stop a non-running service: since we go to monit, at time the process can take much longer to actually do nothing;

autoscaled node don't check for DB tables: this shave off about 1minute to the ready time for autoscaled nodes.

OCD build http://ocd.appscale.net:8080/job/Daily Build/6652/

This PR wis superseded by the move to systemd.


Question: How to set different IdentityServerAuthenticationOptions for integration tests?

Hello, in older version, it was possible to set IdentityServerAuthenticationOptions as a singleton to DI Container like this:
public void ConfigureServices(IServiceCollection services)
{
    services.TryAddSingleton(new IdentityServerAuthenticationOptions
    {
        Authority = Configuration.IdentityServerAuthority(),
        ScopeName = "api1",
        ScopeSecret = "secret",
        //...,
    });
}

public void Configure(IApplicationBuilder app)
{
    var options = app.ApplicationServices.GetService<IdentityServerAuthenticationOptions>()
    app.UseIdentityServerAuthentication(options);

    //...
}

but now I cant find any way ho to do it (I need different settings for production/development and different for integration testing (there identity server runs in memory trough TestServer).
I tried something like this
services.TryAddSingleton(new IdentityServerAuthenticationOptions
{
    Authority = identityServerConfig.Authority,
    RequireHttpsMetadata = false,
    ApiName = Constants.API_ID,
    RoleClaimType = CustomClaimTypes.Role,
    NameClaimType = CustomClaimTypes.Name,
});
services.AddAuthentication("Bearer")
    .AddIdentityServerAuthentication("Bearer");

but it looks I cant create from scratch my own instance of IdentityServerAuthenticationOptions. now I can only use .AddIdentityServerAuthentication(options => ...) to modify existing instance...
Is there any way how to achieve this? Thanks

Here's how we do it: https://github.com/IdentityServer/IdentityServer4/blob/dev/test/IdentityServer.IntegrationTests/Common/IdentityServerPipeline.cs#L74

so there is no way how to do it with .UseStartup() ?

We use the IOptions pattern in ASP.NET Core, so you can look into how to configure that.

the solution for me was IStartup interface which is not well documented. More info for example here: (just in case anyone will have same problem)
https://www.strathweb.com/2017/06/resolving-asp-net-core-startup-class-from-the-di-container/


ponyc: update to 0.37.0

Motivation for this change
ponyc had been stalled at v0.33.2 for a while for three reasons:

The buildsystem had been changed from make to cmake
A bug was discovered in LLVM 9.x which caused SIGSEGV in the test suite.  The fix has been applied to LLVM upstream but is in the 10.x and master branch.  Since ponyc currently uses 9.x, the build requires a patched+fixed llvm for the build only
The additional complexity of the (temporary) build-system in use was a real pain to integrate into a nix package.

I believe I have finally tamed this beast.
Things done

[X] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[X] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[X] Ensured that relevant documentation is up to date
[X] Fits CONTRIBUTING.md.

Note - I do not own a Mac so I'm going to be reliant on hydra to test against that platform.
Thanks!

Is there any way we can use the already in-tree expressions for gtest/gbenchmark?
Not sure if optimal for this case, but I wonder if it would simplify these changes, any.

Is there any way we can use the already in-tree expressions for gtest/gbenchmark?
Not sure if optimal for this case, but I wonder if it would simplify these changes, any.

I actually tried that originally as I wanted to reduce the amount of work I had to do.  I was not successful in pulling it in successfully.  That very well may be due to my lack of knowledge of cmake, but I spent several hours in the attempt.
I consoled myself by reminding myself that it never exits the build process.

Reference https://github.com/NixOS/nixpkgs/issues/98376

While attempting to rebase I made a mess.  Will open a new PR.


Unable to set expires attribute in Cookie

This is the JAXRS-pac4j counterpart of issue https://github.com/pac4j/pac4j/issues/1356.
This issue is about leveraging the additional expiry attribute added to the Cookie class in pac4j-core by https://github.com/pac4j/pac4j/issues/1356, and set the corresponding cookie attribute if specified.
Pending pull request, but requires that issue 1356 @ pacj4j is merged and release (dependancy update is of course required).

Pull request created now that pac4j 3.8.0 is out.

Fixed in v3.0.1.


[ci skip] Add note regarding restarting CI builds to the CONTRIBUTING.md guidelines

Description
Add a section detailing how to restart builds on the CI server(s) Fabric uses to the contributing guidelines + fixing a typo.
Motivation and Context

Fixes #1816
How Has This Been Tested?

N/A (documentation)
Checklist:

[x] I have added a Signed-off-by
[x] Either no new documentation is required by this change, OR I added new documentation
[x] Either no new tests are required by this change, OR I added new tests
[x] I have run goimports, go vet, and golint. I have cleaned up all valid errors and warnings in code I have added or modified. These tools may generate false positives.  The goal is clean, consistent, and readable code.

Signed-off-by: JonathanLevi

@christo4ferris et al., please can you review this?

LGTM thanks

@JonathanLevi : trigger-phrase from PR always should be "reverify jenkins" and it is depend on what we given for trigger-phrase in Jenkins job. We will stick to "reverify jenkins" in Jenkins job configuration. Please modify "reverify Jenkins" to "reverify jenkins"
Also, user should have permissions to re-run the builds from hyperledger/fabric Travis home page. Only Chris, Binhn, Ghaskin, Sheehan can re-run the builds.

@rameshthoomu Thank you. Will change 'Jenkins' to 'jenkins' NP.
Re: permissions, etc. what's your proposed (combined version) please?

Regarding permissions: Only users with push access to hyperledger/fabric repository can re-run the travis builds. Users can ping them in slack (#fabric-ci-status) or request them in PR comment.

@rameshthoomu: I see what you are saying. Trying to simply your suggestion and at the same time already make Jenkins (the upcoming) first class citizen/CI server. Does the below convey the information:

Continuous Integration (CI): Be sure to check Travis or the Slack #fabric-ci-status channel for status of your build. You can re-trigger a build on Jenkins with a PR comment containing reverify jenkins.

Note: While some underlying work to migrate the build system from Travis to Jenkins is taking place, you can ask the maintainers to re-trigger a Travis build for your PR, either by adding a comment to the PR or on the #fabric-ci-status Slack channel.
[the item number is 9. at this point 1. is just for the excerpt]

Yes @JonathanLevi : Changes are LGTM. Thanks

Thanks, LGTM


Incorrect text-field placeholder priority

Consider the following CSS:
label {
   color: red;
}

text-field placeholder {
   color: blue;
}

My intended result is for the textfield's placeholder to be blue, but in fact, it will be red. Since the selector 'text-field placeholder' is more specific, shouldn't it win against the 'label' selector?

Long time ago, but @nunofgs issue is right. when I set a basic label-style, I can't change placeholder styling in any way. Cost me a whole day to find out.
But I guess this project here is dead anyways?!

yes. this project is dead. I am trying to bring it back to life as part of StylingKit
Suggested solution is the best practice for pixate. As label is basic construction block

As a general rule of thumb, styling things like label probably isn't a good idea as it will style ALL labels, across the entire app, and that can lead to undetermined styling.

I am using .p style for all the labels and simply assigned it to all of them
That makes styling much cleaner.
I actually don't use element level selectors, I use only classes


PHP: upgrade phpunit to 8.5.13 to support php8

@stanley-cheung

Apparently recently PHPUnit 8 has been given PHP 8.0 support https://phpunit.de/supported-versions.html (at version 8.5.13). This works out for us as now we can run our existing unit tests against PHPUnit 8 on a PHP 8 docker image.


Correct typo in upsert example

It looks like in this example the result of the initial insert is bound to updated and then referenced on the next line of code as inserted. I think the intention was probably that the initial result be bound to inserted.

Sorry, please ignore. I misread the example.


%functions% role not working when a model is specified in command line

When I specify a model, the function calling seems to stop working:
thomas@[HOSTNAME]:~/projets/llm-functions$ aichat -r %functions% print the millionth prime in python
Call may_execute_py_code '{"code":"import sympy\nprint(sympy.prime(1000000))"}'
> [1] Run, [2] Run & Retrieve, [3] Skip: 1
15485863
thomas@[HOSTNAME]:~/projets/llm-functions$ aichat  --model azure_ai:gpt-4-turbo-1106-preview -r %functions% print the millionth prime in python
Generating the millionth prime number can be achieved in Python by implementing a prime number finding algorithm. Below is [etc]

Only models that support function calling can use function calling.
You can check if your model supports function calling by using the .model <tab>  autocompletion in the REPL.

Models with ⚒ have function calling capabilities.

Adding the following configuration to the model lets AIChat know it supports function calling.
      supports_function_calling: true

https://github.com/sigoden/aichat/blob/5458150ed3203cf13b0371efa2c791ac696cee93/models.yaml#L25-L31

Thanks for your answer! I had not understood that the supports_function_calling: true was needed on top of the documented function_calling: true. Are both necessary ?

function_calling: true
clients:
- type: <some client>
  models:
    - name: <some model>
      supports_function_calling: true


[EMPTY]

This is an empty PR for the Antora previews.

the Antora previews

Jenkins checks for modifications in the doc/ folder, otherwise no preview.

This looks like some of your best work, carry on


Error with caveira's statistics

using the library i noticed that caveira's dictionary in load_all_operators is always zero in all statistics
this is the dictionary:
{ 'dbnos': 0,
  'deaths': 0,
  'headshots': 0,
  'kills': 0,
  'losses': 0,
  'melees': 0,
  'name': 'caveira',
  'statistic': 0,
  'statistic_name': 'Interrogations',
  'time_played': 0,
  'wins': 0,
  'xp': 0 }

Same here, tried a few methods to fetch her stats with the API and always got 0

The only thing I can think of is her internal API name has changed or something, I'll check when I get some time

Will check this later once the new changes are merged, I imagine this will be fixed

Will check this later once the new changes are merged, I imagine this will be fixed


Update lock file for Fastlane.

Summary | Résumé
The lock file needs to be updated to reflect the update to Fastlane.
Test instructions | Instructions pour tester la modification
Run bundle install
Run any of the package scripts that require Fastlane. Running get_latest_version will run Fastlane without needing to upload a build, so it's a good way to test that Fastlane was updated.

Is this from a Fastlane update?

Is this from a Fastlane update?

Yep. When I ran any of the Fastlane scripts, it wouldn't run, saying that I needed to update Fastlane. After updating Fastlane, this lock file was modified.


CSS is broken on first execution

When I first show calendar its css is broken

.fc-axis width is coming as 1px.

When I drag an event it changes to 36 PX and everything is fine. I tried to change this width to 36px but it's not working/implementing.
can anyone please help?

@BrockReece can you please help?


Phones as ship upgrade

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the feature
Would it be possible to add optional config to lock the phones behind a ship upgrade? I would like for the phones to not be available from the start of a run. In addition, there should be a config to set the upgrade price.

I'm planning on adding phone-related ship upgrades in a soonish future update, so I will look into adding this as a config option when I'm working on that portion.


⚠ GLDDNS Update Service API has degraded performance

In 6bb6bdd, GLDDNS Update Service API<EMAIL_ADDRESS>experienced degraded performance:

HTTP code: 404
Response time: 345 ms

Resolved: GLDDNS Update Service API performance has improved in 2e74dd4 after 6 days, 6 hours, 20 minutes.


Bluetooth not work

Hi thaks for the plist!, the bluetooth not work i ask on tonymacx86 and say is not compatible

Hi .
What's your Mac os v ?
What's your laptop model ?
What's your wireless card model ?

Use V7 with mac os 10.14.5


Improved Logging

Log out the timestamps for when things are written to console, for obvious reasons.

Fixed as of the most recent patch. Might need to be revisited later, I guess.


Five sql queries for google_search_console

Write-ups for Levels 1-4 of the SQL project.
Some questions:

I was a bit unsure about what to call the 'Top 20 Worsened Queries Week on Week'. I think 'worsened' sounds a bit archaic but couldn't find an equally succinct word.
For top_100_pages_last_7_days.md, do I need to explain why they are weighted averages? It took me a while to figure out why they were weighted. But I didn't want to put loads of explanation in there if you think it's unnecessary.

Also added Level 5: Top 5 "First Appearance" queries write up


Adding isort for the formatting

Adding some more rules and formatting all files to be more plugin-structure compliant.
@JFRudzinski @ndaelman-hu @Bernadette-Mohr this might affect your branches, when rebasing it can get crazy.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

26 of 43 (60.47%)  changed or added relevant lines in 19 files are covered.
15 unchanged lines in 15 files lost coverage.
Overall coverage increased (+0.01%) to 78.879%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/nomad_simulations/schema_packages/atoms_state.py
2
3
66.67%

src/nomad_simulations/schema_packages/model_system.py
2
3
66.67%

src/nomad_simulations/schema_packages/numerical_settings.py
1
2
50.0%

src/nomad_simulations/schema_packages/physical_property.py
2
3
66.67%

src/nomad_simulations/schema_packages/properties/band_gap.py
0
1
0.0%

src/nomad_simulations/schema_packages/properties/band_structure.py
2
3
66.67%

src/nomad_simulations/schema_packages/properties/energies.py
1
2
50.0%

src/nomad_simulations/schema_packages/properties/fermi_surface.py
0
1
0.0%

src/nomad_simulations/schema_packages/properties/forces.py
1
2
50.0%

src/nomad_simulations/schema_packages/properties/hopping_matrix.py
0
1
0.0%

Files with Coverage Reduction
New Missed Lines
%

src/nomad_simulations/schema_packages/properties/fermi_surface.py
1
76.47%

src/nomad_simulations/schema_packages/model_method.py
1
70.94%

src/nomad_simulations/schema_packages/variables.py
1
83.08%

src/nomad_simulations/schema_packages/physical_property.py
1
93.14%

src/nomad_simulations/schema_packages/properties/energies.py
1
75.0%

src/nomad_simulations/schema_packages/properties/band_structure.py
1
80.18%

src/nomad_simulations/schema_packages/properties/forces.py
1
72.73%

src/nomad_simulations/schema_packages/properties/permittivity.py
1
83.33%

src/nomad_simulations/schema_packages/properties/band_gap.py
1
90.2%

src/nomad_simulations/schema_packages/properties/hopping_matrix.py
1
82.76%

Totals

Change from base Build<PHONE_NUMBER>:
0.01%

Covered Lines:
1647

Relevant Lines:
2088

💛  - Coveralls


npm run docker:run fails on Windows Docker hosts - no /etc/timezone

Specs

Leon version: Any / latest
OS (or browser) version: Windows 10
Node.js version: 10
Complete "npm run check" output:
(optional) Leon package version:

Expected Behavior
npm run docker:run runs Leon on Windows Docker hosts.
Actual Behavior
npm returns an ELIFECYCLE error. This is because the docker:run script in package.json sets the timezone using /etc/timezone which does not exist in Windows Docker hosts. Docker fails to run.
How Do We Reproduce?
Attempt to run Leon using npm run docker:run on Windows hosts.
Extra (like a sample repo to reproduce the issue, etc.)

Hello @zmpetro,
Thanks for opening this issue.
Could you please which of those following way work?

By using Cygwin.
By setting up manually your timezone in the package.json file as follow:"docker:run": "docker run -e TZ=YOUR/TIME_ZONE -d -p 1337:1337 -it leonai/leon"

By replacing the run command in the package.json file, like so:"docker:run": "docker run -d -p 1337:1337 -it leonai/leon"

and adding that at the line 12 of the Dockerfile:ENV TZ YOUR/TIME_ZONE

Hello @louistiti,
I did not try the first method, but methods 2 and 3 both work. I would have submitted a pull request but I thought I would let you guys handle it.
Let me know if you need any help.
Regards,
zmpetro

Got it, thanks for saying!
Could you please try to run with the following command?
docker run -e TZ=UTC -d -p 1337:1337 -it leonai/leon

Hello @louistiti,
Running the following command:
docker run -e TZ=UTC -d -p 1337:1337 -it leonai/leon
works as intended and sets the time in the container to UTC. If I shell into the container and run date, it outputs the time and date in UTC.

 📝 [1.0.0-beta.3] Docker run on Windows

Alright, thanks for your help!
I've updated the docs https://github.com/leon-ai/docs.getleon.ai/commit/840e775d18d89f1e2ad8077ca199a83a9a1a0ae8; https://github.com/leon-ai/leon/commit/f05d155f1b82dd2fd29c5b8fcca16868e8d30e40.
Do not hesitate to report if necessary.


Add support for<EMAIL_ADDRESS>

Details
This PR adds support for<EMAIL_ADDRESS>

Closed by ebaabe7ebd0f37f2368611294083a738963ec603


fix: type error in MealController.createMeal

On running pnpm generate:routes after running pnpm i, I was greeted with the error:
There was a problem resolving type of 'Pick<TMeal, Exclude<keyof TMeal, "meals">>'.
There was a problem resolving type of 'Omit<TMeal, "meals">'.
Generate routes error.
 GenerateMetadataError: No matching model found for referenced type TIngredient.
 in 'MealController.createMeal'
...

On debugging, I found that meals is not a field in TMeal so it cannot be Omitted, hence this fix.

I see.
What else is part of TMeals? Perhaps, ingredients or nutrients?
Maybe that's what was supposed to be omitted?

ingredients is part of TMeal
I changed it to @Body() requestBody: Omit<TMeal, "ingredients"> and there was no error.
I guess that's the solution
I'll just update it.
Thanks


Update introduction.md

adding details on how Holdouts impact SDK calls

🚀 Deployed on https://63742d447288924d0848eba8--cozy-fox-0defec.netlify.app


Mongodb data import

Does ByContity have a database engine like CnchMaterializedMySQL to import Mongodb data into ByContity, or is there a way other than kafka to achieve real-time synchronization of Mongodb data with Bycontiy?

AFAIK currently only mysql is supported


git basics lesson 2024: for next year

Issues that came up:

When resolving conflicts, git returned an error about what default to use when resolving divergent branched:

hint:   git config pull.rebase false  # merge
hint:   git config pull.rebase true   # rebase
hint:   git config pull.ff only       # fast-forward only
We haven't run into this in the past? Didn't come up on my own machine since I must have set this global variable long ago.

For the bonus activity (group using one person's demo repo, causing conflicts), the person whose demo everyone is using must give group members explicit permission to push to it. Need to add this

Nice summary of git merge vs. git rebase:
https://www.atlassian.com/git/tutorials/merging-vs-rebasing


Clarify style rule about testing frameworks

Signed-off-by: Stephen J Day<EMAIL_ADDRESS>

LGTM

LGTM