Use systemd-resolved for rkt container DNS

Hi,
please provide an option to use systemd-resolved for dns in rkt containers.
Why:
Java reads the resolv.conf file once, and the uses the nameserver inside always in the same ordering, if the first one is broken, the responce time grows (default +10s, with tuning +3s) for each request!
with systemd-reselved only  a few requests have longer responce times , systemd-resolved knows the first one fails, and uses the second directly
http://www.myhowto.org/java/42-understanding-host-name-resolution-and-dns-behavior-in-java/
regards f0

This looks for me as issue better suited for https://github.com/appc/cni
@steveeJ what do you think about this?

@jellonek I don't think that this belongs to CNI which only cares about DNS server IPs.

Right. CNI should only give this information to execution engine (in this case - to rkt). How this would be then used by it, in time of application preparation - lays on execution engine side.
So... @alban who in your opinion should look on this?

Shouldn't it be fixed in the application instead? (in Java in this example)
I am worried about the implications of adding systemd-resolved:

systemd-resolved may be used via two interfaces: directly via its D-Bus interface, or via glibc NSS getaddrinfo()

I guess Java does not use resolved's D-Bus interface for DNS resolution. For the glibc NSS plugin, this would need to be added in stage2 to work and I don't think that rkt should inject code in stage2.

@alban the problem is the glibc resolving bahaviour.
glibc uses always the first nameserver (unless you set the rotate option , but this makes the problem bigger, then you have random timeouts), and if this nameserver is not reacheble , for each connection a timeout happems. With resolved, the timeout only happens once, and the you are back to normal operation.
I solved the Problem for me  with using coredns on the backend system and connect via default-restricted network. Coredns has the same behaviour as resolvd

Nice hint to be described in docs, but isn't this too much application specific for now?
Most applications (as based on glibc) expects only that there will be correctly configured /etc/resolv.conf...

@jellonek  hm i do not think this is app specific...
normally a app should use gethostbyname and then goes through the nssswitch.conf  config , this can end in resolv.conf, but also in /etc/hosts or resolve (resolved) depends on the config and order

What you mean by "normally"? This looks like you are assuming a glibc based application.
Do you know how golang based statically compiled application does name resolving? Hint: there is no mechanism to support libnss_resolve or even any other external to app library, so such application which does not require/use any glibc based "base image" (binary blob itself is the only needed thing in such application image).
Most applications based on glibc can use configuration from nsswitch.conf and then they can dynamically include libnss_resolve - but golang is in this area bit specific, so yes, imo it's app specific.

@jellonek  hm yes you are right, i assumed glibc and ignored the rest of the world

@alban @steveeJ @jellonek
This is no longer needed , systemd v231 has a new feature
systemd-resolved now listens on the local IP address <IP_ADDRESS>:53
          for DNS requests. This improves compatibility with local programs
          that do not use the libc NSS or systemd-resolved's bus APIs for name
          resolution. This minimal DNS service is only available to local
          programs and does not implement the full DNS protocol, but enough to
          cover local DNS clients. A new, static resolv.conf file, listing just
          this DNS server is now shipped in /usr/lib/systemd/resolv.conf. It is
          now recommended to make /etc/resolv.conf a symlink to this file in
          order to route all DNS lookups to systemd-resolved, regardless if
          done via NSS, the bus API or raw DNS packets. Note that this local
          DNS service is not as fully featured as the libc NSS or
          systemd-resolved's bus APIs. For example, as unicast DNS cannot be
          used to deliver link-local address information (as this implies
          sending a local interface index along), LLMNR/mDNS support via this
          interface is severely restricted. It is thus strongly recommended for
          all applications to use the libc NSS API or native systemd-resolved
          bus API instead.

Much cleaner resolution with full support of "legacy" applications.
So maybe we could use this as default, if there is no --dns option provided and there is listening process on <IP_ADDRESS>:53?

@f0 is @jellonek's comment what you are suggesting?

@jonboulle  yes

@squeed think you could do a preliminary investigation of this?

Case should be more or less like:

if there is --dns flag - use it, else
if there is process bound on udp <IP_ADDRESS>:53 - use <IP_ADDRESS> like --dns flag value.

I did some quick research. The current systemd documentation states:

It is strongly recommended that local programs use the glibc NSS or bus APIs instead...

Given this, I'm not sure we're going to want to encourage its use. Nonetheless, we may wish to enable the stage1's resolved.

Please look on previous comments in discussion.
Statically linked programs in golang do not use glibc nss, and this is why this discussion was going.

It looks like systemd-resolved introduces some problems around wildcard socket binding and go - so if we do this, it needs to be opt-in.

This seems relevant to the problem we're having in https://github.com/moby/moby/issues/23910#issuecomment-390572398
I'm not sure how we could configure resolved to listen on address <IP_ADDRESS> as well as <IP_ADDRESS> - any ideas?

@nottrobin

I'm not sure how we could configure resolved to listen on address <IP_ADDRESS> as well as <IP_ADDRESS> - any ideas?

You could use DNAT and if necessary - i.e. if systemd-resolved blocks non-local source IPs - also SNAT to redirect packets to <IP_ADDRESS>:53 to <IP_ADDRESS>:53


GitHub doesn't allow an underscore

var regex = require("regex-username");
regex().test("hello_world"); // true

\w is an equivalent to [a-zA-Z0-9_].
\w captures _, but GitHub doesn't allow it in your username.

It's probably something new, cuz i was trying it back than


Common CI is currently broken

Is there an existing issue for this?

[X] I have searched the existing issues

Code improvement description
We currently have a red CI in common/ that goes unnoticed because most PRs don't touch common and CI gets skipped.
We should:

Get back to green.
Bisect the issue to figure out which change caused it, and add the paths edited by the offending change to the list of paths that trigger the CI in common, to prevent a similar issue from being reintroduced in the future.

Awesome, thanks so much! 🎉


Update issue templates

Gathers the instructions for producing actionable bug reports in one place that is presented by the github interface when a user is submitting an issue.  You can try this out at: https://github.com/tesch1/OpenVnmrJ/issues  - Click "New Issue" and it presents options for what sort of issue to submit (currently only "Bug Report") and then the bug report template is presented.

Are you planning any other categories? Maybe something simple for "Questions" or "Suggestions"

How's this?

Excellent


Add glossary for ug

Closes #372
Please let me know if there is anything else to add in the glossary

I just searched online and they said that Glossary should be alphabetical, could you help to update this?

Is it not alphabetical order already?

Is it not alphabetical order already?


Automatically drop NaN values when executing ancom.py

During the analysis, I had some missing metadata values.  This is an inconvenient workflow for the user.
Instead of raising an error when metadata has nan entries, I propose ANCOM would proceed and inform the user which samples were excluded from analysis if any.

Closing this for now in favor of #90  because we're going to rework some of the details, but let me know if you'd like me to re-open this if you're still interested.


Enable immutable string literals in Ruby

Adding these comments to each file enables immutable string literals - which Ruby is working towards for Ruby 3.0.
This was identified by the Rubocop checks failing. This was due to the fact that their release 0.69.0 turned this check on always, instead of optionally or off or something as part of their migration off Ruby 2.2.

NB: Tests are failing because of Rust.


search style in docu and landing page

The search show grey results on landing page.
Blue in normal docu.
In Dark Mode it's a bit hard to read on landing page - different versions of grey.

Oh! I see and I can reproduce it! Thanks for the hint @geisslet


feature/separate-cleanup

This pull request separates the unused snippets cleanup I added recently from the actual snippet generation.
This is because in order to make sure we're not deleting snippets that are referenced, we need to parse a lot of markdown docs, and this takes almost as much time as the generation process. It also doesn't need to happen at the same time/candence than the generation process.
So I separated it to make tests generation cycles faster.
@MIchaelmainer I don't have permissions to add a new pipeline for that repo, can you add the new one (uused-snippets) under that folder for me please? https://microsoftgraph.visualstudio.com/Graph Developer Experiences/_build?definitionScope=\Snippet Generation
Edit: this also adds a conditional statement to avoid trying to commit and push things when there are no changes. This way the pipeline doesn't fail anymore for expected reasons

started a run here https://microsoftgraph.visualstudio.com/Graph Developer Experiences/_build/results?buildId=36462&view=results

started a run here https://microsoftgraph.visualstudio.com/Graph Developer Experiences/_build/results?buildId=36462&view=results

@MIchaelMainer I'm logging off for the day, if the pipeline completes successfully, would you mind merging the PR please? thanks!

@MIchaelMainer I'm logging off for the day, if the pipeline completes successfully, would you mind merging the PR please? thanks!


package: pkg-config error.

The error happens in prepare().
patching file glib/glib/gatomic.c
patching file glib/glib/gcharset.c
patching file glib/glib/gconvert.c
patching file glib/glib/gfileutils.c
patching file glib/glib/glib.h
patching file glib/glib/gutf8.c
patching file glib/glib/gutils.c
patching file glib/glib/gutils.h
patching file glib/glib/gwin32.h
patching file glib/glib/libcharset/localcharset.c
patching file config.guess
      8 [main] sh 328 fork: child -1 - forked process 4008 died unexpectedly, retry 0, exit code 0xC0000142, errno 11
/usr/bin/libtoolize: fork: retry: No child processes
libtoolize: putting auxiliary files in '.'.
libtoolize: copying file './ltmain.sh'
6472320 [main] sh 328 fork: child -1 - forked process 4008 died unexpectedly, retry 0, exit code 0xC0000142, errno 11
/usr/bin/libtoolize: fork: retry: No child processes
libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4macros'.
libtoolize: copying file 'm4macros/libtool.m4'
libtoolize: copying file 'm4macros/ltoptions.m4'
libtoolize: copying file 'm4macros/ltsugar.m4'
libtoolize: copying file 'm4macros/ltversion.m4'
libtoolize: copying file 'm4macros/lt~obsolete.m4'
      7 [main] sh 6136 fork: child -1 - forked process 2648 died unexpectedly, retry 0, exit code 0xC0000142, errno 11
sh: fork: retry: No child processes
      9 [main] sh 2112 fork: child -1 - forked process 4008 died unexpectedly, retry 0, exit code 0xC0000142, errno 11
/usr/bin/autoconf: fork: retry: No child processes
      9 [main] sh 2920 fork: child -1 - forked process 2648 died unexpectedly, retry 0, exit code 0xC0000142, errno 11
/usr/bin/autoconf: fork: retry: No child processes
autom4te: --trace=: no such file or directory
automake-1.12: error: autoconf failed with exit status: 1
autoreconf: automake failed with exit status: 1

Seems you run 32-bit MSYS2.
Close all MSYS2 shells and run "autorebase.bat".
Then start MSYS2 again and work.

Thanks! It Works.


Loop story

@satr - Please note that the Loop article has been included from the public site search (by setting the search flag to false in the heading section)


Support broader allow/deny/warn for fsflibre/OSI approved/deprecated

As mentioned in EmbarkStudios/spdx#1, it would be nice to support the isFsfLibre, isOsiApproved, and isDeprecatedLicenseId fields provided in the SPDX license list.

Resolved by #29


[WIP][SPARK-39319][SQL] Make query context as part of SparkThrowable

What changes were proposed in this pull request?

This PR is to add a new method getQueryContext in SparkThrowable.
It also refactors the data type of Origin.context from String to a case class SQLQueryContext.
Why are the changes needed?

This enriches the information of the error message. With the change, it is possible to have a new pretty printing format error message like
> SELECT * FROM v1;

{
  “errorClass” : [ “DIVIDE_BY_ZERO” ],
  “parameters” : [ {
                   “name” = “config”,
                   “value” = “spark.sql.ansi.enabled”
                 }
                 ],
  “sqlState” : “42000”,
  “context” : {
      “objectType” : “VIEW”,
      “objectName” : “default.v1”
      “indexStart” : 36,
      “indexEnd” : 41,
      “fragment” : “1 / 0” }
   }
}

Does this PR introduce any user-facing change?

No
How was this patch tested?

Existing UT

cc @srielau @cloud-fan @MaxGekk

FYI, I have taken this over, and working on it at the moment.


CDN Link Translation

Ready for review!
@dmoralesm @carburo @alejandronanez
https://github.com/reactjs/es.reactjs.org/issues/4#issuecomment-462069842

Looks good! Just fixed a few typos.


Global Translation File do not generates for Groups in layout page

Describe the bug
While using Features "GenerateCaptions", if we do not put any caption in layout groups, there nothing in .xlf file.
To Reproduce

Create a simple project, with "features": ["TranslationFile","GenerateCaptions"],

Create a Page like
page 50100 "MyCustPage"
{
SourceTable = Customer;
Caption = 'My Cust Page';
PageType = Card;
UsageCategory = None;
layout
{
area(Content)
{
group(General)
{
field(Code; Rec."No.")
{
ApplicationArea = All;
}
}
}
}
}

Compile

Check .g.xlf

Expected behavior
Expected trans-unit section for the group.
May be can not be created if "ShowCaption = false"; but here not the case.
Screenshots

5. Versions:

AL Language: 6.5.413786
Business Central: plateforme 17.0.21485.21985 + application 17.4.21491.22066

GenerateCaptions only supports top level application objects for now.

Thank you for reporting this issue. This is a problem that we are aware of and that we have on our backlog. We will use this issue to track this problem on Github.


Conficts with silverstripe/tagfield

Tagfield (https://github.com/silverstripe-labs/silverstripe-tagfield/) used by the new Blogmodule (https://github.com/silverstripe/silverstripe-blog) uses Select2 4.0.0-rc.2. This module uses 3.2 and this seams to cause a conflict.

@ohararyan is working on a fix for this at the moment :)

Opening a ticket on Tagfield for this as well; I think any modules requiring select2 should just pull it in via composer?
require select2/select2 ~4.0

Thanks. Composer or CDN solution would be fine. Upgrading this module to use select2 4.0.x will be the grunt of the work

Hey! Did anyone figure out a way around this?


fix: exportToSvg does not render mask for arrow text

When calling exportToSvg and exportToBlob, exportToCanvas arrows do not have the mask behind the label.
before:

after:

In
https://github.com/excalidraw/excalidraw/blob/73a45e19889de08be64faf2cdddae2b9ecaa1a20/src/renderer/renderElement.ts#L1087-L1090
boundText is always null, if calling exportToSvg from ExcalidrawLib with elements[] external to the currently loaded scene because `Scene.getScene(element) returns null here:
https://github.com/excalidraw/excalidraw/blob/73a45e19889de08be64faf2cdddae2b9ecaa1a20/src/element/textElement.ts#L524-L537
This can be overcome by creating a temporary scene if elements are not found in the sceneMapByElement
https://github.com/excalidraw/excalidraw/blob/73a45e19889de08be64faf2cdddae2b9ecaa1a20/src/scene/Scene.ts#L27-L28

@zsviczian to confirm is for the package? export to SVG works fine for me in excalidraw.com

Yes, this is for the package. When you call exportToSvg from an existing scene there are no issues, since the scene exists and elements are available in the sceneByElement map and thus Scene.getScene() returns non-null.

Hi @ad1992, what do you think, can we push this into production? or do you have a different solution in mind compared to the one proposed in the PR?

We're running into this problem as well -- and I haven't tested the PR's fix myself but it looks solid. Could we move it forward?


Added leetcode problem Valid Anagram to leetcode problem by Adit Kaushal

#295
Problem statement: Given two strings s and p, return an array of all the start indices of p's anagrams in s. You may return the answer in any order.
An Anagram is a word or phrase formed by rearranging the letters of a different word or phrase, typically using all the original letters exactly once.
Intuition :
using the sliding window concept check if the length is equal to the given string and then check for similiar letters in that window.
@dixitt5 please merge this with the hacktoberfest accepted label

@dixitt5 please merge this.


Fix the doc warning attribute and document remaining items for bevy_window

Objective
Complete the documentation for bevy_window.
Solution
The warn(missing_doc) attribute was only applying to the cursor module as it was declared as an inner attribute. I switched it to an outer attribute and documented the remaining items.

get_display_handle and get_window_handle seems to not be needed as the field they access are public. Could you try removing them instead of documenting them?


Bump version to 0.3.0

Upgrade tfjs-core dependency version to 0.9.1

This change is

Review status: 0 of 4 files reviewed at latest revision, all discussions resolved.

Comments from Reviewable

 can you also bump the tensorflow version to 1.8 for pip package?

Reviewed 4 of 4 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved, some commit checks failed.

Comments from Reviewable


[help] ATOM trouble with installation

I cannot install the terminal into the atom. The Atom has 1.19.2 version on my iMac. Can you explain? Thank you very much

I got the same problem but now it works.
Try to rebuild the package and restart atom

Evle, I did try to rebuild the package and restart the atom but It still appears the same problem.

https://github.com/platformio/platformio-atom-ide-terminal/issues/297


This or accounts-meld?

@splendido talked a ton about merging accounts-meld w/ useraccounts, is this still on the radar?
https://github.com/splendido/meteor-accounts-meld/

What we've decided to do with this is try to change how the core account system handles multiple providers per user, so this will take some time to finish up.
splendido:accounts-meld is "battle-proven", but this code is used in production by at least one live app, and I've yet to see any serious problems with it. I've chosen not to publish it as a package based on the direction me, @brettle and @splendido decided to take with this account merging thing.
If you wanna use this just clone the repo's master branch into your packages folder and do meteor add useraccounts:one-account and you should be able to use this.

I've been looking to get involved, was thinking about working in this space.  Do you use any issue tracking, or just winging it? :)

@brettle Has been the one taking point on this and we're working out of https://github.com/meteor-useraccounts/meteor with this. So have a look in the issues here and over at that repo and then read through the new documents outlining what we have in mind for the "new" system.

Don't have issue access over at that repo.

Oh yeah, we haven't opened the issues over there, we've been working out of the new documents and this issue tracker, but I'll open the issue tracker on that repo so we can keep discussions about the new system over there =)


Make contributing to docs easier

With this PR readers can suggest changes to the docs with an easy to use WYSIWYG markdown editor.
This is how the editor looks like for this repo.
https://github.com/remorses/docusaurus-example/assets/31321188/43c8bc68-3668-4a25-bf1c-a70eceab7bcb

Closed in favor of https://github.com/Louisvdw/dbus-serialbattery/pull/879


dead?

resting

I've resolved this in my own system.  Not sure the best way to suggest the updated code here, so I'll just post it.
Change this:
if (typeof(window.PLAYER)!="object" || typeof(window.PLAYER.nickname) != "string") {
To this:
if (typeof(unsafeWindow.PLAYER)!="object" || typeof(unsafeWindow.PLAYER.nickname) != "string") {
I hope this helps.


Release new version with multi-byte encoding fix

Currently the multi-byte UTF-8 encoding fix has not been released to NPM. Please rev and release. Thanks.

Released #15 as v2.4.2, closing.


Danish translation added

(Added a changelog entry for this + something esle that was missing a changelog entry, will merge this in once it passes)

Thank you for these translations! They'll be in the next release


.source() throws exception when used on a Vimeo video

When using the .source() api method to get the video source on a Vimeo video, the browser throws an exception.
Codepen which displays this behaviour

[x] Issue does not already exist
[ ] Issue observed on https://plyr.io

Expected behaviour
The video url should be returned.
Actual behaviour
The browser throws an exception.
Environment

Browser: Chrome
Version:  60
Operating System: OS X
Version: 10.12.5

Players affected:

[ ] HTML5 Video
[ ] HTML5 Audio
[ ] YouTube
[x] Vimeo

Steps to reproduce

Call the .source() function on an instance of a Vimeo video

Closing this one as I think it's fixed but feel free to re-open if not solved in v3 👍

Hey! Thanks for the work, but I think it's not fixed yet.
I updated the codepen to use v3.0.3 and it still does not log the source url for vimeo videos.
https://codepen.io/NindroidX/pen/MVpNGV

Damn. I’ll take another look then.

On 22 Mar 2018, at 1:55 am, Sam Van Campenhout<EMAIL_ADDRESS>wrote:
Hey! Thanks for the work, but I think it's not fixed yet.
I updated the codepen to use v3.0.3 and it still does not log the source url for vimeo videos.
https://codepen.io/NindroidX/pen/MVpNGV
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or mute the thread.


Fix typo in copyright header

@homu r+

:pushpin: Commit 1ce5e30 has been approved by indiv0

Coverage remained the same at 100.0% when pulling 1ce5e306b9619c89028efa24e7fafa9728ef3268 on chore_fix-typo into f3fdd3ca4019ec99e68cc55f3f1a0bbdeec0cba9 on master.

:hourglass: Testing commit 1ce5e30 with merge 668c857...

:sunny: Test successful - status


A few questions about your implementation

Hi @phuslu, very interesting work, but some things in your repository raise questions for me.
Throughput benchmarks

No need to run benchmarks on a very busy github runner, as its cores are forced to switch context frequently.
b.SetParallelism(32) is also unnecessary, since you run more goroutines than the number of available cores and the scheduler ends up spending a lot of CPU time switching between them.
Ristretto is configured incorrectly, which greatly affects its performance. You specify in its parameters MaxCost: 2 << 30, while specifying 1 as the cost of each item. Because of this, it will store 2 << 30 items, which is much more than one million. The correct configuration is as follows

cache, _ := ristretto.NewCache(&ristretto.Config{
    NumCounters: 10 * cachesize, // number of keys to track frequency of (10M).
    MaxCost:     cachesize,      // maximum cost of cache (1M).
    BufferItems: 64,             // number of keys per Get buffer.
})

Otter is also configured incorrectly, but it's not critical for now. But it will be soon!🙂
And now if you run the original benchmarks without runtime limit on 8 free perfomance cores with the command go test -run='^$' -cpu=8 -bench . -timeout=0 -benchmem
goos: darwin
goarch: arm64
pkg: github.com/maypok86/otter/bench
BenchmarkCloudflareGetSet-8     19329662                59.74 ns/op           16 B/op          1 allocs/op
BenchmarkEcacheGetSet-8         30167113                42.04 ns/op            2 B/op          0 allocs/op
BenchmarkLxzanGetSet-8          19655550                64.18 ns/op            0 B/op          0 allocs/op
BenchmarkFreelruGetSet-8        16393722                76.60 ns/op            0 B/op          0 allocs/op
BenchmarkRistrettoGetSet-8      13675732                91.52 ns/op           29 B/op          1 allocs/op
BenchmarkTheineGetSet-8          7859980               154.2 ns/op             0 B/op          0 allocs/op
BenchmarkOtterGetSet-8          19729608                70.02 ns/op            8 B/op          0 allocs/op
BenchmarkPhusluGetSet-8         23397532                52.11 ns/op            0 B/op          0 allocs/op
PASS
ok      github.com/maypok86/otter/bench 17.342s

After other fixes we get the following
goos: darwin
goarch: arm64
pkg: github.com/maypok86/otter/bench
BenchmarkCloudflareGetSet-8     18841335                61.23 ns/op           16 B/op          1 allocs/op
BenchmarkEcacheGetSet-8         28833619                39.66 ns/op            2 B/op          0 allocs/op
BenchmarkLxzanGetSet-8          24512806                50.89 ns/op            0 B/op          0 allocs/op
BenchmarkFreelruGetSet-8        14905558                79.83 ns/op            0 B/op          0 allocs/op
BenchmarkRistrettoGetSet-8      22861460                52.04 ns/op           28 B/op          1 allocs/op
BenchmarkTheineGetSet-8          7006892               171.7 ns/op             0 B/op          0 allocs/op
BenchmarkOtterGetSet-8          30895054                47.92 ns/op            7 B/op          0 allocs/op
BenchmarkPhusluGetSet-8         26106796                47.16 ns/op            0 B/op          0 allocs/op
PASS
ok      github.com/maypok86/otter/bench 17.371s

Here you should make sure that all implementations use the same number of shards, but I'm too lazy to do it anymore.

In benchmarks you use this construct i := int(fastrandn(cachesize)); i <= threshold. Do you really want to always write the same items and read the same items? Because that is exactly what you are doing.
Next are general questions about key distribution used in benchmarks. The main one is why a unique key distribution is used. It is completely unrealistic and distributes contention across shards, which hides implementation problems. And why the cache is only half full, do you really want to check the speed of cache misses, which are worthless unlike cache hits? You can read more here
And if you run more CPU cache-aware benchmarks the use of sync.Mutex will start to have an impact.

goos: darwin
goarch: arm64
pkg: github.com/maypok86/otter/perf
BenchmarkCache/Zipf_Theine_reads=100%,writes=0%-8               11608960               107.7 ns/op         9283558 ops/s               0 B/op          0 allocs/op
BenchmarkCache/Zipf_Ristretto_reads=100%,writes=0%-8            48676160                25.32 ns/op       39493565 ops/s              16 B/op          1 allocs/op
BenchmarkCache/Zipf_Phuslu_reads=100%,writes=0%-8               32948250                34.86 ns/op       28687550 ops/s               0 B/op          0 allocs/op
BenchmarkCache/Zipf_Otter_reads=100%,writes=0%-8                287808117                4.196 ns/op     238324807 ops/s               0 B/op          0 allocs/op
BenchmarkCache/Zipf_Theine_reads=75%,writes=25%-8               10141712               106.3 ns/op         9405869 ops/s               0 B/op          0 allocs/op
BenchmarkCache/Zipf_Ristretto_reads=75%,writes=25%-8            27029410                46.52 ns/op       21498335 ops/s              27 B/op          1 allocs/op
BenchmarkCache/Zipf_Phuslu_reads=75%,writes=25%-8               33284792                33.89 ns/op       29508730 ops/s               0 B/op          0 allocs/op
BenchmarkCache/Zipf_Otter_reads=75%,writes=25%-8                190370718                6.527 ns/op     153221398 ops/s               2 B/op          0 allocs/op
BenchmarkCache/Zipf_Theine_reads=50%,writes=50%-8               13183764                88.23 ns/op       11334143 ops/s               0 B/op          0 allocs/op
BenchmarkCache/Zipf_Ristretto_reads=50%,writes=50%-8            22789605                52.02 ns/op       19224501 ops/s              34 B/op          1 allocs/op
BenchmarkCache/Zipf_Phuslu_reads=50%,writes=50%-8               32335388                34.77 ns/op       28761246 ops/s               0 B/op          0 allocs/op
BenchmarkCache/Zipf_Otter_reads=50%,writes=50%-8                106489442               12.22 ns/op       81857267 ops/s               6 B/op          0 allocs/op
BenchmarkCache/Zipf_Theine_reads=25%,writes=75%-8               13120154                92.32 ns/op       10832209 ops/s               0 B/op          0 allocs/op
BenchmarkCache/Zipf_Ristretto_reads=25%,writes=75%-8            14231132                70.28 ns/op       14228276 ops/s              60 B/op          1 allocs/op
BenchmarkCache/Zipf_Phuslu_reads=25%,writes=75%-8               32800737                33.55 ns/op       29807980 ops/s               0 B/op          0 allocs/op
BenchmarkCache/Zipf_Otter_reads=25%,writes=75%-8                51792225                23.23 ns/op       43054563 ops/s              12 B/op          0 allocs/op
BenchmarkCache/Zipf_Theine_reads=0%,writes=100%-8               31050308                41.97 ns/op       23827702 ops/s               0 B/op          0 allocs/op
BenchmarkCache/Zipf_Ristretto_reads=0%,writes=100%-8            13311984               128.6 ns/op         7775102 ops/s             122 B/op          3 allocs/op
BenchmarkCache/Zipf_Phuslu_reads=0%,writes=100%-8               33116732                34.05 ns/op       29370837 ops/s               0 B/op          0 allocs/op
BenchmarkCache/Zipf_Otter_reads=0%,writes=100%-8                10602829               120.2 ns/op         8316114 ops/s              64 B/op          1 allocs/op
PASS
ok      github.com/maypok86/otter/perf  30.320s

Memory usage benchmarks

Ristretto is misconfigured again.
Apparently your not storing the key twice unlike other implementations, then using values taking up less space than keys doesn't look very fair (it saves about 15 MB)
Also I didn't find any data structure for deleting expired items. This for these benchmarks saves about 16 bytes per item or about 15MB in total, and will also in real life lead to much higher memory usage than the other implementations. Oh, and not being able to disable clock goroutine looks sad.
Comparing memory usage with ristretto, theine, otter doesn't look particularly fair, because due to cost based eviction support, the hash table grows many times.
Many implementations use a few extra bytes to increase hit ratio. For example, theine and otter should use about 8-16 bytes extra per element, but unfortunately this is 16-24 bytes due to alignment issues😔.

So your implementation does show good memory utilization, but after fixing the implementation and benchmarks the gap will not be so big.
General questions

Do you understand why your implementation is faster than ristretto? This is important because cutting the eviction policy is a very dangerous operation that will end badly in a large number of cases.
I tried to simulate the hit ratio of this implementation and found out some unpleasant subroutines. This one looks very questionable, because in real life you will store many more items than you need. Also, here are the results of the hit ratio simulation. Almost everywhere too many items are used, but the most confusing thing is the performance on S3, are there definitely no bugs?

Thanks for touching.
I corrected two obvious mistakes

ristretto config
shards count

For other points of you mentioned, I need do more study and thoughts.

Throughput benchmarks

Although github actions results is not very stable, but I think it's trustable. We could check all history results in https://github.com/phuslu/lru/actions/workflows/benchmark.yml , it show the benchmark variance is not big and the performance index is stable.
I set b.SetParallelism(32) intentionally and think it's reasonable, because I'd like to simulate a highly concurrent web handler.
corrected.
You mean I need call fastrandn(cachesize) again to get another index for reading/writing?
Half full is a initial state. And I suppose/assume the benchmark result will be same in different fullness

Memory usage benchmarks

Corrected.
I think this is phuslu/lru advantages because the the surface of benchmark code does not favor it (I think)
Yes, but this is a choose/tradeoff in each implementation.
Similar with 3.
Similar with 3.

General questions:

I didn't get your point well, you means the lackness of a active/background eviction is not acceptable?
Yes it's a known issue, I plan to try optimize the overuse by this trick  https://github.com/golang/go/blob/f719d5cffdb8298eff7a5ef533fe95290e8c869c/src/runtime/rand.go#L223 . And for bug, I cannot say it's bugfree, but this implementation is really simple, welcome code review&audit.

Oh, I think I got a bit your point
Q: Why choose to implement a fixed size cache without active/background eviction?
A: I'm a fan of fastcache philosophy, so I'd like to get a "generics" version alternative to fastcache with a simple LRU strategy

Throughput benchmarks
1 Maybe, but in the end the results of such runs don't inspire much confidence, especially since already now in github actions phuslu/lru loses to go-freelru. And in this case, it is absolutely incomprehensible how a hash table that is faster than ConсurrentHashMap from Java on this workload loses to maps with mutexes on the same workload.
2 We'd all like to test this, but the bottom line is that you can't execute more than the number of goroutines in parallel than the number of available CPU cores physically. Which makes you check the speed of the scheduler, although you still want to see the speed of the caches.
4 (and 5) I don't think I explained it well. Let's do it again. First, you will never insert an element with i > threshold. Threshold = cachesize / 10, and the cache would fill for all i < cachesize / 2. The end result is that all set operations are updates, and half of get operations are cache misses, which is pretty sad considering that cache misses are cheaper than cache hits, and with updates phuslu/lru just doesn't waste time maintaining the expiration policy. The second thing I've also run into is that using a distribution from unique keys for benchmarks is pretty bad at reflecting real-world behavior. Usually in real life the workload at least remotely resembles a zipf distribution with hot and cold elements. In your benchmarks, because of the unique key distribution, there is no difference between using sync.Mutex and sync.RWMutex, because the fastpath is always triggered. There is also a problem with fastrand, because in the first iteration you can have i = 16, and in the second i = 999_999 and so on, which kills the CPU cache. For most implementations this is not critical, because xadd operations in mutexes at contention also reset the cache line, but for otter it is critical and almost completely neutralizes all its advantage.

Memory benchmarks
2 Yes, it's an advantage, no one is even against that advantage. The only problem is that in your benchmarks keys take up more memory than values, which benefits phuslu/lru and otter, but in real life values usually take up more space than keys.
3 Ok, although that's pretty sad considering facebook and twitter's statements about expiration policies. Oh and memory usage comparisons don't look fair then.

General questions

No, what I meant was that under attack or with a cache capacity comparable to the number of shards it would result in a very small hit ratio. And this will cost a lot of system uptime.
To be honest, I wouldn't want to try to find a bug in the implementation. I'll just say that on S3 the behavior is very strange, the hit ratio didn't grow for a very long time and then skyrocketed 3 times, becoming more than the optimal value for this trace and cache capacity. With a high probability it indicates that there are too many elements in the cache. Why the hit ratio didn't grow, I didn't understand. I didn't really see how cheaprand would help solve this, but okay. The fastcache philosophy is great for offheap caches, but here you have to realize that the pointerless lru implementation is still in the heap.

Perhaps my questions may seem picky, but I really want to make better comparisons and learn something new. I've given up trying to make memory usage comparisons honest, but throughput benchmarks and hit ratio simulation I believe we can try to make them honest, although rounding cache sizes make that difficult too.

Throughput benchmarks
1 github actions is cheap(trigger on each push) and visible, I'd like to keep it.
2 I changed to concurrency to 8(same as the cpu number).
4 (and 5) Understood, let me try to fix it(read/write distribution) later. And for reason I choose "unique key test"(I call it randomly read/write) because it's easier to implemented and I think it's fair to major go lru caches. I agree that it's not fair/honest for RTO(Ristretto/Theine/Otter), let me add zipf benchmark later.

General questions

Every shards implementation has this defect/issue(And the low performance of zipf), but I think this is a choose/trade. shards is slow than RCU in zipf distribution but It has potential memory/gc benefits (if we implement it carefully).
althrough "the pointerless lru implementation is still in the heap", but thanks to "pointerless/continuous/compact", the most operation will be a "memcpy" instead of implicit memory allocation.

Many of the questions you raised are very insightful and let me learn/study.
One of motivation that I spend a lot of time to tweak/revise this implementation is: I'd like show that the traditional implementation (shards + lru) still have its value if we implement it carefully.

The initial motivation https://twitter.com/phuslu/status/1743236300977922502 and during the implementation I was greatly influenced by fastcache philosophy.

I checked your one of benchmark results here https://github.com/maypok86/benchmarks/blob/main/ratio/ratio.txt
the Ristretto results is very abnormal ， any reason/root cause?

Throughput benchmarks
1 This was actually the main reason why I came at all. Unfortunately the github actions don't look like cheap, at least because of the huge latency in benchmarks. Especially since I have a suspicion that you were given much less than 8 CPU cores, because for server CPUs those are very weak numbers.
2 It is not required and even harmful, as you can make sure of it just by opening the SetParallelism description
SetParallelism sets the number of goroutines used by RunParallel to p*GOMAXPROCS. There is usually no need to call SetParallelism for CPU-bound benchmarks. If p is less than 1, this call will have no effect.
4 (and 5) Yes, such a comparison for lru implementations makes sense in principle, but here it is the amount of work the implementation does that comes to the fore. And in the end it's the number of features that slow down the implementation but give more guarantees of convenience to users that decides everything. This is quite controversial, but okay, let's consider it a tradoff. Although as a user then I would like to know from the README about the lack of an expiration policy for example, because for ttl it is usually implied. To be honest, I don't really care about RTO performance on such benchmarks, challenging them was not part of my motivation for this issue. Especially since on 8 free cores their performance is much better. I was rather interested to know why people choose an unrealistic distribution that hides problems.

I checked your one of benchmark results here https://github.com/maypok86/benchmarks/blob/main/ratio/ratio.txt the Ristretto results is very abnormal, any reason/root cause?

This is a fairly well known issue, which was the main reason for the creation of otter. There are already many issues about ristretto's abysmal performance on their own hit ratio benchmarks (first and second). And the authors do not respond to it in any way unfortunately.  Even on such a simple zipf distribution the ristretto performance is abysmal.

Memory benchmarks
2 and 3 A little controversial, but basically okay.
General questions
1 The problem there isn't zipf workloads, it's RTOs spending time synchronizing a common eviction policy for all shards, but using some tricks to speed up reads. My main argument here was just that you have to be careful with cutting eviction policies, as it can be reflected in hit ratio. If this is known, then all is well.
2 Makes sense, I rather meant here that when storing strings for example, you will no longer achieve the compactness of fastcache and put a pressure on gc.

Throughput benchmarks

OKay, I partly agree with you but I really need a "cheap" x86_84 server to run and visualize these benchmarks. (background: my laptop is windows and my linux server is oracle cloud arm machine)
Disabled the SetParallelism
4 (and 5): Studying and re-writing zipf cases based on your benchmark suite

P.S. It would be cool if you tried running hit ratio tests for ristretto. Maybe me and the author of theine have developed schizophrenia😂, but I have never been able to replicate his hit ratio. Although the hit ratio simulators in ristretto, theine and otter show about the same results. And they correlate quite well with simulators from caffeine and libcachesim.

General questions

Thank you for pointing this out, I only vaguely know that there would be a cost because we get benefits by cutting off it so we need pay the price in somewhere.
I also believe the minor throughput gap between phuslu/lru and other traditional implementation (shard lru) comes from the compactness

For ristretto, yeah I will try verify it's hit rate. It is alarm for me and our projects.

4 (and 5) Studying and re-writing zipf cases based on your benchmark suite

Actually, it doesn't have to be. The only thing I was trying to point out is that in such benchmarks the amount of work under sync.Mutex makes a big difference and you won't even notice the difference compared to sync.RWMutex for example. And that it is worth specifying in README that there is no expiration policy, because usually specifying ttl implies that items with expired lifetime will be automatically removed from the cache by some algorithm. I was very surprised not to find this in phuslu/lru and go-freelru. I think it might be a surprise for users too. And I wanted to know the motivation to use random key distribution, but you explained it, so it's ok.

I add "limitation" in readme to explain the eviction policy
Added zipf benchmark results. But I'm not sure that I do it correctly.
Cloneing the ristretto hit ratios repo (git lfs), will update my results later.

Anyway, I decided to splurge and run benchmarks on a server processor.
Used configuration: AMD EPYC (3.4GHz), 32 GB RAM.
Golang version
root@v2310093:~# go version
go version go1.21.6 linux/amd64

Benchmarks were run with the following command: writeratio=10 go test -run='^$' -cpu=4,8,16,32 -bench . -count=5 -timeout=0 | tee bench.txt
The results are as follows.
Well, as I said before, in these benchmarks almost all implementations have approximately the same speed and the amount of work under shards decides very much, but when the number of available cores increases, contention-free implementations still come out ahead even in spite of more work they do.

Thank you very much
Generating zipf values is a very heavy operation, you can understand this just by opening the zipf.Uint64 code, but in idea I would like to see benchmarks with thread-local operation, which will increase contention, but again, to compare lru implementations it is probably really redundant.
Yay, I'm not crazy.🙃

Okay, I think we've pretty much covered everything, so that leaves one last question. I still don't get it, is increasing the cache size an expected behavior? (also applies to the author go-freelru) Because it seems very cheaterish. For example, in the hit ratio measurements phuslu/lru was running, for example, on a P8 trace with capacity = 400_000. But after transformations inside phuslu/lru this becomes much larger. For example, these measurements were run with GOMAXPROCS = 16, so the number of shards is 16 * 16 = 256. Now we calculate the size of each shard. 400_000 / 256 = 1562.5. The next degree of two is 11, i.e. the size of a shard is 2^11 = 2048 and the capacity of the whole cache = 2048 * 256 = 524_288. Exceeding the size by such a number of elements looks critical. All the more so, because of this the results of the hit ratio simulation are unfair. And in real life it will lead to much higher memory consumption.

With capacity = 600_000, real capacity = 1_048_576.
And on DS1 trace, where capacity is always more than a million, things get even worse. For example, when capacity = 3_000_000 the real size is 4_194_304 and so on.

Ah, I think I realized, cheaprand was meant as an example of the remainder-of-division trick.

I found cheaprand is work for my case, I guess it only work for rand cases [0, 2<<32)
I rip a fastmod POC here https://go.dev/play/p/cGJ6SCtWh0W base on bmkessler/fastdiv

I gave up trying to make hashtable size to fit the shard size. Just make the LRU nodes list to fit the needed shard size https://github.com/phuslu/lru/commit/5c217c3dec43ea879f373d2175df0f18858ee336

the real capability of my LRU is limited by the LRU "list" now, so its results in  hit ratio tests should no longer be cheating.
the extra/additional spaces of hashtable should be OK, because waste a bit []uint64(2 x uint32) is acceptable and it does not affect/impact the hit ratio tests.

after I run the benchmark in high end linux server with intel cpu, I found the results is more stable as your said and it looks like more  objective .
But I still thought a continuous&checkable online results is more acceptable to users than just pasting the oneshot&personal data to readme.

I gave up trying to make hashtable size to fit the shard size(fastmod is slow/broken during my implementing). Just make the LRU nodes list to fit the needed shard size https://github.com/phuslu/lru/commit/5c217c3dec43ea879f373d2175df0f18858ee336

It can still store a len(shards) more there, but that's already acceptable, thanks.
Oh, that's now fixed the behavior on S3 too, and now the behavior is almost exactly the same as the standard LRU (ended up being a slightly worse version of it).
As usual, here are the results.

Many thanks for these ( questions & guide & tests).
Now hopefully I can claim that I tried my best to work out a correct/effective/scaleable(?) LRU cache of golang, it matches the repository title, "High performance LRU cache" 😄

But I still thought a continuous&checkable online results is more acceptable to users than just pasting the oneshot&personal data to readme.

Okay, your solution and it even makes sense, but it's not even about stability. It's about the fact that there the distribution of results is very different (as I have already written it here) and there is almost no gap in favor of phuslu/lru even at 4 cores on the distribution of unique keys. And as the number of available cores increases, it starts to disappear and eventually at 16 cores phuslu/lru already starts to fall behind the others (there is very small difference with ecache at >= 8 cores). And I got the same results on m1 processor.


Post generate doesn't run on it's own

New Issue Checklist

[x] Updated struct to the latest version
[x] I have read the documentation to make sure my issue isn't covered already

Struct version
[2.1.1]
Spec version
[2.1.1]
Which type of spec are you using?

[x] project.yml
[ ] project.json
[ ] Specfile

Context
The post-generate script doesn't run unless the pre-generate script is also defined.
Reproduction steps
include the following in project.yml
scripts:
  post-generate: scripts/post-generate.rb

Expected result
scripts/post-generate.rb script is run
Actual result
Script isn't run. If pre-generate is added it does run. Interestingly, if pre-generate is on it's own it runs fine

Interesting! Thanks for filing this. I'll aim to get this fixed up for 2.2.0


Problem with saving in Arethusa

I am using the staging version with smyth3 tagset
When I deselect a morphological analysis and select a new one, it may happen that the new selection is not saved, but the old one is simply deleted (I saw this in the xml file). Unfortunately, this error seems to be unpredictable, in the sense that it does not apply to all changes.

I haven't been able to reproduce this but some more details from an email discussion with @gcelano
"This is another typical scenario:
I have a morphological analysis which is selected (because I first annotated that way). I select a different option among the ones proposed and I go ahead with another word...but when I get back to check if everything is fine, I do not find the option ticked and not even the lemma translation (i.e., blanck field), even though it seems that they are saved in the xml."
(I think the lemma translation problem was separate and is indeed fixed now)

A quick update:
The scenario is similar: Marcel annotated something. I corrected his morphological analysis (which were previously selected and saved by him): I simply selected a new morphological analysis and saved. What happened when I refreshed the browser was that no option was selected. I saw the XML and I found that the content of the attribute pos was empty. So I assume that when I selected and saved the new option, the system simply deleted the old selection but did not substitute it with the new one.

Hey @gcelano, I also have not been able to reproduce this. What do you mean with "Marcel annotated" something? Are you working on a file that's not your own?

I meant that he was annotating a normal file that we put in Arethusa. To
reproduce this you should select a morphological option save, refresh and
then try with changing the previous selection. I can tell that for some
strange reason this is sometimes possible but some other times it happens
that by modifying the previous morphological annotation the system simply
deletes the content of the pos attribute. This means that when you come
back/refresh the web page you find nothing selected. At that point you need
to select something again and this usually solves the problem.
2015-02-01 14:48 GMT+01:00 Gernot Höflechner<EMAIL_ADDRESS>
Hey @gcelano https://github.com/gcelano, I also have not been able to
reproduce this. What do you mean with "Marcel annotated" something? Are you
working on a file that's not your own?
—
Reply to this email directly or view it on GitHub
https://github.com/latin-language-toolkit/arethusa/issues/608#issuecomment-72365319
.

--
Dr. Giuseppe G. A. Celano
Postdoctoral Researcher
Universität Leipzig
Institute of Computer Science,
Section of
Digital

Humanities

Augustusplatz 10
04109 Leipzig
 (DE)
E-mail<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Web site 1: http://www.dh.uni-leipzig.de/wo/team/
Web site 2: https://sites.google.com/site/giuseppegacelano/

Is this happening with specific words only? Ideally you would have a very concrete example for me, because when I do the steps you just described, everything works fine for me :(

Unfortunately, when I find an example I cannot reproduce it anymore... and
as I was saying I have not been able to understand why this happens with
some words only, without a clear explanation :( But I will keep you
informed if I can say more about this.
2015-02-01 15:09 GMT+01:00 Gernot Höflechner<EMAIL_ADDRESS>
Is this happening with specific words only? Ideally you would have a very
concrete example for me, because when I do the steps you just described,
everything works fine for me :(
—
Reply to this email directly or view it on GitHub
https://github.com/latin-language-toolkit/arethusa/issues/608#issuecomment-72366112
.

--
Dr. Giuseppe G. A. Celano
Postdoctoral Researcher
Universität Leipzig
Institute of Computer Science,
Section of
Digital

Humanities

Augustusplatz 10
04109 Leipzig
 (DE)
E-mail<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Web site 1: http://www.dh.uni-leipzig.de/wo/team/
Web site 2: https://sites.google.com/site/giuseppegacelano/

This is the link I am working on : http://www.perseids.org/tools/arethusa/app/#/perseids?chunk=28&doc=9258
In the sentence there the 6th word (pole/mous) had a wrong morphological annotation. I selected the right annotation and I have not saved yet. If I go on with the annotation and then get back (without saving ever), the annotation of the word pole/mous is never in place (and in the xml I find an empty pos). I infer that when I first selected the right option the value of the attribute pos was deleted but was not replaced with the correct one. I know that if I save, refresh, and then select the right option, in the end it gets saved... but I need to do this for too many words...since I am correcting an old annotation. Is this of any help?

Continued problems reported by @gcelano from one of his students:
" I've been playing around with the Lemma Translations a little bit, and the problem does seem to be when you change the morphological tab.  The definitions weren't affected by me changing the syntactic or semantic tab.  Also, when you change the morphological tab the definition doesn't disappear right away, but only after you load a different page and then come back to it.  If, however, you simply save and refresh the page after changing the morphological tab, the definition will remain."

@Masoumeh this would be a good item to go through with @gcelano -- please ask him to show you the behavior

As I checked, the problem appears when the changes are made in both "Lemma Translation" and checkbox together such that first the "lemma translation" is changed and then the checkbox!!!
It is reflected just in saving the "Lemma Translation" in both GUI and XML with keeping the latest value and doesn't effect the checkbox.
This doesn't happen when one of the lemma or checkboxes is changed.
Also, refreshing the page doesn't seem to make a problem. The behavior is the same.

@Masoumeh do you think you can tackle fixing this?

@balmas I think there is a relation between the forms and the lemma translation. I will work on it and inform you how it goes further.

Hi @balmas,
How can I load the "smyth3" format locally?
I mean, how to open the setting with lemma translation in morph tab locally? Thanks.

Here are the steps to adding other formats. I think these should be added to the arethusa documentation but I'm not sure where they go really.
The path to auxiliary configuration files is set via the following arethusa configuration setting
main.auxConfPath
You can find the auxiliary configuration files themselves in the arethusa-configs repository.
You can clone and build them, and then copy them to wherever your auxConfPath points, or you could just point that at either the location they are currently deployed on Perseids
http://services.perseids.org/arethusa-configs
Or in GitHub, although I think it's not guaranteed right now that the github dist folder is completely up to date, as the process of building is not completely automated right now.
See related item #661

Actually, I don't know how to access the setting via URL.
After adding the config files you mentioned, the URL with staging config (i.e. http://localhost:8081/app/#/staging?doc=1) shows the old setting, even when I change the "format" attribute to "smyth3" in example data files (doc).
It happens that I can see other changes (like SG tab) when using some example files with format attribute changed to 'smyth3' (like smyth2.xml), but lemma translation doesn't appear.
I also tried to put the "perseids" config in the URL. But I get the access denied error.
Could you please explain how can I see "lemma translation" in a page locally? And why I get access denied error when using perseids config (should I add ssh key)? Thanks.

Hi @Masoumeh the triggers to enable the lemmaTranslation field for the morph plugin are:

the use of the morph3.html template
the gloss property set to true

E.g.
"morph" : {
      "retrievers" : {
        "BspMorphRetriever" : {
          "resource" : "morphologyService"
        }
      },
      "template"  : "templates/morph3.html",
      "contextMenu" : true,
      "contextMenuTemplate": "templates/arethusa.morph/context_menu.html",
      "lexicalInventory" : {
        "retriever" : {
          "LexicalInventoryRetriever" : {
            "resource" : "lexInvFusekiEndpoint"
          }
        }
      },
      "gloss" : true,
      "matchAll" : true,
      "@include" : "morph/gr_attributes2.json"
    },

If you set the auxConfPath in the staging.json as follows, this should work, because the live smyth3.json there does have the gloss attribute turned on
 "auxConfPath": "http://services.perseids.org/arethusa-configs",

If instead you built the config files locally, or used them from github, this might not work, because I think only the version of smyth3.json in the gcelano branch of arethusa-configs has gloss turned on right now, and what is in the GitHub dist directory might be built from master.  (We really need to merge all the branches in arethusa-configs back into master but that is another story...)
As for the access deined message when using perseids config, it's because it's trying to access a remote session on the perseids server, and you cannot do that locally due to the cors restrictions.

I think the problem was caused by "not" passing the variable "form"
as the second argument of the function "updateGloss". I create a pull request.

Fixed in ae7f5b0


Question: Do you please have a Vue2 compiled version that works with CDN / script tag?

Hi there
Is there a a compiled Vue2 version that one can use via the script tag / cdn?
Thank you so much!

Hi.
Sorry. Compiled script is not now.

This library was distributed at unpkg.com
Please try it.
<EMAIL_ADDRESS>


visualization

Hi, the code can visualize the RGB images. May I know how to visualize the multispectral images from this datasets?

See this issue.


How to convert collected libero trajs to RLDS format?

Thanks for your excellent work!
I follow the instruction in Dataset part and have collected the success/fail trajs in Libero.
But I find that the trajs are npy and the RLDS format is needed to finetune the openvla.
Can you give me some advice to convert the npy to RLDS format?
Best

Hi Haozhan! The next step is to use dataset construction scripts to convert npy to RLDS, which I mentioned in #3. We will release it ASAP.


Chromium

Fixes #2

[x] Need to handle properly dev chromedriver

I fixed composer 2 compatibility in main, so if you merge it in the CI should work again.
PR is looking good! Thanks for the effort 👍

Changes done and branch updated with main one

Thanks @laurent35240


Additional repository setup

Add make unit|images|push tasks, dockerfiles and a task for verifying vendor content

@elmiko I copied envtest setup from cccmo

Hm... @alexander-demichev can you please explain, why you want two different dockerfiles there?

Hm... @alexander-demichev can you please explain, why you want two different dockerfiles there?

iiuc, the Dockerfile allows community build, whereas Dockerfile.rhel is restricted to people with RHEL subscription or our CI systems.

@elmiko you are right, dockerfiles reference different images. One is publicly available for anyone outside of red hat, second is rhel based

/approve


Fix form allignment in mobile view.

This is a Bug Report

Problem:
The alignment of the form is not proper.

Proposed Solution:
The alignment of the form should be proper.
Page to Update:
https://kubernetes.io/...

/kind cleanup
/priority backlog

Fixed the issue.


Remove some unused codes

Remove some unused fields in module instance and the related codes so as
to reduce the footprint, which are introduced by emsdk non-standalone
mode (e.g. -DSIDE_MODULE=1) and are not required now.

LGTM


fairseq-generate doesn't generate all input sentences.

❓ Questions and Help
What is your question?
I trained mBART below instruction. (but datas are differ from this page)
(https://github.com/facebookresearch/fairseq/blob/main/examples/mbart/README.md)
Then I tried to generate Japanese sentences from English, but mBART doesn't generate all input sentences like following image.
(Input sentences are 1,159 but generated senteces are 1,152)

I have no idea which part is wrong, does anyone have any idea?
Code
fairseq-generate ./data-bin/jparacrawl/${NAME_LIST[i]} \
   --path checkpoints/jparacrawl/checkpoint_best.pt --beam 5 \
   --batch-size 512 \
   --langs $langs --truncate-source \
   -t tgt -s src \
   --task translation_from_pretrained_bart \
   --gen-subset test \
   > ./en-ja.ja_pred

What have you tried?

fairseq-interactive doesn't support $langs option, but mBART needs to $langs option. So I think if want mBART to generate sentences, I have to use fairseq-generate.
I checked there is no empty line in input sentences.

What's your environment?

fairseq Version (e.g., 1.0 or main): latest
PyTorch Version (e.g., 1.0):1.13.1
OS (e.g., Linux):Linux
How you installed fairseq (pip, source): git clone
Python version:3.7.15
CUDA/cuDNN version:11.3

It fixed by changing fairseq version 0.12.2 -> 0.10.2


Added ListenIP to server configuration

SofEtherVPN now can be binded to specific IP address, by default it listen in <IP_ADDRESS>.
In server configuration, section ServerConfiguration will find parameter ListenIP.
This work is based on macvk, almost two year without news about him and i thinks this feature is very important to the project to be forgotten, thanks macvk.

Does this work for V6 interfaces as well?
More importantly can the Administrative login be bound to a specific IP
whilst others are not?
On 11 June 2017 at 17:03, Maikel Ortega Hernández<EMAIL_ADDRESS>wrote:

SofEtherVPN now can be binded to specific IP address, by default it listen
in <IP_ADDRESS>.
This work is based on macvk, almost two year without news about him and i
thinks this feature is very important to the project to be forgotten,
thanks macvk.
You can view, comment on, or merge this pull request online at:
https://github.com/SoftEtherVPN/SoftEtherVPN/pull/331
Commit Summary

Ãadded parameter ListenIP to server configuration
added parameter ListenIP to server configuration
Merge branch 'master' of https://github.com/maikelohcfg/SoftEtherVPN

File Changes

M src/Cedar/IPsec.c
https://github.com/SoftEtherVPN/SoftEtherVPN/pull/331/files#diff-0
(4)
M src/Cedar/Interop_OpenVPN.c
https://github.com/SoftEtherVPN/SoftEtherVPN/pull/331/files#diff-1
(9)
M src/Cedar/Interop_OpenVPN.h
https://github.com/SoftEtherVPN/SoftEtherVPN/pull/331/files#diff-2
(2)
M src/Cedar/Listener.c
https://github.com/SoftEtherVPN/SoftEtherVPN/pull/331/files#diff-3
(13)
M src/Cedar/Logging.c
https://github.com/SoftEtherVPN/SoftEtherVPN/pull/331/files#diff-4
(4)
M src/Cedar/Logging.h
https://github.com/SoftEtherVPN/SoftEtherVPN/pull/331/files#diff-5
(2)
M src/Cedar/Server.c
https://github.com/SoftEtherVPN/SoftEtherVPN/pull/331/files#diff-6
(13)
M src/Cedar/Server.h
https://github.com/SoftEtherVPN/SoftEtherVPN/pull/331/files#diff-7
(2)
M src/Mayaqua/Network.c
https://github.com/SoftEtherVPN/SoftEtherVPN/pull/331/files#diff-8
(42)
M src/Mayaqua/Network.h
https://github.com/SoftEtherVPN/SoftEtherVPN/pull/331/files#diff-9
(12)

Patch Links:

https://github.com/SoftEtherVPN/SoftEtherVPN/pull/331.patch
https://github.com/SoftEtherVPN/SoftEtherVPN/pull/331.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/SoftEtherVPN/SoftEtherVPN/pull/331, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AARerydYQHFetiOGVpcZEuX-unsbEkt8ks5sC3UjgaJpZM4N2TBC
.

Ipv4 only for now, later i can try ipv6

this option is already added two years ago. see here for details https://github.com/SoftEtherVPN/SoftEtherVPN/pull/202

please stop copy-paste my code. this looks like plagiarism and this is plagiarism

macvk sorry the miss understanding if you see my pull request note i always mentioned this your job
"This work is based on macvk, almost two year without news about him and i thinks this feature is very important to the project to be forgotten, thanks macvk."
as i wrote two years has past without any news from you i dont want credit for this, all the credit is for you. i dont want plagiarize your job at all.

maikelohcfg right solution for that conflict is DELETING "your code" from repository and leave my code.

Seriusly i dont expect this will make you so upset, because i always say this is your job, but i will not explain me anymore. For the good sake of the community i will close my repository  but i will like that you dont wait two years more for reply a question that allowed to merge your code in important projects like this making think every one that you are not Interested in that code. For me is end of toppic.

The right way to do this is to form the Repo from GitHub. Then create a new
fork of your own then create a all in one patch on your branch from The
original one. Then submit the all in one patch as a pull request to
upstream. This will keep the originators commit logs and author intact and
you can submit it as a pull request.
On 25 Aug. 2017 11:26 pm, "Maikel Ortega Hernández" <
<EMAIL_ADDRESS>wrote:
Closed #331 https://github.com/SoftEtherVPN/SoftEtherVPN/pull/331.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/SoftEtherVPN/SoftEtherVPN/pull/331#event-1221568842,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AARer01VQ2pOFE_DagCQlq2N44IEG_eRks5sbtmcgaJpZM4N2TBC
.

The right way to do this is to form the Repo from GitHub. Then create a new
fork of your own then create a all in one patch on your branch from The
original one. Then submit the all in one patch as a pull request to
upstream. This will keep the originators commit logs and author intact and
you can submit it as a pull request.

thanks i will have in mind for future.


External calls to the print API ？？

Attach (recommended) or Link to PDF file here:
Configuration:

Web browser and its version:
Operating system and its version:
PDF.js version:
Is a browser extension:

Steps to reproduce the problem:
1.Does PDf.js provide an external printing function that can call the API
2.
What is the expected behavior? (add screenshot)
What went wrong? (add screenshot)
Link to a viewer (if hosted on a site other than mozilla.github.io/pdf.js or as Firefox/Chrome extension):

No, you'll have to implement this in a custom deployment.


Update Chinese translation

@vickyhella can you rebase your PR with the latest on master branch? I don't think it will pass the CI/CD gates (failing at e2e tests step) and without all checks passed we cannot merge. 🙏 Thanks!

Already rebased and force-pushed. Praying for the checks to pass. 🙏


add memcached for did cache to relay

tired of waiting for the did cache to warm up when relay restarts? wait no longer! put the cache in memcached (still also have local in-process cache, but backed by "L2" cache in memcached)

memcached is configured when starting that service; the L1 DID cache is set by --did-cache-size (default 5 million entries)

just for visiblity, we have some pretty similar code for caching DID and handle resolutions in the atproto/identity package (in redisdir sub-package): https://github.com/bluesky-social/indigo/tree/main/atproto/identity
that was written after relay was written, so relay code doesn't use it. would be good to get it migrated over though. if there are specific features needed in the identity package (eg, bypass handle resolution, because relay doesn't care about that), we can add them


feat: invalid state on text inputs

:tada: This PR is included in version 5.0.0-next.7 :tada:
The release is available on:

npm package (@next dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.0.0-next.1 :tada:
The release is available on:

npm package (@next dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


update adunit when the video grows or shrink when playing an ad

In responsive full videos, or in phones when the viewport is rotated, we need to update the vpaid ad's size

we could listen simply to window resize or/and orientation change
or we want something more flexible maybe using this:
https://github.com/sdecima/javascript-detect-element-resize
or this:
http://www.backalleycoder.com/2013/03/18/cross-browser-event-based-element-resize-detection/

I would listen to the 'resize' and 'orientationchange' events that is probably the most common use.

but in cases the video is resized in by script's maybe is better to create some api that allow's the user maybe to control this instead of we assuming what the user's wants.

we could listen to a custom player event like 'resize.ads' and update the vpaid ad's size.
I would still listen to the 'orientationchange' and 'resize' events though.

I think that event is too generic, maybe we should prefix with 'vast' or something like that.
And if we listen to this events we should at least ad some flag in the configuration of the plugin to disable if for some reason it needs a different implementation or they don't need the plugin to resize.

how about 'vast.resize'??
Regarding to the configuration option, it makes sense.

ok I will implement like that


Add DesiredStateChange Event Filter

Description of your changes
Added the DesiredStateChanged() Event Filter to avoid unnecessary reconciliations
I have:

[X] Run make reviewable to ensure this PR is ready for review.

How has this code been tested
Executed this code in a development cluster running multiple deployments

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Throws error after updating to the latest version <IP_ADDRESS> and <IP_ADDRESS>

Question
I am receiving the following error after updating to the versions higher than <IP_ADDRESS>.
chart._lines[0].update(sma_tick)
File "/home/anil/pyprojects/trado/venv/lib/python3.11/site-packages/lightweight_charts/abstract.py", line 236, in update
if self._last_bar and series['time'] != self._last_bar['time']:
Please Help.
Code example
No response

@anilwarbhe I believe you need to update your pandas lib, I faced the same problem but fixed it with pip install --upgrade pandas
this will cause you to fix your pandas implementation, in some parts if the pandas version is not the lates
regards,

Pandas library was of already a latest version. But still tried reinstalling but still receving the same error.
File "/home/anil/pyprojects/trado/venv/lib/python3.11/site-packages/lightweight_charts/abstract.py", line 236, in update
if self._last_bar and series['time'] != self._last_bar['time']:
File "/home/anil/pyprojects/trado/venv/lib/python3.11/site-packages/pandas/core/generic.py", line 1519, in nonzero
raise ValueError(
ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

Your pandas implementation has one problem, are you using df.at ? To select values or how are you selecting values ? You must use df.Olof to avoid this problem of yours

Hey @anilwarbhe,
Appeared to be an ambigous check issue; should be fixed in the latest hotfix.
Louis

I am using iloc to access the values. I checked out the data types and its'
self._last_bar['time']
1698796800
<class 'numpy.int64'>
series['time']
1698796800.0
self._last_bar['time']

Thank you so much for the hotfix!


Ft nested doe

Create class NestedLHS

Add test test_nested_lhs in test_mfk.py

run_mfk_example edit to use the new class NestedLHS

:x: Build SMT 0.4.599 failed (commit https://github.com/SMTorg/SMT/commit/1a4c2117a7 by @Flo-code)

:white_check_mark: Build SMT 0.4.601 completed (commit https://github.com/SMTorg/SMT/commit/56bd363631 by @relf)


docker start.sh: allow to provide gunicorn.conf from env

just setting interface and port is not enough

Note, you can set the environment variable GUNICORN_CMD_ARGS that allow you to pass additional command line options to gunicorn. It is not the same as a configuration file, but it allows for customizability.

Wrt readability I would prefer the config file

merged as #278


Allow client certs to be configurable

Our syslog endpoint requires a client cert. This PR adds optional client cert/key properties.

We're still open to this (or a similar) PR, with the above-discussed changes. However, it's been a few months with no activity, so we're closing for now.


Create light font weight variant

Current lines are 3px-thin, which can be considered as a bold font weight. The light font weight variant can be created using 1px-thin lines.

fixed in ded16b479ea00869d734c193d3b2f98effa90912 and 940ec399c06416915787370f83cee47b27a795a1


Add requirements about Storage account firewall

As per this tutorial, storage account firewall needs to be open to all networks- https://learn.microsoft.com/en-us/azure/cost-management-billing/costs/tutorial-export-acm-data?tabs=azure-portal
Please add this pre-requisite as well as other storage account requirements in this page also to be consistent and avoid confusion.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 42794f22-7d47-22a7-6a0d-b0c350a00fbb
Version Independent ID: 123230ab-8682-221d-1c80-cd267996bd66
Content: Export cost data with an Azure Storage account SAS key - Microsoft Cost Management
Content Source: articles/cost-management-billing/costs/export-cost-data-storage-account-sas-key.md
Service: cost-management-billing
Sub-service: cost-management
GitHub Login: @bandersmsft
Microsoft Alias: banders

@haider392
Hi there. Thanks for the tip. I've updated the article as you suggested.
https://github.com/MicrosoftDocs/azure-docs-pr/pull/240737
#please-close

@bandersmsft
Thanks for your quick response.

@haider392
It looks like your issue is resolved, this thread is closed.
If there are further questions regarding this matter, please tag me in a comment. I will reopen it and we will gladly continue the discussion.


Suggestion: RS API Extension - Network Energy Core

Hi,
I'm currently thinking about an Extension Mod for RefinedStorage. The basis for the idea is the ability to make refined storage significantly heavier in terms of energy consumption, both in early and in late game.
I already have some ideas for this, but I fail because of the fact that I'm struggling with the access of the energystorage of the refined storage network. At the Moment i can only access through Reflection, which would be a total impact on performance.
So I would suggest an extension of the RS API that makes it possible
e.g. place additional energy cells in the network and the sum then gives the network energy capacity. 100000 RF controller + 2x 50,000 RF power cell = 200000 total capacity.
The ability to consume more energy sources via connector blocks, several blocks away from the controller block.
And many more Ideas about this topic.
If you would agree, I would take on the development and send you a PR upon completion.

I think it's a good idea.
If you do want to do this however, keep in mind:

a simple getter that returns an EnergyStorage object isn't sufficient;
the extension point needs to be able to hook into the point where Refined Storage machines extract energy (perhaps remove the INetworkNode#getEnergyUsage method and make every INetworkNode#update call a method in INetwork (something like INetwork#extractEnergy?)

A simple getter is definitely not enough, I have here something like a proxy per node in mind.
Because the controller behaves quite differently than a power connector or a power cell.
In addition, it is completely open at this time, whether an energy cell receives at all the ability CapabilityEnergy.ENERGY.
The idea of removing the INetworkNode#getEnergyUsage method and replacing it with a slightly more complex logic is not bad.
This would make it possible to differentiate between active and passive energy consumers.
For example: An importer passively consumes x RF / t and actively y RF per item.

I initiated a little poll on Twitter to check if we should maybe move to active energy only.
https://twitter.com/vdbergeraoul/status/1009187099361366018

Have already voted. But the option "Configurable per node type" would be my current wish option.
This would allow to configure some basic consumption for the network and at the same time direct consumption spikes for massive operations such as Quarry Imports or Disk Manipulator.
This would significantly affect the gameplay depending on the modpack.


cargo: shuffle deps to use workspace.dependencies

[ ] Added an entry to CHANGELOG.md if this change could be valuable to users

⚠ DO NOT MERGE ⚠

Codecov Report

Merging #1940 (159ab40) into master (e304826) will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master   #1940   +/-   ##
======================================
  Coverage    8.89%   8.89%
======================================
  Files         133     133
  Lines       57991   57991
======================================
  Hits         5160    5160
  Misses      52831   52831

Impacted Files
Coverage Δ

lapce-data/src/svg.rs
0.00% <ø> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

textwrap and dyn-clone are transitive deps, they got updated by cargo but the bump is small and I don't want to deal with cargo anymore so I leave them like that
moved all shared deps under workspace.dependencies, now we can control version for all of them in single place, it also revealed few inconsistencies, fixed in this pr
I also removed few properties under package and set to inherit them from workspace.package

I also replaced all branch/versions for git submodules in tree-sitter grammars to locked revisions, otherwise it often is bumped accidentally by people who try to add new grammar and I have to fix it for them
(and because cargo updated them for me too, grrr...)

workspace.dependencies is a rust/cargo 1.64 feature, on which we already depend, so no rust-version bump needed

locked tree-sitter grammars to specific revision should also prevent our builds fail when someone force pushes the branch


Sensor.Close calls SDL_SensorGetNonPortableType

The implementation of Sensor.Close calls SDL_SensorGetNonPortableType. It should probably be calling SDL_SensorClose right?
This is the same thing that happened in #377. I am currently going through your whole API, removing the CGo from the Windows port by loading the DLL and calling it directly rather than going through the C layer. I am almost done and will be opening a large pull request soon. Maybe I find some more of these along the way :-)

Thanks for catching this! I've fixed it in master branch.

OK, closing this as well.


Unexpected character

I tried to view a warc file just now with openwayback and it outputs the following. Is this a problem with the warc or with httrack2warc?
WARNING: Bad Record. Trying skip (Record start 782): Unexpected character 41(Expecting d)
Mar 02, 2020 10:47:38 AM org.archive.wayback.resourcestore.indexer.IndexWorker doWork
SEVERE: FAILED to index or upload (crawl.warc)
java.lang.RuntimeException: After retry (Offset 782)
	at org.archive.io.ArchiveReader$ArchiveRecordIterator.next(ArchiveReader.java:512)
	at org.archive.io.ArchiveReader$ArchiveRecordIterator.next(ArchiveReader.java:436)
	at org.archive.wayback.resourcestore.indexer.ArchiveReaderCloseableIterator.next(ArchiveReaderCloseableIterator.java:40)
	at org.archive.wayback.resourcestore.indexer.ArchiveReaderCloseableIterator.next(ArchiveReaderCloseableIterator.java:29)
	at org.archive.wayback.util.AdaptedIterator.hasNext(AdaptedIterator.java:56)
	at org.archive.wayback.util.AdaptedIterator.hasNext(AdaptedIterator.java:55)
	at org.archive.wayback.util.AdaptedIterator.hasNext(AdaptedIterator.java:55)
	at org.archive.wayback.resourceindex.updater.IndexClient.addSearchResults(IndexClient.java:158)
	at org.archive.wayback.resourcestore.indexer.IndexWorker.doWork(IndexWorker.java:111)
	at org.archive.wayback.resourcestore.indexer.IndexWorker$WorkerThread.run(IndexWorker.java:244)
Caused by: java.io.IOException: Unexpected character 43(Expecting d)
	at org.archive.io.warc.WARCReader.readExpectedChar(WARCReader.java:80)
	at org.archive.io.warc.WARCReader.gotoEOR(WARCReader.java:68)
	at org.archive.io.ArchiveReader.cleanupCurrentRecord(ArchiveReader.java:176)
	at org.archive.io.ArchiveReader.get(ArchiveReader.java:144)
	at org.archive.io.ArchiveReader$ArchiveRecordIterator.innerNext(ArchiveReader.java:562)
	at org.archive.io.ArchiveReader$ArchiveRecordIterator.exceptionNext(ArchiveReader.java:537)
	at org.archive.io.ArchiveReader$ArchiveRecordIterator.next(ArchiveReader.java:505)
	... 9 more

The command I used to download the website:
httrack "https://web.archive.org/web/20180611033123/https://github.com/adlio/usgs-waterdata/tree-commit/89c97a80cdd6fba90972fd137fcd5a7a92ad1fff" '-*' '+https://web.archive.org/web/20180611033123*' '+https://archive.org/includes*' '+https://web.archive.org/_static*' '+https://archive.org/images*' '+https://archive.org/services*' '+https://archive.org/components*' '+https://www.archiveteam.org*' -N1005 --advanced-progressinfo --can-go-up-and-down --display --keep-alive --mirror --robots=0 --user-agent='Mozilla/5.0 (X11;U; Linux i686; en-GB; rv:1.9.1) Gecko/20090624 Ubuntu/9.04 (jaunty) Firefox/3.5' --verbose

The command I used to create the warc:
java -jar /Users/fabiansturm/Documents/projects/httrack2warc/target/httrack2warc-0.4.0-shaded.jar /var/folders/rw/35q09zqj5yv5pz4wwg3yjfkm0000gn/T/webcache-download731331670 -o /var/folders/rw/35q09zqj5yv5pz4wwg3yjfkm0000gn/T/http2warc115706301 -C none

After that I renamed it to crawl.warc since I used -C none.
To run the container:
docker pull iipc/openwayback
docker container run -it --rm -v /tmp/owb:/data -p 8089:8080 iipc/openwayback

This is a bug in httrack2warc, I should have a fix shortly.

Fix released as v0.5.0. Thanks for reporting this. I missed it because I'd only been testing with Pywb which has a rather forgiving parser. I've tested the fixed version with pywb cdx-indxer, openwayback cdx-indexer, jwat and jwarc and all appear to have no complaints about the generated WARCs.


Add a way for worker subclasses to handle settings changes

Continuing off of https://github.com/ManageIQ/manageiq/pull/22196

Non-rails workers get their settings via STDIN on startup. This means there is no easy way to "reload" the worker settings after startup. As settings don't change that frequently we can just restart the worker if settings have changed.
TODO:

[X] Use Vmdb::Settings.changes or diff_to_deltas and compare with worker_settings_paths to see if any settings that the worker cares about changed

Ref:

https://github.com/ManageIQ/manageiq/issues/21992

@miq-bot assign @agrare
@miq-bot add_labels enhancement, core/workers, core

@miq-bot add_labels enhancement, core/workers

@miq-bot add_reviewer @kbrock

@agrare Reworked this PR based on feedback, ready for another look


Have trouble to find which paper correspond to which code?

Thank you @jiweil

https://vmobile.douyu.com/show/o8KxjMdxXroMVLwb


COMMONSRDF-54: overloads of RDF4J#asRDFTerm

Resolves: COMMONSRDF-54
This PR adds three overloaded methods into the class RDF4J:
RDF4JBlankNode asRDFTerm(final org.eclipse.rdf4j.model.BNode value)
RDF4JLiteral asRDFTerm(final org.eclipse.rdf4j.model.Literal value)
RDF4JIRI asRDFTerm(final org.eclipse.rdf4j.model.IRI value)

Thanks, this looks sensible to me!
Should we add unit test for each of the new asRDFTerm methods? They are in a way already tested through createLiteral etc.., but just so we don't break it later?

I added  a test caseRDF4JMethodOverloadsTest and one more method in RDF4J
public RDF4JBlankNodeOrIRI asRDFTerm(final org.eclipse.rdf4j.model.Resource value)


Implement an extensible card deck.

Implement a Deck with Cards. This should create a framework by which any arbitrary Card implementation could be used. A canonical implementation with normal playing cards should also be provided.

:white_check_mark: Dev complete; unit tests passed.


Question

How do i compile? seriously idk how do i just use an online tool

download visual studio, open the file and compile and run :)


Migration of db fails when upgrading to 2.19.4 when using network file system for data volume

Before you start please confirm the following.

[X] Yes, I've searched similar issues on GitHub.
[X] Yes, I've checked whether this issue is covered in the Portainer documentation or knowledge base.

Problem Description
Error when deploying 2.19.4 CE docker image. Failed migration | error="while backing up database: read /data/portainer.db: permission denied"
I am able to rollback to the image 2.19.4 CE and access the db fine and container starts. I had this issue previously here: https://github.com/portainer/portainer/issues/7144
Expected Behavior
Able to apply latest version.
Actual Behavior
Error when migrating.
Steps to Reproduce
Pulled new image (2.19.4) using docker-compose and started container.
Portainer logs or screenshots
2023/12/07 12:45AM INF github.com/portainer/portainer/api/cmd/portainer/main.go:369 > encryption key file not present | filename=portainer
2023/12/07 12:45AM INF github.com/portainer/portainer/api/cmd/portainer/main.go:392 > proceeding without encryption key |
2023/12/07 12:45AM INF github.com/portainer/portainer/api/database/boltdb/db.go:125 > loading PortainerDB | filename=portainer.db
2023/12/07 12:45AM INF github.com/portainer/portainer/api/datastore/backup.go:16 > Backing up database | from=/data/portainer.db to=/data/backups/portainer.db.bak
2023/12/07 12:45AM WRN github.com/portainer/portainer/api/datastore/backup.go:19 > failed to create backup file | error="read /data/portainer.db: permission denied"
2023/12/07 12:45AM FTL github.com/portainer/portainer/api/cmd/portainer/main.go:142 > failed migration | error="while backing up database: read /data/portainer.db: permission denied" stack_trace=[{"func":"(*Store).MigrateData","line":"45","source":"migrate_data.go"},{"func":"initDataStore","line":"140","source":"main.go"},{"func":"buildServer","line":"395","source":"main.go"},{"func":"main","line":"640","source":"main.go"},{"func":"main","line":"250","source":"proc.go"},{"func":"goexit","line":"1598","source":"asm_amd64.s"}]

Portainer version
2.19.4
Portainer Edition
Community Edition (CE)
Platform and Version
24.0.5, build ced0996
OS and Architecture
Ubuntu 22.04 AMD64
Browser
No response
What command did you use to deploy Portainer?
services:
  portainer:
    container_name: portainer
    image: portainer/portainer-ce:latest
    ports:
      - "$PORTAINER_PORT:9000/tcp"
    volumes:
      - portainer_data:/data

volumes:
  portainer_data:
    driver_opts:
      o: "username=xyz,password=xyz"
      type: "cifs"
      device: \\<IP_ADDRESS>\portainer\data

Additional Information
No response

Thanks for opening this as a separate issue - our team are looking into this and we'll update when we learn more.

Having similar trouble. Can't fix, because Docker image for 2.19.4 seems to be gone:
https://hub.docker.com/r/portainer/portainer-ce/tags
https://hub.docker.com/r/portainer/portainer-ee/tags
2.19.3 cannot be used because database is already on 2.19.4.

same issue

I fixed the missing 2.19.4 tag by doing a database rollback to 2.19.3:

In the stack file, revert Docker tag to 2.19.3
cp backups/portainer.db.bak portainer.db
Restart Portainer

I don't know what's going on here, but removing an existing Docker tag seems like bad practice to me.

Having the same trouble starting of today.
portainer  | 2023/12/07 08:35AM INF github.com/portainer/portainer/api/cmd/portainer/main.go:369 > encryption key file not present | filename=portainer
portainer  | 2023/12/07 08:35AM INF github.com/portainer/portainer/api/cmd/portainer/main.go:392 > proceeding without encryption key |
portainer  | 2023/12/07 08:35AM INF github.com/portainer/portainer/api/database/boltdb/db.go:125 > loading PortainerDB | filename=portainer.db
portainer  | 2023/12/07 08:35AM FTL github.com/portainer/portainer/api/cmd/portainer/main.go:403 > The database schema version does not align with the server version. Please consider reverting to the previous server version or addressing the database migration issue. |

We have now pushed a new version of the 2.19.4 image that fixes this issue. As part of this we removed the 2.19.4 image temporarily and re-tagged latest to 2.19.3 while we worked on it, and it looks like this caused other issues in the interim. Thanks for your patience while this got sorted out!

I don't have a copy of the DB to rollback to 2.19.3... should I just wait for the new 2.19.4 to pop out again? (using BE edition)

The new version is out, as I mentioned above. Re-pull the image from Docker Hub and you're good to redeploy.

I'm using command line docker pull portainer/portainer-ce:latest and it updated, but I'm getting same error after docker start.  Is there something else I need to do to fix this? Thanks

I'm using command line docker pull portainer/portainer-ce:latest and it updated, but I'm getting same error after docker start. Is there something else I need to do to fix this? Thanks

I too am still having issues.
Just ran:
sudo docker pull portainer/portainer-ce
docker stop portainer
docker start portainer

Still getting this error in container logs:
2023/12/07 02:09PM FTL github.com/portainer/portainer/api/cmd/portainer/main.go:403 > The database schema version does not align with the server version. Please consider reverting to the previous server version or addressing the database migration issue. |

Just wanted to update that completely removing the portainer container and recreating using docker a compose file resolved my issue.
docker rm portainer
docker compose up -d

Ok what if I don't have my docker compose file?  Can I pull without removing my Portainer configuration?

they marked this as completed, but I don't see a new release on the latest tag.
https://hub.docker.com/r/portainer/portainer-ce/tags?page=1&name=latest

Had to come looking for this. My Portainer was left in broken state.


wrong count of CPU cores on Torque clusters

on compute canada, when launching on 5 nodes with 12 cores each, the number of cores seen by SCT  is 12 instead of 5x12=60.

will be solved by refactoring the job submission process within SCT: will send one job per core instead of using Python pool with multiproc


Image gallery

Hi Paul,
Thanks for a great library.
I was wondering if there are any guidelines or recommendations on how to display a set of images with some navigation. I know that many lightboxes often have some support for these use cases, I was wondering if there's some way of extending luminous to do that or if there are any future plans for that feature.

Luminous doesn't currently have image gallery support, but I'm certainly not opposed to adding it. I'm not quite sure offhand yet when I'll be able to fit this in. If you wanted to take a stab at it, I'd also happily look at a PR. Feel free to bounce ideas off me here!

Hi Paul, I would also be very interested in this feature 👍
Simple next, prev. buttons to flip through all images with the luminous class would be great.

Hey @tompazourek  and @alexv9. I've just added this in 0.3.0 with the new LuminousGallery class. It should be documented pretty well in the readme, but let me know if anything needs clarification!


Add ranges for warnings

https://github.com/stylelint/stylelint/blob/main/CHANGELOG.md#1470

This is automatically infered from string length.


Port to iOS CoreML

THANKS for the project! I am currently doing a project that tracks face in mobile AR and surprised to see your face 3d reconstruction paper. I hope I can use 'vrn' as part of my face tracking. Would you appreciate I port the torch model you've trained to the CoreML format in iOS?

Hey!
I know absolutely nothing about CoreML. The model is very large but if you succeed I'd be happy to hear about it.
Thanks.

@ssaichixbg : did you manage to successfully port vrn to CoreML? Would appreciate to learn about your experience on this.


(v3.0.0-rc) defaults are overriding chart configuration

Expected Behavior
I am setting some defaults (both scripted and static) for a few parameters, but for one particular chart I want to override them. I expect the default values will not be used for a specific chart if I add explicit values in the chart configuration.
Current Behavior
The chart dataset configuration seems to be ignored, and the defaults are used. If I remove the defaults, the chart configuration is applied.
Possible Solution
not sure, but possibly related to changes done in https://github.com/chartjs/Chart.js/pull/8374 ?
Steps to Reproduce
https://codepen.io/santa-85/pen/BaQgqYe
Environment

Chart.js version: since 3.0.0-beta.13
Browser name and version:
Link to your project:

A generic backgroundColor is not overriding a type specific pointBackgroundColor. It is working as designed.
To get that use case working, you should define the point defaults in defaults.elements.point, or you should use the pointBackgroundColor in the options to override that default.
I really dislike the prefixed option names overall, but there is a proper use case for those in a line chart when avoiding scriptable options (for performance)

Ok, this used to work before the options prefixes were changed, I guess that namespacing them in the dataset was not the right fix. Will try your suggestion and report back.

Removed point defaults.


[2.0] default named child routes not being matched

Vue.js & vue-router.js version
2.0.0.rc3
Reproduction Link
https://jsfiddle.net/u5jjwzft/9/
Steps to reproduce
Click on the links and you will see that when using the router-link with :to="{name: 'parent'}" wil not load default
What is Expected?
When either you use to="/parent" or :to="{name:'parent'}" you expect to see the default child route component.
What is actually happening?
It is not showing the default child route component.
On first page load of /parent it works but when clicking on one link and then to the named route it doesn't work.

I'd say this is expected behavior - named routes are precise targets, i.e. when you specify { name: 'parent' }, you are telling the router that you explicitly want to navigate to that route with no child route. If you want to navigate to the default child route, then you should be using the name of that child route instead.

I know this issue is old but I completely disagree. I made the default child the default for a reason and now I can never refer to my route with { name: 'my-route-name' } for this particular view in my app. That makes no sense. If I change the URL of that view I can't rely on the named route logic to save me from the work of updating the links throughout the code.
I found this issue specifically because I was confused by this so-called "expected" behavior. I actually expected my default child to be rendered any time I visit that parent route/component, regardless of how I link to it

@joemsak I also found this behavior intuitive and it was problematic for me when I attempted to implement an "up" button to navigate up the site hierarchy.  I could programatically find the name of the current route's parent, but finding the name of that parent's defaultChild was problematic.
Fortunately, there is a workaround described at https://github.com/vuejs/vue-router/issues/822#issuecomment-255685008 to give the behavior that we expect.
new Router({
	routes: [{
		path: "/parent/:someProp",
		component: Parent,
		name: "parent",
		// explicitly reroute to default child on named navigation
		// https://github.com/vuejs/vue-router/issues/822#issuecomment-255685008
		redirect: "/parent/:someProp",
		children: [
			{
				path: "",
				name: "defaultChild",
				component: DefaultChild
			},
		// ...


How can I get values as specific type when using 'with'?

From document, default query is here.
select * from cypher('test_graph', $$ 		match(v:Part) return v $$) as (result agtype)
That result of type is 'agtype'.
In this situation, I create basic query using 'with'.
with a as(select * from cypher('test_graph', $$ 		match(v:Part) return v $$) as (result agtype)) select a from a
Result of datatype of this query is 'record' as a.
So, I have a some questions.

Can I cast default datatype except on 'record'?
In this query, how can I get specific data(ex. select a.id as id)?

I don't know English well, so I don't know if I understood your question well. If I understood your question well,
I think this question can explain your curiosity.
CREATE EXTENSION age;
LOAD 'age';
SET search_path = ag_catalog, "postgres", public;
SELECT create_graph('test_graph');
SELECT * from cypher('test_graph', $$
CREATE (a:Part {part_num: '123'}),
(b:Part {part_num: '345'}),
(c:Part {part_num: '456'}),
(d:Part {part_num: '789'})
$$) as (a agtype);
select * from cypher('test_graph', $$ match(v:Part) where id(v) =<PHONE_NUMBER>31969 return v.part_num  $$) as (result text);
with a as(select * from cypher('test_graph', $$ match(v:Part) where id(v) =<PHONE_NUMBER>31969 return v.part_num  $$) as (result text))
select a from a;

If the cypher query result is in text format, you can cast it in text format.
To query using a specific id value, you can use the id function where id function.

Of course, it returns text type when I use cypher query only(as (result text)).
I want know when I use 'with', how I get data that I wanted type.
Here is returned value in queried that.

Is this what you want?
혹시 원하는 의도가 이것 인가요?
select "result", "id" from cypher('test_graph', $$ match(v:Part) where id(v) =<PHONE_NUMBER>31969 return v.part_num , id(v) $$) as (result text, id text);
with a as(
select * from cypher('test_graph', $$ match(v:Part) where id(v) =<PHONE_NUMBER>31969 return v.part_num , id(v) $$) as (result text, id text)
)
select "result", "id" from a;

Oh yes. it's correct.
Thanks to reply.