celery: message signing

Adds optional setup of celery message signing and makes docker to use this technology by default.
Rebased against: maint-2.0

I'm not sure I see the added security benefit of adding message signing in our setup. I can see that message signing in a more open system (i.e. one where clients/workers/brokers doesn't intimately know each other), can be used to add extra trust. However in our production setups RabbitMQ has a virtual host per instance, requires authentication, and firewall is blocking connections from anything but clients and workers.
That said, it's not I'm against adding message signing, but I would favour using a solution that uses our already existing shared secret key. I think that using public key signing just adds extra complexity in our setup due to the need for private/public keys and sharing the public keys (and if the same private/public key's are used for all involved parties, there's little difference to a shared secret key).
I aware that you are just using features already in Celery, but perhaps it's not too much work writing a task serializer that uses our secret key to sign messages.

It is optional and only activated if CELERY_TASK_SERIALIZER = u'auth'

Longer answer:
RabbitMQ is not the standard way proposing in our docs. We use Redis, for different purposes, without write protection.
Our "secret key" is NOT a state-of-the-art key. It is just a secret. If exposed, it is possible to use it for different API and entry points. We do not even have a good rejection mechanism which tells us which effects a change of the secret will have.
Using the same key for different purposes is a bad idea.
Arguing that we already have a firewall does not help. Should you somehow get write access to the network (in case the firewall fails) you can execute every possible command our workers (remote code execution). Adding additional layers of security is good practice and is expected by project from our size.
The size of our codebase is so enormous that I would even bet that it contains at least one critical security bug.
A last word on firewalls: it might protect one specific way to enter your systems (like a wall), but it won't protect you from other (more creative) ways, e.g. when other systems behind it have vulnerabilities.
Alternative proposal for easier setup:
Dump the secret_key and use a proper CA architecture by default to generate all secrets for all subsystems.

ref #3015

Thanks for answer. Please note, I'm not arguing against adding support for messaging signing. Also, I do recognise that the most easy implementation is to use what Celery already have (and it means we  have less code to maintain).
My concern is purely related to trade off between complexity and security. Security is also a matter of simplicity and easily manageable solutions. If I understand correct, you would need one private/public key per node, and share all the public keys between the machines to have any benefit over a solution based on a shared secret (which doesn't have to be the one in SECRET_KEY)?
In the end, I'm concerned that adding something which is hard to manage in a production system won't be used, and thus have no effect, whereas if we enable message signing by default using a shared secret we already have, then it adds security by default.
Comments on specific points:

RabbitMQ is not the standard way proposing in our docs. We use Redis, for different purposes, without write protection.

For all current production system RabbitMQ is the defacto standard. The fact that only Redis is mentioned in the docs is a bug, and an artefact of that we keep dev setups as simple as possible. Also, our docs basically lacks any kind of production setup guide (the little that is here http://invenio.readthedocs.org/en/latest/getting-started/overlay.html was never running on an actual production system)

The size of our codebase is so enormous that I would even bet that it contains at least one critical security bug.

Yes, I'm quite sure about that as well, and I don't think it's only 1. Please do go hunt for them. A big reason here I believe is the complexity fo the code base, which is being addressed by our road map for Invenio 3.0

Our "secret key" is NOT a state-of-the-art key. It is just a secret. If exposed, it is possible to use it for different API and entry points.

Isn't the same true for the private key which could get expose as easily as our secret key? If our secret key is weak then please ticketize so we can fix it.

We do not even have a good rejection mechanism which tells us which effects a change of the secret will have.

Agree. Will you ticketize so that we can start working on it? Somekind of deprecation mechanism that would allow us to easily change our secret key on a regular basis would be great.

Using the same key for different purposes is a bad idea.

For my understanding, do you have some examples why it would be bad in this case?

Arguing that we already have a firewall does not help. Should you somehow get write access to the network (in case the firewall fails) you can execute every possible command our workers (remote code execution).

I see this as an argument against using Redis as the broker. RabbitMQ requires authentication (which should naturally happen over an encrypted connection).

Adding additional layers of security is good practice and is expected by project from our size.

Yes, and I'm also not arguing against adding message signing or the extra security. My concern is tradeoffs between two different solutions to adding extra security.

A last word on firewalls: it might protect one specific way to enter your systems (like a wall), but it won't protect you from other (more creative) ways, e.g. when other systems behind it have vulnerabilities.

Naturally not. Gaining access to any of the nodes where the private key is stored, would also allow you to do remote code execution.

What is SECRET_KEY good for?

If I understand correct, you would need one private/public key per node, and share all the public keys between the machines to have any benefit over a solution based on a shared secret (which doesn't have to be the one in SECRET_KEY)?

All nodes that submit tasks to the worker should ideally have their own client.{pem,crt}. These are signed by the common ca.pem and therefore can verified using ca.crt. The CA data should be protected as good as possible. The client data should be protected as well, but in case of leakage they can be revoked using the CA.

Our "secret key" is NOT a state-of-the-art key. It is just a secret. If exposed, it is possible to use it for different API and entry points.

Isn't the same true for the private key which could get expose as easily as our secret key? If our secret key is weak then please ticketize so we can fix it.

The secret can not be used for asymmetric cryptography because it does not contain the entropy to generate the required keys. The signing procedure with this secret is only be possible when using an HMAC and therefore the secret itself is required to verify the signatures. This is OK for some solutions (oauth, signed cookies) but is unacceptable when our clients want to verify that they got valid data from us.
Broker questions (aka move to RabbitMQ)

For all current production system RabbitMQ is the defacto standard.

Then let's change the docs. I would change the Docker setup as well.

Arguing that we already have a firewall does not help. Should you somehow get write access to the network (in case the firewall fails) you can execute every possible command our workers (remote code execution).

I see this as an argument against using Redis as the broker. RabbitMQ requires authentication (which should naturally happen over an encrypted connection).

See point before.
Secret reusage

We do not even have a good rejection mechanism which tells us which effects a change of the secret will have.

Agree. Will you ticketize so that we can start working on it? Somekind of deprecation mechanism that would allow us to easily change our secret key on a regular basis would be great.

I think it would require multiple secrets (or keys) to realize this, see next point.

Using the same key for different purposes is a bad idea.

For my understanding, do you have some examples why it would be bad in this case?

Imagine the following: We use the secret to produce HMACs for:

oauth
cookies
celery task signing

We usually do not use the secret directly. For technical reasons we usually use the SHA{1,256.512} for this. So in fact our true secret is z=sha1(SECRET_KEY).  While the primary impact (unusable signatures for these mails) is not that heavy, the secondary impact (leakage of z) affects all the components mentioned above. I rate this mistake as quite likely, because honestly cryptography is a very complicated thing (see the TLS/SSL problems of the past 12 months).
Addition stuff

In the end, I'm concerned that adding something which is hard to manage in a production system won't be used, and thus have no effect, whereas if we enable message signing by default using a shared secret we already have, then it adds security by default.

I share this concern and would volunteer to automate this process.

A last word on firewalls: it might protect one specific way to enter your systems (like a wall), but it won't protect you from other (more creative) ways, e.g. when other systems behind it have vulnerabilities.

Naturally not. Gaining access to any of the nodes where the private key is stored, would also allow you to do remote code execution.

Imagine the following scenario: You have multiple nodes for different purposes in you network (complete Reds-driven Invenio [Redis, Web, Worker, MySQL] + mail server + analysis/machine learning). They are of course protected by a central firewall. A hacker is able to exploit one of the non-Invenio nodes, say machine learning, because it made a not-very-good (i.e., unverified) update PyPi or whatever. The impact is quite low because in your setup it (=machine learning node) does not contain any critical data. But now he has access to your network. Your Redis machine has his own firewall/iptables to only allow access from Invenio:Web and Invenio:Worker by checking either the MAC-Address or IP-Adress of incoming connections (or even both). For the first filter the hacker changes the MAC-Address of the network card (it is programmed by software, the hardware data is only a suggestion). The second one can be bypassed by ARP-Spoofing, at least in the most not-very-good networks (which I guess are 80% out there). Because your admin (sorry) choose a to short AUTH-secret for redis, the hacker can bruteforce it and gets access to your celery queue. This enables remote code execution. Attack complete. This is why deep security can help and is never to much. In other words: Better wear a helmet.

Blocked until https://github.com/mher/flower/issues/408 gets resolved.

Closed, since I'm not a part of the Invenio dev. team anymore.


5009566ea broken on Debian Stretch

Got the following when running 5009566ea.
Something is wrong with winit.
thread 'main' panicked at 'XOpenIM failed', /local/home/wei/.cargo/git/checkouts/winit-5993d63686bf7f83/200ef9c/src/platform/linux/x11/mod.rs:692
stack backtrace:
   0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace
             at /checkout/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::sys_common::backtrace::_print
             at /checkout/src/libstd/sys_common/backtrace.rs:71
   2: std::panicking::default_hook::{{closure}}
             at /checkout/src/libstd/sys_common/backtrace.rs:60
             at /checkout/src/libstd/panicking.rs:355
   3: std::panicking::default_hook
             at /checkout/src/libstd/panicking.rs:371
   4: std::panicking::rust_panic_with_hook
             at /checkout/src/libstd/panicking.rs:549
   5: std::panicking::begin_panic
             at /checkout/src/libstd/panicking.rs:511
   6: winit::platform::platform::x11::Window2::new
             at /local/home/wei/.cargo/git/checkouts/winit-5993d63686bf7f83/200ef9c/src/platform/linux/x11/mod.rs:692
   7: winit::platform::platform::Window2::new
             at /local/home/wei/.cargo/git/checkouts/winit-5993d63686bf7f83/200ef9c/src/platform/linux/mod.rs:181
   8: winit::window::<impl winit::WindowBuilder>::build
             at /local/home/wei/.cargo/git/checkouts/winit-5993d63686bf7f83/200ef9c/src/window.rs:113
   9: glutin::platform::platform::x11::Context::new
             at /local/home/wei/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.9.2/src/platform/linux/x11.rs:213
  10: glutin::platform::platform::Context::new
             at /local/home/wei/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.9.2/src/platform/linux/mod.rs:52
  11: glutin::GlWindow::new
             at /local/home/wei/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.9.2/src/lib.rs:322
  12: alacritty::window::Window::new
             at src/window.rs:193
  13: alacritty::display::Display::new
             at src/display.rs:141
  14: alacritty::run
             at src/main.rs:92
  15: alacritty::main
             at src/main.rs:45
  16: __rust_maybe_catch_panic
             at /checkout/src/libpanic_unwind/lib.rs:98
  17: std::rt::lang_start
             at /checkout/src/libstd/panicking.rs:433
             at /checkout/src/libstd/panic.rs:361
             at /checkout/src/libstd/rt.rs:59
  18: main
  19: __libc_start_main
  20: _start

Same issue on Ubuntu 16.04.3

Hmm, works fine for me on Ubuntu 16.04.2. Never a good thing when winit fails in an X call; it's really hard to get at the actual errors. Does anything change if you remove the with_transparency(true) call that was added in that PR?

Hmm... I cleared cargo caches, cloned a fresh copy of alacritty and rebuilt. Could see the issue anymore.

Thanks for the feedback @liuw.
@VegarM can you try running cargo clean and rebuilding?

Same symptoms as #732?

@jwilm
I've now tried (on fresh master branch):

cargo clean, cargo build --release
rm -fr ~/.cargo, curl https://sh.rustup.rs -sSf | sh, cargo build --release
Also tried this in a newly cloned repository.
Also tried default alacritty.yaml

Nothing has worked for me, still the same error.

I ran git bisect and concluded that this was introduced on 4c4c2f5
This is the backtrace when running from there:
(using elementary OS 0.4.1 loki - based off Ubuntu 16.04.2 LTS)
thread 'main' panicked at 'XOpenIM failed', /home/breda/.cargo/git/checkouts/winit-5993d63686bf7f83/cc89d56/src/platform/linux/x11/mod.rs:680:16
stack backtrace:
   0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace
             at /checkout/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::panicking::default_hook::{{closure}}
             at /checkout/src/libstd/sys_common/backtrace.rs:71
             at /checkout/src/libstd/sys_common/backtrace.rs:60
             at /checkout/src/libstd/panicking.rs:380
   2: std::panicking::rust_panic_with_hook
             at /checkout/src/libstd/panicking.rs:396
             at /checkout/src/libstd/panicking.rs:611
   3: std::panicking::begin_panic_new
             at /checkout/src/libstd/panicking.rs:553
   4: winit::platform::platform::x11::Window2::new
             at /home/breda/.cargo/git/checkouts/winit-5993d63686bf7f83/cc89d56/src/platform/linux/x11/mod.rs:680
   5: winit::window::<impl winit::WindowBuilder>::build
             at /home/breda/.cargo/git/checkouts/winit-5993d63686bf7f83/cc89d56/src/platform/linux/mod.rs:181
             at /home/breda/.cargo/git/checkouts/winit-5993d63686bf7f83/cc89d56/src/window.rs:113
   6: glutin::platform::platform::x11::Context::new
             at /home/breda/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.9.1/src/platform/linux/x11.rs:213
   7: alacritty::window::Window::new
             at /home/breda/.cargo/registry/src/github.com-1ecc6299db9ec823/glutin-0.9.1/src/platform/linux/mod.rs:52
   8: alacritty::display::Display::new
             at src/display.rs:140
   9: alacritty::run

I really messed up a lot of things. Cleaning up the cache won't work.
My bisection points to 4c4c2f5f5c81e6efb580e50ea993606160809469.

Then this is most likely a duplicate of #732
Try starting IBus or fcitx or equivalent before alacritty

@addobandre That's just a workaround. I now know why it failed -- it is actually an issue in winit, which insists IM to be valid or panic otherwise. I will ask winit developers for opinion. I think their current setting is too strict.


Dependencies in the classpath configuration in the buildscript block of build.gradle are marked unused

[ ] I have read the guidelines for submitting issues

🐛 Describe the bug
I just changed to 0.60.0, and now the dependency in the buildscript block to a classpath configuration is marked as unused.
The strange thing is, if i remove the line from version.properties, it is added again with the refreshVersions task, but marked as unused, also different version choices are added, as there are snapshot versions available.
⚠ Current behavior
The dependecy is marked as unused
✅ Expected behavior
It is not marked as unused.
💣 Steps to reproduce
Don't know, maybe using classpath dependencies in a buildscript block are enough.
📱 Tech info

Build Scan URL:

I changed the version in my report.
OK, this is my build.gradle in the root project
buildscript {
    repositories {
        maven {
            url "http://local.nexus.com/erpo"
            allowInsecureProtocol true
        }
    }
    configurations.classpath {
		resolutionStrategy.cacheChangingModulesFor 0, 'seconds'
		resolutionStrategy.cacheDynamicVersionsFor 0, 'seconds'
	}
    dependencies {
        classpath "group:scripts:_"
    }
}

apply plugin: 'rootProject'

Currently there are two versions of script available, 1.0 and 1.1-SNAPSHOT.
Everything is fine with 0.51.0, but with 0.60.0, I get (after deleting the line in the versions.properties) something like
#unused
version.group..script=1.0
###                  =1.1-SNAPSHOT

(I wrote this by hand, might look sloghtly different)

Which Gradle version are you on?

I use gradle 7.6.2

Here's a reproducer.
This also reproduces #624 if you comment out the buildscript in the root build.gradle.kts.

Same issue here, bunch of my dependencies marked as unused (example here) even though they are used in buildSrc (see here)

I found the culprit and fixed it. Will release 0.60.2 this Tuesday.
You can already try the 0.60.2-SNAPSHOT version.

Thank you for the simple reproducing projects!
They were helpful to see a project config reproducing the issue, I knew what to put in my test projects.

Hi @LouisCAD,
Apparently version 0.60.2 still producing unused comments, please see in https://github.com/attila-fazekas/kolibrium/pull/40/files

This is for dependencies declared in the buildSrc only I can see.
Can you try specifying the version of refreshVersions in the buildSrc too, as documented in the setup guide on the website?

Have tried, but got the following error:
Error resolving plugin [id: 'de.fayard.refreshVersions', version: '0.60.2']
> The request for this plugin could not be satisfied because the plugin is already on the classpath with an unknown version, so compatibility cannot be checked.

Please, look at the documentation again, there's a part specific to buildSrc.

Gradle is so unpredictable!
I found another case of dependencies marked as unused, caused by the fact that dependencies are not always there when you expect it. I have no clue why it works fine on a given project, but not on another. Anyway, I will publish another version today or tomorrow.

@attila-fazekas Look for the heading "If you have a buildSrc module" here on the setup page. Notice it's slightly different from the root project.

Hi @LouisCAD,
Thanks for investigating it.
I did look at the provided link a couple of weeks ago and applied the suggested setup, but it did not make any difference.
Have just bumped the refreshVersions version to latest, you can see the resulted versions.properties here

@attila-fazekas I see you have defined no repositories in the buildSrc. Can you try adding mavenCentral() as done here?

@LouisCAD that solved the issue 🎉
Thanks a lot!


Add ‘fileExtension’ and ‘setFileExtension’

Allow to get and set file extension. I'm tired of converting to and form FilePath every time I need this.
I tried to follow your naming style, but well, filext may be not the best name, let me know if you have a better naming idea.

I wonder what happens though when you try to setFilext "something/slashy"… Need to check.

I think we must check extension for validity so it's not anything that may invalidate given path. Adding a new data constructor to PathParseExtension and makng setFilext live in a MonadThrow may be a good idea.

The failing build is not related to the change.

Another idea is to add Extension type with similar smart constructors and TH helpers and use that as argument. Then we don't need to make setFilext  living a MonadThrow. Maybe it's a bit heavy-weight solution though.

@chrisdone First mergeable version is there, please take a look. I can change the check if necessary and add tests.

Ping?

Apart from the unsafeCoerce this looks fine.

@chrisdone This use of unsafeCoerce is safe AFAIK. A better idea how to make it work without it/reason we should not use it here?

It just converts between two newtype wrappers around the same String type.

I've fixed the commit here.
This was a pretty poor PR, @mrkkrp. You really have to test out your code:
bash-3.2$ pwd
/Users/chris/Work/path
bash-3.2$ git checkout 4aea22c04f96f897867154c71a7caaaf15539080
Note: checking out '4aea22c04f96f897867154c71a7caaaf15539080'.
HEAD is now at 4aea22c... Add ‘fileExtension’ and ‘setFileExtension’
bash-3.2$ stack ghci
Prelude Path Path.Internal> setFileExtension "hs" $(mkAbsFile "/foo/bar.txt")
<interactive>: internal error: stg_ap_v_ret
    (GHC version 7.10.3 for x86_64_apple_darwin)
    Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug
Abort trap: 6

Even correcting the unsafeCoerce with a coerce, the arguments to replaceExtension are the wrong way round:
Prelude Path Path.Internal> setFileExtension "hs" $(mkAbsFile "/foo/bar.txt")
"hs./foo/bar.txt"

Be more careful next time! 😅 👍

This use of unsafeCoerce is safe AFAIK. A better idea how to make it work without it/reason we should not use it here? It just converts between two newtype wrappers (with different phantom types) around the same String type.

In this case we can use the recent roles-based Data.Coerce function, which only performs safe coercions of data types. So the code ends up like:
setFileExtension :: MonadThrow m
  => String            -- ^ Extension to set
  -> Path b File       -- ^ Old file name
  -> m (Path b File)   -- ^ New file name with the desired extension
setFileExtension ext (Path path) =
  if FilePath.isAbsolute path
    then fmap coerce (parseAbsFile (FilePath.replaceExtension path ext))
    else fmap coerce (parseRelFile (FilePath.replaceExtension path ext))

Where fmap coerce simply coerces the Abs or Rel type into the more general b.

Uh oh, indeed! (This is was rather a sketch, I didn't have much time.) Don't forget to drop InvalidFileExtension now.

Ah I forgot to add @since declarations to the docs. Can you do it now?

@mrkkrp What's the syntax like for that?

https://hackage.haskell.org/package/base-<IP_ADDRESS>/docs/src/Data.Bool.html#bool
See docs for bool, it has @since <IP_ADDRESS>.

Added @since to both and bumped as 0.5.11.


docs: publish consolidated docs using mkdocs

Issue being fixed or feature implemented
Given the changed project structure, it appears that docsify won't work in a monorepo format. Prior to having this monorepo, a number of its packages hosted documents on their own repository Github Pages sites (js-dapi-client, js-dash-sdk, js-dpp, and wallet-lib). These sites were built via docsify.
What was done?
I found MkDocs which accomplishes a similar purpose. While it doesn't directly support monorepos, I found it to be more flexible than docsify. This PR uses it, along with a GH action to publish an aggregated set of documentation for those 4 packages. Fortunately it works relatively well with no modification to the documentation source files.
How Has This Been Tested?

I experimented using a fork of this repo and deploying the resulting documentation on that repo's GH Pages site: https://thephez.github.io/platform/. There are a few broken links and malformed markdown in the existing files that will need to updated, but those changes would be a separate PR after this is merged.
Breaking Changes

Checklist:

[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have added or updated relevant unit/integration/functional/e2e tests
[x] I have made corresponding changes to the documentation

For repository code-owners and collaborators only

[ ] I have assigned this pull request to a milestone

Maybe this PR should be opened from a branch of this repo rather than a fork :thinking:


Pre-commit hook doesn't block on YAPF diff

The pre-commit hook should block the commit if the YAPF diff turns up a delta, but currently it seems to just print a message and keep going. We should change the build so that it returns a failing exit code on a non-zero diff.

Apparently this is a behavior change in YAPF:
https://github.com/google/yapf/issues/325
They're considering a fix, but there's a workaround possible:
https://github.com/google/yapf/issues/325#issuecomment-261552890


Getting this issue suddenly/

Suddenly crashing VS code without any reason. It's really too disgusting to work with. Please solve this issue from mac.

Which version are you seeing the crash ?
Does it repro if you use the insiders version https://code.visualstudio.com/insiders
Launch with --crash-reporter-directory <some-absolute-path> and if it crashes, provide the crash dump generated at the directory location. Thanks!


QueryBuilder.modify(undefined) throws in 1.3.0

Before v1.3.0, it was possible to call buider.modify(undefined) without side-effects. Changes introduced in v1.3.0 now cause an unhandled ModifierNotFoundError to be thrown from builder.applyModifier().
After inspecting the changes in https://github.com/Vincit/objection.js/commit/93b3a1277234ca225814be9ee43ee0c87c0eb922#diff-6d0c7b01cd221943c0d8d7124a25e665L125 this doesn't work anymore:
From:
 modify() {
   return isString(arguments[0])
     ? this.applyFilter.apply(this, arguments)
     : super.modify.apply(this, arguments);
 }

To:
modify() {
  return !isFunction(arguments[0])
    ? this.applyModifier.apply(this, arguments)
    : super.modify.apply(this, arguments);
}

The important bit is the change from isString() to !isFunction()
So before, when I passed undefined to modify(), it got routed to super.modify() and this was fine. Now it gets routed to this.applyModifier(), and this throws the ModifierNotFoundError that doesn’t get caught.

@koskimas what was the reasoning behind the change from isString() to !isFunction()? What other values than strings are supported in applyModifier() now?

All other places that take modifier names, now also take an array of modifier names, and I wanted to allow the same here. Currently applyFilter (and applyModifier) are a bit weird as they only allow multiple arguments and not an array of modifier names.
We probably should check isString(arg) || Array.isArray(arg) instead then?

Or simply add
if (!arguments[0]) return

Yeah, i think it should be a return this though?

I've worked around it here for now: https://github.com/ditojs/dito/commit/be865ad9c20e29b596f37aa90562e270f10407dc#diff-14a6d8574584205a94c7e4c5e0c45340R223

Yes that sounds good. I think it should be a return this though?
Yes absolutely. I'll add that soon


Need a models directory to store configuration files

This is a todo listed under #3.
The models directory should live under src/dendron. For now it will contain files for config data classes, but may hold other model-related code in the future.

Resolved by 1aaff02. Ended up going with the name configs instead of models.


Improve term frequency adjustments

We currently use an ex-post term frequency adjustment described in the fastLink paper's annex.
This is an ex post adjustment that is only applied in the case that, in the particular pairwise comparison, the term matches.
So, for instance first_name_l = 'John' vs first_name_r = 'John' can adjust for the fact John is a common first name.
We still think that the approach described in the Fastlink annex is theoretically valid, relying on the independence conditional on match status assumption.
However, in practice we've found it can be problematic due to violations of this assumption.
In particular, errors due to a failure in this assumption can compound - due to the same mechanism that errors in match weights are exacerbated by the EM algorithm.
To see why,  consider an extreme example where a person's first name and first initial are both included as comparison columns.  This is a clear violation of the assumption of independence conditional on match status.
Where there is a match on initials, this will be double counted because the match weights for both initials and first name are counted separately.  The match_probability will be estimate too high.  As EM proceeds, these errors can get worse as estimates re-enforce one another.
However, the term frequency adjustment for, say, John, uses these same estimated match_probabilities to calculate the adjustment for John - potentially triple counting.

We now believe a different approach is preferred.  We have had conversation with various other expert data linkers who have also suggested this alternative approach.
Rather than an ex post adjustment, we instead calculate u probabilities row-wise, rather than the current approach of a single average u across all rows.
They are calculated based on the term frequencies.  For instance, if "John' appears 1% of the time, the u probability for 'John' vs. 'John' should be 0.01.  i.e. 1% of the time, amongst records which truly do not match, you'll observe a match on John vs John
The u probability for John vs. not John just uses the average - e.g. x% of all non-matching records do match on surname.

I suggest we start with an implementation of ☝ .
There's then an extension I believe we can use that allows us to apply term frequency adjustments to fuzzy matches:
Let's suppose we had set up m and u parameter estimates for a 'city' field:
level 2:  exact match
level 1: jaro match above some threshold
level 0: no match
The London vs Londn comparison would fall into level 1.
So we somehow want to use the level 1 u value, but adjust the match weight log2(m/u) down for the fact that London is a really common city.    More generally, I think you want the adjustment to be made on the basis 'commonest' side of the match (London, not Londn) to reflect the fact that's the one you're most likely to 'mistakenly' match to.      (one side is a misspelling or alias of the other.   so if we're going to match Robin to Rob, then the frequency of Robs is more relevant than Robins).***
My current thinking is as follows.  (I am not 100% that this works, but my gut tells me it'd do a pretty good job):
a) Compute the average u value for a level 2 (exact) match. Splink does this using the 'take record comparisons at random and see how often you observe level 1' approach.
b) Compute the u value for a level 2 match where the values are both London.  Do the same for Londn.
c) Compute the delta (in log bayes factor terms) between (a) and (b) for London and Londn.  Take the delta associated with the commonest term (in this case London, as opposed to Londn).
d) Compute the average u value amongst records of level 1.  Again, Splink does this already.
e) The final u value is obtained by adjusting (d) by the delta.
*** It it possible you might want to add the term frequency of robin and rob?    But we've already accounted for fact number of collisions are more probable in the average u factor.  What we're accounting for here is that, in addition to collisions being more common, this particular collision is likely because Rob (/London) occurs a lot.

We have a downwards adjustment due to the fact average u is lower
An then we adjust down because it collided with London, i.e. there will be loads and loads of comparisons where Londn matches London, because London is a commonly occuring term.  This makes Londn vs London comparisons particularly devoid of information, more so than say Warrington vs Warringtn
I think both adjustments are legitimate - although the scale of the adjustment for term freq may not be 100% right, i don't think it would be far off.

This can be explored further here:
https://observablehq.com/d/65332d708285fead
Note, for instance, that the delta log 2 bayes factor is the same for e.g.:
1/6 -> 1/12
and
1/60 ->1/120

A useful way to think about whether the suggested approach (to TF adjustments on non-exact matches) ☝  works might be extreme examples.
For instance, what happens when you have a very permissive level with a jaro threshold of >0.01, or a location threshold of 'within 150 miles' (i.e. where two records selected at random are very likely to match, on average).
On the face of it, I can't see why

Here's an exploration of how term frequency adjustments might work on fuzzy matches:


CSS

CSS
User Story
As a boot camp student
I want the prework notes to be structured on a webpage
So that i can easily find and read the information
Acceptance Criteria
Given a Prework Stud Guide website
When i view the study guide
THEN I see a dark blue header and footer, and four boxes with a shadow

CSS has been added


No card posted when following video/instructions exactly

I have been trying to use an incoming webhook to post to a Teams team that I am in, and followed the instructions here exactly (new webhook, new team, using postman for POST request), but no new card appears. I continue to receive the response:
{"version":"1.1","content":{"headers":[{"key":"Content-Type","value":["text/plain; charset=utf-8"]}]},"statusCode":200,"reasonPhrase":"OK","headers":[],"trailingHeaders":[],"requestMessage":null,"isSuccessStatusCode":true}
when sending a request via postman or via the curl command when I expect a response of '1'. I also get this same response regardless of what headers I use. I do get "Bad payload received by generic incoming webhook" when I invalidate the json format of my payload. Has the way to use these webhooks changed? Any help would be appreciated.
Additional Note: My team has a service that uses these webhooks regularly, and they all stopped working after 9/30/2022. My investigation into why they all stopped lead me to trying to create one from scratch, which then lead to creating this issue.

@HulonJenkins - We tried sending cards following Create and send messages documentation via postman. The cards are getting posted in Teams properly.

We tried sending a message using cURL command as well, that is also posted in Teams.

Can you please let us know if you have done it differently, so that we can check it locally. Thanks.

Hi, I ran the exact same curl command that I did yesterday (still had the environment open, and just hit up arrow many times to ensure it was the same command) and the card now posted in my Teams chat as expected. I am extremely confused as to why this failed yesterday but worked this morning.
I was using this command in Ubuntu.
curl -H 'Content-Type: application/json' -d '{"text": "Hello World"}'

I am still getting the same response from yesterday.

It also works in Postman now too (again, exact same command that I had open from yesterday). I guess it works now but don't know what I was doing wrong/differently if anything from yesterday. Does a webhook take time to work properly? Is there any possible reason that could explain this behavior? Thank you.

@HulonJenkins -

We are checking with the engineering team, the cause of above-mentioned temporary issue. There is no delay/lag for the webhook to work.
I have deleted a team and tried the POST request via curl and postman. I got the below responses.

Hi, thanks for all the responses. I tested your webhook and I get the same response that you showed in your image. When I test my own webhook in a Teams team that I deleted, I get status 200 instead of error 404. Please attempt with my webhook to ensure you get the same status 200 response.
https://microsoft.webhook.office.com/webhookb2/8bf7d335-df6e-4b15-ab6a-47d61bb4992f@72f988bf-86f1-41af-91ab-2d7cd011db47/IncomingWebhook/4307125a29214a04afaf117c2c90c3d4/8c76b776-e81c-470e-8926-5e065782ec1a
I noticed our webhooks differ significantly (as opposed to when I make multiple webhooks for the same channel). Is it possible that webhooks are handled differently based on the Tenant of the webhook? The response you got with your webhook is what my team would expect in our system that uses these webhooks, but now it appears that all the ones that we create (with the tenant being 'microsoft') are returning different responses.

@HulonJenkins -

The temporary issue happened because of a certificate issue, ICM was raised, and the fix was deployed.
We are also getting status 200 with your webhook. We are checking this behavior with engineering team; we will get back to you.

Thank you, sounds good.

@HulonJenkins - We raised an issue for the 2nd issue mentioned above. We will keep you posted if there are any updates.

Thanks

Are there any updates on this from the engineering team on the expected behavior of webhooks? In particular ones that are in deleted teams?

We do not have any update to share currently. We will keep you posted.

Do you have any updates on the expected behavior of webhooks? I am still blocked on this.

@HulonJenkins - Engineering team has confirmed that currently this is by design behavior.

I see that you can restore a deleted team channel for 30 days, and I'm assuming the webhook stays active (in a deleted channel) for that case, but I'm now wondering if the webhook will be deleted along with the channel after the 30 days pass, or will the webhook remain active indefinitely?

We are closing this issue, please feel free to reopen if you need help.

Hello @HulonJenkins - Hope you are doing well!!
Please let us know if you are looking for any further help related to this issue?


refac: improve way to extract and validate information from pandas series in AppBio QuantStudio

Improve way to extract and validate information from pandas series in AppBio QuantStudio

first part of https://github.com/Benchling-Open-Source/allotropy/pull/38

Can you wait for this PR: https://github.com/Benchling-Open-Source/allotropy/pull/119 from @bdworth to go in, and sync up these method names with the naming there?
Specifically, functions that start with assert_ should become try_ and try_<> that allow None will become try_<>_or_none


Memory leak in +[JNFWeakJObjectWrapper wrapperWithJObject:withEnv:]

zcash4mac 1.0.4-208
Steps:
Launch zcash4mac.
What happened:
Memory leak in +[JNFWeakJObjectWrapper wrapperWithJObject:withEnv:]
https://www.dropbox.com/s/pf54fy2ge7yx5o9/zcash4mac.trace.zip?dl=0
Expected result:
No leaks.
Notes:
Unzip and open file with Apple Instruments (part of Xcode tools).

This project is dead so retracting.


KafkaOutput Trigger throws an error if null value passed

I'm using kafka out trigger to push message but there is some business scenario which identify whether to pass message or not. But if we pass null to message it throws an error "Message value was not defined".  Does this mean we cannot pass null or empty value in the message ? it force to pass something valid value if this is the case then how to handle the business scenario where there is no need to pass message.

Hi @Alpesh1984909090,
Can you share more information on how you are setting the output value?

Hi @alpeshdhanesha,
Can you share the function code? I tried setting the output message as null and it worked for me.


-O argument with v4l2 device returns just first frame when capture

Describe the bug
After creating device using v4l2loopback i am trying to run v4l2server using v4l2rtspserver -O /dev/video5 -W 1920 -H 1080 -F 30 -P 7447 /dev/video0, rtsp stream works but when i am trying to capture /dev/video5 i get just first frame.
Desktop (please complete the following information):

OS: [Ubuntu Mate]

Hi,
The -O option was more for debug purpose, I will probably remove it.
Best Regards,
Michel.

Hi, @mpromonet, I was able to use it, it is very usefull if you want to stream with minimum delay and get frames for another program on device, additional reencoding from one virtual camera to another using ffmpeg helped me.

Hi,
Using v4l2loopback as a source, allow you to connect multiple consumer.
Best Regards,
Michel.

@mpromonet Yeah, but in my case using v4l2loopback as a source for rtspserver multiplied the delay


[Feature Request] Add support for Valve's Steam and Trade Confirmations

Heya, excellent app! I love the fluent design language!
If I may, I have a feature request. I would love if you added support for Steam and steam trade confirmations; that's the only thing that I still need to use a secondary authenticator for, of which there's only one working version which last had an update three years ago but functions (though crashes when confirming trades) and a mostly broken, old program called WinAuth. I would be highly grateful if you'd consider adding it so I can completely switch over to Protecc. I really don't want to install the steam app just to authenticate steam.
Regardless, thanks for providing Protecc; it's been a pleasure using it so far

It looks like they are using this library. https://github.com/geel9/SteamAuth/ and based on the functionality provided it will be better if i made this a seperate app. Unfortunately I don't use steam often and there are some higher priority features I need to work on for Protecc but if anyone wants to give this feature request a go feel free to make do it.

https://github.com/BeyondDimension/SteamTools
this repo can do some reference


Enable 'Web Access' link for XML files

Closes #170
Example:

I thought about unifying the links when the file path is the same as the project path, but the links do different things and it seems better to keep that.
TypeScript support required changes that were more extensive than I felt comfortable doing without testing, but it should be easier to do that part now if anyone wants to.

Looks great overall, just need to make sure we don't lose https://github.com/KirillOsenkov/SourceBrowser/commit/ad901faf42a0d3e516b7d856f626f427e724eee2

Not sure what magic you did, but the diff looks much cleaner now ;)

@KirillOsenkov I amended the commit that accidentally contained a revert for XmlSupport.cs, and then force pushed. I think what happened was that I had XmlSupport open with unsaved changes, then I rebased on top of your latest changes, then I saved the edit in VS which was based on XmlSupport before your latest changes.

Ah, I see. Cool.


Used ByteBuffer instead of arrays in Rasters

Used ByteBuffer instead of arrays in Raster objects to avoid usage of arrays and multidimensional arrays. This will allow us more optimized use of memory and possibility to use library in low-memory environment.

With the current apporach of having Number[], millions of objects are created on the heap. This is a problem especially on Android. Therefore I'd like to see this pull request merged which replaces arrays of Number by ByteBuffer.


Blank Map

My configuration
OS : ubuntu 18.04
GDAL: 2.4.2, released 2019/06/28
MapServer: 7.0.7
Link
http://localhost/cgi-bin/mapserv?map=/home/noaa_enc/map/SeaChart_DAY_BRIGHT.map&SERVICE=WMS&REQUEST=Getmap&VERSION=1.1.1&LAYERS=SeaChart_DAY_BRIGHT&srs=EPSG:4326&BBOX=133870,5798110,1541520,7459340&FORMAT=application/openlayers&WIDTH=2000&HEIGHT=1100
on my config I tried to run your script but in the end I got a blank page

Hello,
Thank you for your interest.
We will need a bit more details.
What command did you ran?
What was the output?
Is there any layers created in the resulting mapfiles?

the command
cd ../SMAC-M
python3 bin/generate_toml_config.py \
          --chart ../noaa_enc/ENC_ROOT \
          -rule-default-color IHO \
          --chartsymbols ./chart-installation/generate_map_files/resources/chartsymbols/chartsymbols_S57.xml \
          -enhancedchartdata ../noaa_enc/shp \
          --tablename Paper \
          --displaycategory Standard \
          --rule-set-path ./chart-installation/generate_map_files/resources/rules/ \
          -o configfile.toml

give me as outpu many line like that
+ ogr2ogr -append -skipfailures -f 'ESRI Shapefile' --config S57_PROFILE iw /home/gabin/swail/noaa_enc/shp/3/CL3_resare_lcase_POLYGON.shp -where PRIM=3 /home/gabin/swail/noaa_enc/ENC_ROOT/US3EC11M/US3EC11M.000 resare
+ ogr2ogr -append -skipfailures -f 'ESRI Shapefile' --config S57_PROFILE iw /home/gabin/swail/noaa_enc/shp/3/CL3_topmar_lcase_POINT.shp -where PRIM=1 /home/gabin/swail/noaa_enc/ENC_ROOT/US3EC11M/US3EC11M.000 topmar

the command
python3 ../bin/generate_shapefiles.py config.enc.noaa.toml

give me this output

python3 ../chart-installation/generate_map_files/generate_map_config.py config.enc.noaa.toml
Existing color tables found at /home/gabin/swail/SMAC-M/chart-installation/generate_map_files/resources/rules/color_tables
Existing layers found in /home/gabin/swail/SMAC-M/chart-installation/generate_map_files/resources/rules/layer_rules
/home/gabin/swail/SMAC-M/chart-installation/generate_map_files/mapgen/symbol.py:412: UserWarning: Pattern command not implemented: CI
  warnings.warn('Pattern command not implemented: ' + command)
Check geometry of all layers...
Loading DAY_WHITEBACK.csvON.shp
Loading NIGHT.csv
Loading DAY_BLACKBACK.csv
Loading DUSK.csv
Loading DAY_BRIGHT.csv
/home/gabin/swail/SMAC-M/chart-installation/generate_map_files/mapgen/symbol.py:412: UserWarning: Pattern command not implemented: CI
  warnings.warn('Pattern command not implemented: ' + command)

I am getting many layers in resulting map file ( there are 6 layer I thing )

@juliensam I tried this code on a virtual machine with the same configuration as yours (ubuntu 16.04) to finally get the same result, a white image.
I also tried to open the file maps with qgis and I get an error on the layers.

Is it possible for you to share a sample dataset?

yes at this link wget https://www.charts.noaa.gov/ENCs/All_ENCs.zip

in fact I used the documentation that is provided for the NOAA charts

you can get my map file at this link https://drive.google.com/file/d/1hivMS-Jlp81cCSLn4cu35lt7rKxhs_im/view?usp=sharing

Please check if the BBOX is correct both in term of location and zoom level.

Yes it was a problem with my BBOX. I found this site who help to compute BBOX
http://bboxfinder.com


Cache the Rank in GetUpper/LowerBound

After dotnet/coreclr#4993, I decided to take a look at the corert implementation of Array.GetLowerBound (and others) to see if any similar optimizations could be made here.
Changes:

Array

Port original optimizations from coreclr to GetLowerBound, GetUpperBound
Cache the rank in these methods as well
Array.Copy: Don't get the rank if the EETypePtrs are the same
Removed some branching by using a trick with uint wrapping (not sure if this is optimized by the C++ compiler)

MDArray

Massively reduced the size of the CheckBounds methods (multiple throw new ... means mutliple throws will be in the generated IL, not sure how the AOT compiler handles this)

Also avoided some branching using the aforementioned trick

Initialize: When you're checking if something is negative, you're really checking if it's highest bit is set. Therefore, we can just AND everything and check if it's != 0 to check if any of the arguments are negative (again, please let me know if this is taken care of by the AOT compiler and if I should revert this)

cc @jkotas

LGTM. Thanks!


Custom runtime chunk loader (because urls are unknown during build time)

Feature request
Load chunks from locations, which are unavailable during build time.
When uploading JavaScript assets to Hubspot CMS, the files are automatically minified, post-processed and pushed to a CDN, for caching reason Hubspot delivers each asset from a different location. So I am looking for a solution to load chunks from a location, which is not known during build time.
Find more details at the end.
What is the expected behavior?
Possibility to inject a custom chunk loader which provided the correct filename without the need of hacking the created JavaScript files.
What is motivation or use case for adding/changing the behavior?
During upload to Hubspot, JavaScript files are post processed and served from different locations for caching reasons.
How should this be implemented in your opinion?
Unclear.
Are you willing to work on this yourself?
yes
More details
when uploading JavaScript assets to Hubspot CMS, the files are automatically minified, post-processed and pushed to a cdn, for caching reason Hubspot delivers each asset from a different location. So I am looking for a solution to load chunks from a location, which is not known during build time.
Locally

js/widget.js
js/41.widget.js
js/393.widget.js

Remote

http://f.hubspotusercontent-eu1.net/hub/xxx/hub_generated/template_assets/50239061735/1656576587321/index.js
http://f.hubspotusercontent-eu1.net/hub/xxx/hub_generated/template_assets/50238026977/1656576587779/41.widget.js
http://f.hubspotusercontent-eu1.net/hub/xxx/hub_generated/template_assets/unknown/location/393.widget.js

And here the challenge starts, as 41.widget.js or 393.widget.js cannot be loaded, as they are not within the same url as index.js. Unfortunately, the final urls are not available during build time, the are created during upload to Hubspot.
During the upload to hubspot, some placeholders within the JavaScript assets are replaced (Hubspot Template language HubL). So, if I manage to define the output.publicPath for each chunk as https://{{get_asset_url('./js/<chunk>.js')}} Hubspot could replace this by the correct url.
I tried to manually patch the created index.js which does the trick, but seems a bit risky for me, unstable and hard to automate.
!function () {

    /* much more */

    o.u = function (e) {
        console.log("Load chunk");
        console.log(e);
        var result = "";
        if (e === 41) result = "{{get_asset_url('/theme/js/widgets/41.widgets.js')}}";
        if (e === 393) result = "{{get_asset_url('/theme/js/widgets/393.widgets.js')}}";
        console.log(result);
        return result;

    }, o.p = "https://",

    /* much more */
}();

for caching reason Hubspot delivers each asset from a different location

thats is useless if you using [hash] ( or other hash placeholders ) in output.. maybe there is setting for this..

So, if I manage to define the output.publicPath for each chunk as https://{{get_asset_url('./js/.js')}} Hubspot could replace this by the correct url.

you can post process assets replacing chunk.js to {{get_asset_url()}}


Introduce autostart field on ServiceInstallCtx

Hey Chip,
Here is another PR with a few changes we have found necessary for using the crate in our project. We have a release we need to get out by this Monday, so if it would be possible, a quick merge and publish on this would be greatly appreciated!

b24eb18 Obtain WinSW location from environment variable
If WinSW is not at any of the locations specified by the system or user Path variable, check the
WINSW_PATH variable for a potential location.
This is useful because it enables putting the WinSW binary in a location that is not on Path. On
Windows, by default there are no locations on Path the user can write to without administrative
privileges. Callers of the crate can obtain the WinSW binary and put it in any location they wish,
then set WINSW_PATH in the same process, and not have to worry about their users not having the
binary in a Path location.

17ad9cf Introduce autostart field on ServiceInstallCtx
BREAKING CHANGE: users who upgrade will need to explicitly add an autostart field into their
ServiceInstallCtx definition.
This controls whether a service should automatically start upon rebooting the OS. It's an option
common to all service managers and it's useful for developers to think about whether their services
should automatically start up, which I think justifies the breaking change. If the service is
resource intensive or uses a lot of bandwidth, some users actually don't want automatic start
because it can potentially render their machine unusable.
It could be that rc.d needs a little bit of additional work here. I am not very familiar with this
system.

For rc.d, the test is disabled because I had issues with the test runner. Will need to test that manually. Happy to merge for now.

Awesome, thank you! I accepted the invitation. My crates.io account is tied to my Github account, so it should be same username, jacderida.

jacderida

You should have gotten an invite for crates.io

Amazing, thanks! I will do a release and publish soon.


Reputation farming

Describe the bug
Users are able to farm reputation on their own questions and answers.
To Reproduce
Steps to reproduce the behavior:

Create a new threaded question
Post a message and convert it to an answer
Accept the answer and receive reputation reward

Expected behavior
Inability to accept your own answer on your own questions.
Screenshots

Invalid - No code is currently implemented to award rep upon accepting an answer.


Add counsel-outline to :not ivy-prescient-sort-commands

It doesn't make sense to sort the outline, better to leave the order as-is.

Thanks! (For the other PRs too)


update OHIF release and add multipart support

Hi @tangy5 , is this still an active PR?

Let's keep this PR for a moment, will decide to delete or merge it after EA4. Thanks

Closing as the doc is available elsewhere.


Combining Multiple Styles

As the title says, I'm curios to know if there's a way to combine or average the style of multiple speakers to synthesize a new target style? how do you think it should be implemented if it's possible to do so?

You can just do convex combinations of different styles. The style space is roughly a sphere so it is convex, so any convex combination of N styles is still a style and it represents some speakers as a combination of other speakers.


Tier 1: Remove Nav Condense Lag

So, when the nav shrinks up to it's smaller state, it takes like, a second after you begin scrolling - which sometimes fucks up the way it locks up with the product page sub navs.
Can we just have it shrink up immediately on scroll?

It's a 200 millisecond lag which I reduced to 100. I don't want to make it zero because you'll start to notice scrolling becomes a little shitty on mobile devices

roger that!


Prevent loading readline config on fennel stdio

This fixes #166. The bug was caused by readline inserting control
characters into the fennel repl. By skipping the user readline config,
they are prevented from modifying the repl in arbitrary dangerous ways.
This is fine, because the readline editing facilities are completely
useless in the context of conjure, since we rely solely on neovim's
editing capabilities.
Note: The user is still able to configure the lua readline library
through their ~/.fennelrc file, for example, to persist their history.

I now realise you aren't exposing this as an option, so it's totally fine as it is. I will merge and adjust it slightly so that all stdio clients benefit from this since it's not just limited to Fennel really. Thank you very much!

Hmm it doesn't seem to work for me :thinking: the fennel stdio client doesn't start up the REPL (no output) if I have :env set... will try to debug!

Fixed it I think, see https://github.com/Olical/conjure/commit/4eab13e0a60e047db7f87376a1fcb810ca8dbd1b
Not sure why it was required for me? Maybe NixOS? I need to explicitly inherit from the parent env, which is kinda okay but weird that there's no option in libuv to say "override and inherit" or "replace". I think this approach is okay? The default (NULL :env`) uses the parent env, setting it to anything totally replaces it.

Will wait to confirm you're happy then probably do the release dance :smile:

Yeah. By default it's inheriting the nvim environment and if env is set, it completely overrides it, so you need to merge with the current environment. Apparently it didn't break anything for me. Glad you caught it. I'm sorry, that could have been a nasty bug. Your improvement is alright for me.


Publish use ReadyToRun show the Microsoft.CodeAnalysis.VisualBasic.dll error

I use the asp.net core 3.0 ,,when i use ReadyToRun ,,the ide show me these errors:
compilation failed for "C:\Users\Administrator\.nuget\packages\microsoft.codeanalysis.visualbasic\1.3.0\lib\netstandard1.3\Microsoft.CodeAnalysis.VisualBasic.dll" (0x80004005)

Error compiling C:\Users\Administrator\.nuget\packages\microsoft.codeanalysis.visualbasic\1.3.0\lib\netstandard1.3\Microsoft.CodeAnalysis.VisualBasic.dll: Unknown exception

Optimizing assemblies for performance failed. You can either exclude the failing assemblies from being optimized, or set the PublishReadyToRun property to false.

my project file like this
<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>netcoreapp3.0</TargetFramework>
    <RuntimeIdentifier>win-x64</RuntimeIdentifier>
    <!--<PublishTrimmed>true</PublishTrimmed>-->
    <PublishReadyToRun>true</PublishReadyToRun>
  </PropertyGroup>

........

I never use vb,,,why it show these errors?How can I fix it

@kugarliyifan thanks for contacting us.
We believe this is an external issue. To provide a better and faster response I suggest you file the issue in https://github.com/dotnet/sdk

OK....thks


Bug: fungible tokens not recognised in post condition + rounded down to zero

Was testing the web wallet build <IP_ADDRESS>4 in preparation of Arkadiko testnet, and noticed the following. xUSD is not recognised in the post conditions (is a SIP10 token) and the amount is rounded down to 0. I was paying 0.009782 xUSD and it is displayed as 0.

just FYI, the transaction goes through successfully (so the post condition of 0.009782 xUSD is passed to the transaction), but it is shown incorrectly.

why is this being closed?  even though the transaction goes through, the information being displayed to the user is not helpful, if not not misleading.  The code should try to display enough significant digits, not just round.

@andresgalante was this addressed in a recent build or something?


Add Yarn to global NPM packages

Issue Type
Feature Idea
Summary
When using Node.js inside a Drupal VM you want to have a npm package manager available but npm is slow so people want to install yarn but cannot figure it out how to add it quickly to nodejs_npm_global_packages (just guessing because I also forgot the name first).
So what about switching from nodejs_npm_global_packages: [] to
nodejs_npm_global_packages:
  - name: yarn

I can totally understand if this is a customization you don't want to have in drupal-vm but I had to do this in every project because themes are using tools gulp etc. nowadays so yarn and a recent Node.js version (#1348) are always necessary.

I think I'd rather not include yarn at this time; every time I've considered adding a front-end library, I've sat and waited on the decision, and 3-6 months later, the tool of choice has changed.
Maybe instead, we could add a link to the node.js role docs, or a page in Drupal VM's docs explaining the variable better? That way no matter what tool(s) are needed per project, they can be added without someone having to figure out it needs to be name: yarn as an array item.

@yannickoo - Note that I made a change to the upstream role so you can now install packages globally by name or with a structured name/version object:
nodejs_npm_global_packages:
  - yarn # this is equivalent to `name: yarn` now!

See: https://github.com/geerlingguy/ansible-role-nodejs/issues/67

This is awesome! <3


Callback types

Hey!
Thanks for such great hooks!
I'm going to use your hooks in our projects, but some missing types make me unhappy, so I'm glad to help the project with PR;
Changes in the PR:

Keep callback args types, to do not lose them after passing my callback to the hooks
Use same types from React.useState()

Looks good to me, thank you! Will get new versions out today.

Published. Thanks again.


Catch socket.error exceptions when checking for GCE metadata

GAE's runtime monkey-patches calls to urllib2.urlopen, and instead of throwing URLError, it throws a socket.error when a URL cannot be resolved. This catches that error so it doesn't fall through and crash the app.

Current coverage is 96.19% (diff: 100%)

Merging #16 into master will increase coverage by <.01%

@@             master        #16   diff @@
==========================================
  Files            28         28
  Lines          2887       2888     +1
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           2777       2778     +1
  Misses          110        110
  Partials          0          0

Powered by Codecov. Last update e6de19d...1372ea0


Adding backup mechanism to report vulnerabilities in security.md

What this PR does / why we need it:
We have a dedicated mailing list for reporting vulnerabilities, however we had a situation earlier where this was not possible.
The CNCF recommends the use of the (GitHub vulnerability reporting mechanism)[https://docs.github.com/en/code-security/security-advisories/guidance-on-reporting-and-writing-information-about-vulnerabilities/privately-reporting-a-security-vulnerability] for projects, however this would make it not visible to the vendor security teams that help us with vulnerabilities. As a result we would like to have the mechanism available to people, but only as an alternative for when the mailing list cannot be used.
Checklist
This checklist is not enforcing, but it's a reminder of items that could be relevant to every PR.
Approvers are expected to review this list.

[x] Design: A design document was considered and is present (link) or not required
[x] PR: The PR description is expressive enough and will help future contributors
[ ] Code: Write code that humans can understand and Keep it simple
[ ] Refactor: You have left the code cleaner than you found it (Boy Scout Rule)
[ ] Upgrade: Impact of this change on upgrade flows was considered and addressed if required
[ ] Testing: New code requires new unit tests. New features and bug fixes require at least on e2e test
[x] Documentation: A user-guide update was considered and is present (link) or not required. You want a user-guide update if it's a user facing feature / API change.
[x] Community: Announcement to kubevirt-dev was considered

Release note:

NONE

/cc @stu-gott

/approve

/retest-required

I have confirmed that the Private Vulnerability Reporting mechanism is enabled for the KubeVirt repository.


[React Version] Add Animation on Scroll to Open Source Section

Reference

I want to work on this under JWOC

I want to work on this under JWOC

Assigned for 2-4 days. Extension available if required.


chore: adds _flaky suffix to flaky tests

Overview
As per our call, this PR adds a _flaky suffix to flaky tests identified so far.
Checklist

[x] New and updated code has appropriate documentation
[x] New and updated code has new and/or updated testing
[x] Required CI checks are passing
[x] Visual proof for any user facing features like CLI or documentation updates
[x] Linked issues closed with keywords

Codecov Report

Merging #1833 (0b2c17e) into main (5b35a01) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1833   +/-   ##
=======================================
  Coverage   21.76%   21.76%
=======================================
  Files         116      116
  Lines       13236    13236
=======================================
  Hits         2881     2881
  Misses      10063    10063
  Partials      292      292

@evan-forbes Can we confirm whether we have reached an agreement regarding the changes in this PR? If not, do we prefer to retain the current test names as they are? please let me know your decision! Thanks!


feat: add form for editing user details

@Kif-Fando please drop 7dad8f0 commit, it was a part of https://github.com/NeuraLegion/brokencrystals/pull/147


Improve test reliability on platforms with clock skew

On platforms with significant clock skew between performance.now()
and (new Date()).getTime(), tests that do not use spf.now() are
unreliable.  Fix this by using spf.now() in tests.
Also, standardize on (new Date()).getTime() instead of
(+new Date()), which was only being used in tests.

LGTM


Fix C13 lines CMake build and clang warnings. Update README.md

Fix CMake build by including new C13 and lines example source files.
Fix clang warnings on Windows and Linux.
README.md:

Update to reflect that C13 lines are supported.
Add C13 lines output example to Performance section.
Add short descriptions of Lines and Types examples.

Good work Luke, thanks a lot!


Suggest to fix number typo

@yoichinak : Thanks for your contribution! The author, @rothja, has been notified to review your proposed change.

@yoichinak Thank you for the correction!


bug 2005901: guard controller: create the pdb if it does not exist

The guard controller is missing the code that creates a PDB. Also, do not render the guard pod if its underlying node is not schedulable (tainted with NoSchedule:node.kubernetes.io/unschedulable).

/hold cancel

/lgtm
/approve


When it can support kafka 0.8.0?

Hi
When it can support kafka 0.8.0?
thx

No plans to support older versions of Kafka.  However, I am not sure if there are any Zookeeper specific changes between 0.8.0 and <IP_ADDRESS>.  Meaning you may be able to add your cluster as <IP_ADDRESS>.  Try it out on a dev cluster.

I try it on our prod cluster , Awesome ！
thanks


Chibuzo: Wireframing the different aspects of the app to show functionality

@Domicaren1 looking forward to seeing your wireframe for app functionality.

@Domicaren1 what part of the app functionality wireframe are you working on?

@Domicaren1 please let us know if you need help finishing this out, or if you are unsure about anything.

@istoga I wasn't so sure of how to go about it. But from yesterday's meeting I think all the needed wireframes have been done.

Closing this issue; no work to be done.


Use system account as creator of gdns provider app

Regular user cannot create gdns provider, because it's now launched
as a catalog app, and the chart for this app needs to list clusters.
So creating a system account with right permissions solves this.
The app needs to be launched only in local cluster. So we're using
the system account for local cluster which gets created during server
start if local cluster is enabled. We also provide cluster-owner role
to this system account after creating it from cluster controller, hence
if used as the gdns provider app's creator, it has all right permissions.
https://github.com/rancher/rancher/issues/19709

@mrajashree can you add a test.

@alena1108 Added the test

LGTM


MSVC fixes

With this PR, abseil can be built with CMake on Windows successfully.

NOMINMAX is required to prevent windows.h from defining min and max macros, which mess up with things like std::numeric_limits<T>::max() as seen in #34 (Also added to Bazel).
_CRT_SECURE_NO_WARNINGS is required to stop MSVC from complaining about the usage of insecure getenv over secure getenv_s. (Also added to Bazel).
Deprecated _sleep is replaced with Win32 Sleep function.
VS 2015 and above does not understand -std=c++11 or /std:c++11, default is /std:c++14.
CMAKE_CXX_FIX_MSVC is removed (not initialized by anyone).

The following code is needed in both abseil side and user side because add_subdirectory will not export CMAKE_CXX_FLAGS outside the directory.
add_definitions(/DNOMINMAX /DWIN32_LEAN_AND_MEAN=1 /D_CRT_SECURE_NO_WARNINGS)
add_compile_options(/W3 /WX /wd4005 /wd4068 /wd4244 /wd4267 /wd4800)

With google/googletest#1337, Abseil can be built with clang-cl on Windows as well.

Thanks for the PR. Sorry for the delay.
FYI, I'll need to run some test before I can merge this.


cast send returns server returned an error response: error code -32602: too many arguments, want at most 1

Component
Cast
Have you ensured that all of these are up to date?

[X] Foundry
[X] Foundryup

What version of Foundry are you on?
forge 0.2.0 (6179312 2024-06-10T00:18:46.177980000Z)
What command(s) is the bug in?
cast send
Operating System
macOS (Apple Silicon)
Describe the bug
I'm trying to send a simple transfer with the following command
cast s --private-key <<PRIV_KEY>> --rpc-url https://rpc.chiado.gnosis.gateway.fm --value 0.1ether <<ADDRESS>> "0x"
I also tried
cast s --private-key <<PRIV_KEY>> --rpc-url https://rpc.chiado.gnosis.gateway.fm --value 0.1ether <<ADDRESS>>
The server returns
Error:
server returned an error response: error code -32602: too many arguments, want at most 1

It most likely fails on eth_estimateGas, since when I specify --gas-limit it works fine. I also tried connecting to a local RPC where I have access to the logs to and I saw that estimateGas is called multiple times.
I downgraded foundry to find the first release where the error pops up. It's https://github.com/foundry-rs/foundry/releases/tag/nightly-cafc2606a2187a42b236df4aa65f4e8cdfcea970

This seems to be related to Chiado testnet. On gnosis chain mainnet it works fine. Will test more networks

could you share the logs RUST_LOG=trace cast...
too many arguments is weird, perhaps it's the state override arg
checking

log_foundry.txt

we're def sending a valid request:

{"method":"eth_estimateGas","params":[{"from":"0xc34b4858cf5f89916559fab2f864384d74b2f996","to":"0xc34b4858cf5f89916559fab2f864384d74b2f996","maxFeePerGas":"0xb368f487","maxPriorityFeePerGas":"0xb368f479","value":"0x38d7ea4c68000","input":"0x","chainId":"0x27d8"},"latest"],"id":2,"jsonrpc":"2.0"}

so it looks like the eth_estimateGas endpoint doesn't fully support the spec?

cast rpc web3_clientVersion --rpc-url "https://rpc.chiado.gnosis.gateway.fm/"
"erigon/2.60.1/linux-amd64/go1.21.5"

Thanks for digging into it. I will get in touch with the Erigon team.
So far I can confirm that it only occurred on Erigon clients. However, what bothers me, is that older foundry versions are compatible (as I mentioned April 24 and older)

Erigon v2.60.1 has a breaking change that removes the 2nd argument from eth_estimateGas. See https://github.com/ledgerwatch/erigon/pull/10495

The breaking change was reverted in Erigon v2.60.2.

we also fixed this on the client side and exclude the argument if missing


Unhealthy Internet connection would lead to infinite loop in library examples

Is your feature request related to a problem? Please describe.
Assume there is no healthy internet connection over the wifi that the ESP is connected to
then for each example written in this library, that any normal user would use to include in their project that surely has other functionality, there would be an infinite loop that the ESP would stuck in in the authHandler function
Describe the solution you'd like
Perhaps it's better to manage that by default a number of retries is set and for every retry, the interval to execute the next one would go higher and higher
Describe alternatives you've considered
GPT 4 offered this
void authHandler()
{
    static unsigned long lastAttemptTime = 0;
    static int attemptCount = 0;
    const int maxAttempts = 5;  // Maximum number of attempts
    const unsigned long retryInterval = 30000;  // Time between retries in milliseconds

    unsigned long currentMillis = millis();

    // Check if it's time to retry based on last attempt time and retry interval
    if (currentMillis - lastAttemptTime < retryInterval && attemptCount > 0) {
        return;  // Not yet time to retry
    }

    // Check if maximum attempts exceeded
    if (attemptCount >= maxAttempts) {
        Serial.println("Maximum authentication attempts reached. Please check the network settings or credentials.");
        return;  // Stop trying after maximum attempts
    }

    // Proceed with authentication if connected and attempts are within limits
    lastAttemptTime = currentMillis;
    attemptCount++;

    Serial.printf("Attempting authentication... Attempt %d\n", attemptCount);

    // Blocking authentication handler with timeout
    unsigned long ms = millis();
    while (app.isInitialized() && !app.ready() && millis() - ms < 120 * 1000)
    {
        JWT.loop(app.getAuth());
        printResult(aResult_no_callback);
    }

    // After attempting, check if authenticated
    if (app.ready()) {
        Serial.println("Authentication successful.");
        attemptCount = 0;  // Reset attempt count on successful authentication
    } else {
        Serial.println("Authentication failed. Retrying...");
    }
}

I don't see any issue from intensively testing 24/7 for three months since this library first released and half years of development in all network conditions.
The library is async and provides a timeout for sync and deInitialization which you can manage it easily.
The auth handler task was never run infinitely but retry every 5 seconds after failure.
The network reconnection is longer at 10 seconds if network is still disconnected.

Check the library examples for deInitialization.

Check the library examples for deInitialization.

The auth handler operation is natively asynchronous running.
The code you post above is for the synchronous usage that adapts from the async operation by running an async task in a loop.
The completely async examples are available for non-blocking code and I don't get why you request something like that instead of bypassing it.
Blocking or stopping the auth task from running is already requested by some user some months ago and I decide to provide him a more efficient approach by using the de-Initialization instead.
You can de-Initialize the app and remove the auth task from the Firebase services app efficiently any time you want in case you want to do some specific task and don't want the library to run for a while to save the memory.

You will see that the loop has timeout.
while (firebaseApp.isInitialized() && !firebaseApp.ready() && millis() - ms < 120 * 1000)
For non-blocking application, this is one of true async example which you should follow.
https://github.com/mobizt/FirebaseClient/blob/main/examples/RealtimeDatabase/Async/Callback/Get/Get.ino
You have many choices to do with this sync native library unless you don't understand the logic.
Please read the documentation thoroughly for elaborate usage.

The AppInitialization examples, the de-initialize app is already commented.
https://github.com/mobizt/FirebaseClient/blob/4417bcedbd7f8acd6e6ee1d09c9be054a9a633e9/examples/App/AppInitialization/Async/Callback/UserAuth/UserAuth.ino#L120-L124

The documentation.
https://github.com/mobizt/FirebaseClient?tab=readme-ov-file#deinitializeapp-or-firebasedeinitializeapp

I already wrote in the documentation. You should read it.

The migration is not possible because it is totally different from old library.


Make URLs count as 23 characters for Mastodon

I've seen this a few times though it's rare:
https://respublicae.eu/@nilstorvalds/109377434782905923
The link was truncated and the last word replaced with ….

It looks like the post is 507 characters long, so it was truncated as Mastodon's maximum is 500.
However, URLs (even unshortened ones) actually count as only 23 characters in Mastodon:
https://docs.joinmastodon.org/client/guidelines/
So, adjusting the truncating logic on the bot to account for this would be the best course of action.
In the case of this particular post it would have brought down the character count to 289, which wouldn't result in truncating it.
The problem is, even after accounting for this, in some extreme cases like quote tweets they could go over the character limit on Mastodon.
Twitter's limit is 280 characters, so just the body of a quote tweet could potentially be 560 characters long (280 * 2). Making truncation the only option there and potentially creating dead links in the process. Not much we can do there I can think of right now.
Anyway, counting URLs as 23 characters internally on the bot would at least mitigate this issue somewhat and make it run into it less frequently.

Il 20/11/22 21:44, robertoszek ha scritto:

However, URLs (even unshortened ones) actually count as only 23 characters in Mastodon:

In recent instances, the exact amount (which might be configurable?) is
advertised in the API https://respublicae.eu/api/v1/instance at
.configuration.statuses.max_characters and
.configuration.statuses.characters_reserved_per_url.

Right, we should honor those if they are present.
As always I need to find some time and finally get around to writing coverage tests for it, but this should do it:
70c91872cb0b17ee3a77e5703a2214c82bc2806c
Feel free to report back or provide any feedback if you encounter issues with it:
pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple pleroma-bot==1.1.1rc19

I think there might be something wrong with the regex, because I'm regularly getting errors of this sort:
2022-11-25 21:58:56,364 pleroma_bot ERROR: Exception occurred for user, skipping...
Traceback (most recent call last):
  File "/home/7/federico/mastodon/bot/lib/python3.9/site-packages/pleroma_bot/cli.py", line 589, in main
    tweets_to_post = user.process_tweets(tweets)
  File "/home/7/federico/mastodon/bot/lib/python3.9/site-packages/pleroma_bot/_processing.py", line 160, in process_tweets
    len_text = self._mastodon_len(tweet["text"])
  File "/home/7/federico/mastodon/bot/lib/python3.9/site-packages/pleroma_bot/_utils.py", line 538, in _mastodon_len
    text = re.sub(group, group[:char_count_url], text)
  File "/usr/lib/python3.9/re.py", line 210, in sub
    return _compile(pattern, flags).sub(repl, string, count)
  File "/usr/lib/python3.9/re.py", line 304, in _compile
    p = sre_compile.compile(pattern, flags)
  File "/usr/lib/python3.9/sre_compile.py", line 764, in compile
    p = sre_parse.parse(p, flags)
  File "/usr/lib/python3.9/sre_parse.py", line 948, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
  File "/usr/lib/python3.9/sre_parse.py", line 443, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
  File "/usr/lib/python3.9/sre_parse.py", line 549, in _parse
    raise source.error("unterminated character set",
re.error: unterminated character set at position 190

The nitter URLs are sometimes truncated too, specifically in RT with comment: https://respublicae.eu/@EC_StockholmRep/109406454535266575 .

The nitter URLs are sometimes truncated too, specifically in RT with comment: https://respublicae.eu/@EC_StockholmRep/109406454535266575 .

Oh, I see how that could've happened. Once we have processed the length in case of a signature or original date, we do a final check to see if it still needs to be truncated, which wasn't using the Mastodon's length to calculate it.
5fb6b96aeeb20075886f230c25807064bdf8ba3e
And the regex for finding the URLs seem to work but then fails for you when using the pattern of each URL to substitute them, so let's just try a simple string replace instead:
7e27c71a0eaf03fa1d9156e0c5583db12a63668e
I've added the changes to 1.1.1rc27:
pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple pleroma-bot==1.1.1rc27

Thanks, testing now.


Add documentation or config parameter for h2 enablement

Hi,
the Haproxy supports h2 frontend support and I tried to enable it with marathon-lb. I took a while but finally I was successful.
I created the following overwrite template HAPROXY_HTTPS_FRONTEND_HEAD file (including the leading and post empty line) and added into the template folder.
``
frontend marathon_https_in
bind *:443 ssl {sslCerts} alpn h2,http/1.1
mode http
``
Can you document it as example or add a config parameter to be able to configure alpn protocols?

If you keep the defaults ssl_fc_sni based routing and multi domains or wildcard certificates, you will face the http2 reuse problem, all backends using the same ssl certificate will be mixed.
https://discourse.haproxy.org/t/http2-reuse-problem/3286/4
As a workaround, use host header based templates, which IMO should be the marathon-lb defaults
cat HAPROXY_HTTPS_FRONTEND_ACL
  acl host_{cleanedUpHostname} hdr(host) -i {hostname}
  use_backend {backend} if host_{cleanedUpHostname}

cat HAPROXY_HTTPS_FRONTEND_ACL_WITH_AUTH
  acl auth_{cleanedUpHostname} http_auth(user_{backend})
  acl host_{cleanedUpHostname} hdr(host) -i {hostname}
  http-request auth realm "{realm}" if host_{cleanedUpHostname} !auth_{cleanedUpHostname}
  use_backend {backend} if host_{cleanedUpHostname}

cat HAPROXY_HTTPS_FRONTEND_ACL_WITH_AUTH_AND_PATH
  acl auth_{cleanedUpHostname} http_auth(user_{backend})
  acl host_{cleanedUpHostname} hdr(host) -i {hostname}
  http-request auth realm "{realm}" if host_{cleanedUpHostname} path_{backend} !auth_{cleanedUpHostname}
  use_backend {backend} if host_{cleanedUpHostname} path_{backend}

cat HAPROXY_HTTPS_FRONTEND_ACL_WITH_PATH
  acl host_{cleanedUpHostname} hdr(host) -i {hostname}
  use_backend {backend} if host_{cleanedUpHostname} path_{backend}

cat HAPROXY_MAP_HTTPS_FRONTEND_ACL
  use_backend %[req.hdr(host),lower,map_end({haproxy_dir}/domain2backend.map)]


"error while loading shared libraries" on starting interpreter for Remote SSH

Environment data

VS Code version: 1.58.2
Extension version (available under the Extensions sidebar): v2021.7.1060902895
OS and version: Remote SSH running Scientific Linux 7
Python version (& distribution if applicable, e.g. Anaconda): 3.8.3
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): N/A. installed on server without virtual environments.
Value of the python.languageServer setting: Pylance

Expected behaviour
When executing the command "Select Interpreter", interpreter is correctly set
Actual behaviour
Interpreter is not set, Python reports error: Failed to get interpreter information for '/path/to/python' [Error: Command failed: "/path/to/python" "$HOME/.vscode-server/extensions/ms-python.python-2021.7.1060902895/pythonFiles/interpreterInfo.py" /path/to/python: error while loading shared libraries: libpython3.8.so.1.0: cannot open shared object file: No such file or directory
The path to libpython3.8.so.1.0 is present in a directory set in $LD_LIBRARY_PATH (which should be generated correctly after .bashrc is sourced). When running the command "/path/to/python" "$HOME/.vscode-server/extensions/ms-python.python-2021.7.1060902895/pythonFiles/interpreterInfo.py" directly in shell, a json file containing interpreter information is correctly generated.
This is similar to #12255. However, no custom env-vars setting for the workspace is required for python to run in the shell.
Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

> pyenv root
Error 2021-07-26 04:08:25: Failed to get interpreter information for '/path/to/python' [Error: Command failed: "/path/to/python" "$HOME/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/interpreterInfo.py"
/path/to/python: error while loading shared libraries: libpython3.8.so.1.0: cannot open shared object file: No such file or directory

	at ChildProcess.exithandler (child_process.js:308:12)
	at ChildProcess.emit (events.js:315:20)
	at maybeClose (internal/child_process.js:1048:16)
	at Socket.<anonymous> (internal/child_process.js:439:11)
	at Socket.emit (events.js:315:20)
	at Pipe.<anonymous> (net.js:673:12)] {
  killed: false,
  code: 127,
  signal: null,
  cmd: '"/path/to/python" "$HOME/extensions/ms-python.python-2021.7.1060902895/pythonFiles/interpreterInfo.py"'
}
> conda --version
> python3.7 -c "import sys;print(sys.executable)"
> python3.6 -c "import sys;print(sys.executable)"
> conda info --json
> python3 -c "import sys;print(sys.executable)"
> python2 -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)" //NOTE::THIS IS THE CORRECT INSTALL
> /path/to/python -c "import sys;print(sys.executable)"
Error 2021-07-26 04:08:25: Failed to get interpreter information for '/path/to/python' [Error: Command failed: "/path/to/python" "$HOME/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/interpreterInfo.py"
/path/to/python: error while loading shared libraries: libpython3.8.so.1.0: cannot open shared object file: No such file or directory

	at ChildProcess.exithandler (child_process.js:308:12)
	at ChildProcess.emit (events.js:315:20)
	at maybeClose (internal/child_process.js:1048:16)
	at Socket.<anonymous> (internal/child_process.js:439:11)
	at Socket.emit (events.js:315:20)
	at Pipe.<anonymous> (net.js:673:12)] {
  killed: false,
  code: 127,
  signal: null,
  cmd: '"/path/to/python" "$HOME/extensions/ms-python.python-2021.7.1060902895/pythonFiles/interpreterInfo.py"'
}
>pyenv root
... ERROR REPEATS

Hi @JamesJieranShen, thank you for reaching out!
How do you connect to your remote machine, do you use the Remote - SSH VS Code extension?

Hi @JamesJieranShen, thank you for reaching out!
How do you connect to your remote machine, do you use the Remote - SSH VS Code extension?

Yes that’s correct.

which should be generated correctly after .bashrc is sourced

Would you mind checking if the extension picks it up correctly? For that, could you modify "$HOME/.vscode-server/extensions/ms-python.python-2021.7.1060902895/pythonFiles/interpreterInfo.py" directly to write the value of the LD_LIBRARY_PATH environment variable in a log file, and compare it with the value you get when printing it in a terminal?

Hello! I am unable to to do that, as the extension is unable to source libpython and was not able to run the script at all.
However, I do believe that I have fixed the issue. In this server cluster python is installed on a drive shared by multiple computing notes. It appears that this error occurs when there is a lingering vscode-server instance running in another node, reading this library. Not exactly sure why that happens. This can be solved with killing the vs-code-server.

Thank you for reporting back that you managed to find a workaround! Feel free to report the issue of the lingering vscode-server instance on the Remote SSH repo if you think there's something we could fix.
In the meantime, I will close this issue since it isn't cause by the extension, but feel free to leave a comment and we can reopen it 🙂


fix(button): ghost button styled  incorrect font color

fix #208

:tada: This PR is included in version 5.3.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 5.3.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


MM-13405 Change warning message to be singular when adding last user to GM

Summary
Change warning message in the UI to be singular when adding last user to GM
Ticket Link
MM-13405
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] Added or updated unit tests (required for all new features)
[x] Has UI changes
[x] Includes text changes and localization file updates

Device Information
This PR was tested on: [IOS emulator]
Screenshots

@mkraft https://github.com/yahoo/react-intl/wiki/API#formatplural has a note about multiple language support and not using formatplural in those cases.
and one more thing is we don't use FormattedText directly in app. we use a customised version of it in app/components/formatted_text.js. If  does indeed work fine i think we need to create another similar component for our use case here.


MTProxy Telegram Bot AD_TAG

Always getting:
Sorry, an error has occurred during your request. Please try again later.
How to fix that?

Try it on another account. The account must have been created before. Most do not respond with virtual numbers.

Try it on another account. The account must have been created before. Most do not respond with virtual numbers.

sorry about aksing this here but about ad tag if we have more than 1 user on same port is that ad tag apply to both or ?

Thanks for the hint. It worked on one account and not the other. Weird.
Both are accounts don't really have anything weird to them.

Not working for 4+ accounts

5+ accounts, even ask people to help. Only Sorry, an error has occurred during your request. Please try again later.Sorry, an error has occurred during your request. Please try again later.Sorry, an error has occurred during your request. Please try again later.Sorry, an error has occurred during your request. Please try again later.Sorry, an error has occurred during your request. Please try again later.Sorry, an error has occurred during your request. Please try again later.Sorry, an error has occurred during your request. Please try again later.Sorry, an error has occurred during your request. Please try again later.Sorry, an error has occurred during your request. Please try again later.


Improve project setup, add extra configuration in details page, and train in review page

This PR implements many changes and refactors:

[x] Make use of Context for project_id in SetupDialog
[x] #1665
[ ] Training of the first iteration of the model is moved to the review page. This makes it possible to work on a quick project setup later on, where you can start screening right after the dataset is uploaded.
[x] Completely redesigned details page
[x] Option to change model in the details page
[x] Option to add tags in the details page
[x] Refactor the setupDialog dialog
[x] Reduce the number of unnecessary API calls
[x] Reduce the number of state changes
[x] Remove custom details page components
[x] Integrate prior knowledge selection better with SetupDialog
[x] Screen without prior knowledge
[x] Option to see what model was used for training the displayed record

The save and next button are integrated now when project info is stored:

This is useful because updating this info should not auto-update because of the server-side checks like a project-title check. This required several changes (including a visual one), such as making the Stepper part of the DialogHeader.

As the entire dialog is now optional, there is no need to collapse the accordion anymore.

I implemented an indicator to show the details of the model used for ranking the displayed record. The presented record has a green indicator (ModelTraining icon from Material) with info when you hover when the record comes from a specific model. The presented record comes from the random or top-down stack when the indicator is orange.
I'm planning a red indicator when model training has failed.

Model switching and projects without prior knowledge:

"Tag categories" are now "Tag groups". This makes more sensw given the implementation imo.

The "I can't wait" button is available when the model is training. This button offers the option to screen random or top-down.
I'm working on the review screen when a project without prior knowledge (and therefore is not training) is presented. Picking one of these indicators results in orange indicators during screening until enough info is found to train a model.


Accessibility on Input field (design)

Error and Disable states
Detailed description
We were having a discussion here about the Error and Disable states for Input does not look accessible having only Color (in the case of error) or Transparency (in case of Disable).
Maybe would be better to add an Icon and .... line for disable?
If the person has any problems with vision, such as color blind or bad sight might get tricky on both states.
I made this exploration comparing both:

Oh nice!
One of the visual designers at Hybrid Cloud pointed out this to me yesterday, so we were talking about how would be usable for them. They are trying to incorporate Carbon on their UI as we speak. And this was a problem for them.
About the disable state, the dashed line for the 50% opacity is for example used by Google apparently, to point disable fields not just with making them transparent.
;)

Icon will also be added when input is updated in Carbon X, so going to close this now


Alleviate image size limitation

Current implementation compress the image into 800px / jpeg due to the dynamodb item size limitation (400KB).
Possible Solution
If the conversation size is large, store it to S3

#124


AffNet features & patch extraction

Hi,
I have happily noticed that you are using kornia :)

Could we be even more useful for you? I have recently created a wrapper around OpenCV detectors into kornia_moons, so one can easily combine them with modern patch descriptors, or even AffNet affine shape, like this example, which combines AKAZE features with AffNet affine shape and then describes them with HyNet descriptor.

import cv2
import kornia.feature as KF
from kornia_moons.feature import *
detector = OpenCVDetectorWithAffNetKornia(cv2.AKAZE_create(threshold=3e-3))
feature = KF.LocalFeature(detector, KF.LAFDescriptor(KF.HyNet(True)))

lafs, resps, desc = feature(timg)

Here is a result of such matching

Is there any reason to use floor instead of round? If you observe that it leads to the better results, we can fix it directly in kornia

https://github.com/cvg/Hierarchical-Localization/blob/master/hloc/extractors/dog.py#L33

Hi Dmytro,
It seems that we share the same goal of providing a standard interface for feature extraction and matching :)

We are pretty happy that kornia now provides GPU-accelerated patch extraction and patch descriptors - we care a lot about speed (now we run hloc on large datasets) so this is very helpful. Interfacing with OpenCV is cool, maybe that will come handy later on.
One major remaining pain point is SIFT - for now we use the VLFeat implementation, but it's too slow for large images. It would be great if kornia's DoG detector and SIFT descriptor could match its accuracy. I am aware of this PR but know that a good implementation of SIFT requires a ton of work so I can't afford to look into this myself.

I was getting a lower performance when using kornia.feature.laf.extract_patches_from_pyramid instead of your extract_patches package, so I looked for differences and found out that floor vs round was the only one:

# extract_patches.core.convert_xyA/convert_cv2_keypoints
pyr_idx = max(0, int(math.log(s,2)))
# vs kornia.feature.laf.extract_patches_from_pyramid
pyr_idx = (scale.log2() + half).relu().long()

My intuition is that round over-blurs patches while floor retains some sharpness. Happy to send a PR if deemed useful.
cc @mihaidusmanu

@Skydes PR is not needed, I wanted confirmation :)
https://github.com/kornia/kornia/pull/1616

@Skydes since the thing was fixed in kornia, how about removing code from hloc and rely on kornia instead? I can make a PR if needed.

Thanks for the reminder, done in https://github.com/cvg/Hierarchical-Localization/pull/216


Improve Test Coverage (100%)

Do the checklist before filing the issue:

[x] Do you have Node.js and all the relevant dev-dependencies installed?
[ ] Is this a bug fix?
[x] Is this an enhancement?
[ ] Is this a feature request?

NOTE: Provide a clear and concise description of the feature that needs to be added! Or if its a bug, then provide the necessary steps to reproduce it along with screenshots.

Give your answer below:

Provide more unit tests so as to increase the code coverage upto 100%

Screenshots are provided below:

Steps to reproduce:
In the project directory, make sure you have all the dev-dependencies installed! If not then run npm i -D
For generating the test reports just type npm run test.

Hey @pbiswas101, may I work on this issue?

@thisisabdus Of course! You're assigned

This issue is OPEN again for it to be claimed!

can I work on this?

@devRD You're assigned but remember the PR will only be merged if you increase the coverage up to 100%


The string did not match the expected pattern in isomorphic fetch

am new to react,and am getting this error in fetch... Can you help me to resolve this

what response you guys need?

Thank you for bringing this to our attention. However, it's not quite clear how this 'react' inquiry is related to TestCafe. If you believe that it is related to TestCafe and it is an issue, please fill the bug template and update the description with full information about your problem, so we can examine it and put you on the right track.
If you simply want to ask a question, I recommend that you forward it on StackOverflow - an amazing platform for users to ask and answer questions. We try to keep the GitHub issue tracker for bugs and feature requests only.


Have a unique constraint enforced in the DB for attendance

Avoids a race condition

I don't think this is doable, since we do want to allow "duplicate" records for teams.