NHibernate v5

Just to let you know as I am not using Redis, NHibernate v5 has been published some weeks ago and is not binary compatible with previous versions.
Caches projects need to implements new async methods. A minimal implementation can be seen in nhibernate/NHibernate-Caches@f800358435f6c051cb2596a4aa5a831df66368d7, where they just fallback on synchronous calls.
Your fork base has an ongoing PR on the subject: TheCloudlessSky/NHibernate.Caches.Redis#31. There is an alternate PR too but I do not consider it a viable long term option:  TheCloudlessSky/NHibernate.Caches.Redis#33.

Thanks for the contribution! I'm currently not using NHibernate, but good to know if that changes.


fix TweenManager not work after Director.restart

After call Director.restart, scheduler will clear all updates.

I made a fix.

got it, i will test it tonight


[Mozilla add-ons site] You have wrong order of the screenshots in addons.mozilla.org

Actual order of screenshots: 2019 > 2016 > 2017 > 2013 > 2012
Expected order of screenshots: 2019 > 2017 > 2016 > 2013 > 2012
Here: https://addons.mozilla.org/en-CA/firefox/addon/customtube/

That was a deliberate decision. There a 4 watch page screenshots and 1 homepage screenshot. I thought it would look better with the homepage screenshot in the middle.
Please slow down with the issues. The majority of them are issues that I'm aware of and/or are not a big deal.

The real problem I have here is the sheer amount of issues you've created in the past day or two.
Not every little "problem" needs it's own issue. It floods the issue page, and makes it very unorganized, which is not very helpful. It's also pretty annoying honestly. If truly feel the need to point out all these issues, I'd recommend you make one or two issues that cover everything.
A perfect example of this would be #10, which pointed out several things in just one issue, instead of creating like 10 separate issues.
How about this: Make one issue for bugs, and another issue for Feature requests. If you do it that way, it would be much more helpful. I'm not saying never create new issues, but most of your issues have fallen into one of these two categories anyway, with the exception of the PocketTube one, which was definitely worth it's own issue.
You can leave the current issues as is.

But issues should be reported.

You're right, I'm sorry for my last statement. I guess I was feeling pressured to respond to all the issues as quickly as possible, which is just putting unnecessary stress on myself. That's probably the real reason I was getting annoyed. Really, this is my first time having a project that so many people care about, so I guess I'm still figuring out how exactly I'm supposed to run it.
But I definitely shouldn't be stopping people from reporting issues. I realize that now.
I would still encourage you to include multiple similar issues in one single issue (#22 and #23 would've been better as a single issue), but if you can't, that's fine. Just understand that it might take a day or 2 for me to respond from now on.
I'm also thinking of making a list of issues I know about. Is that a good idea?

Fine, and if the issue won't be an issue, but question, then I will post it rather to “Discussions”.
And if two issues would be similar, then I will rather open a single issue.

I'm also thinking of making a list of issues I know about. Is that a good idea?

Maybe meta bug would be good. Meta bugs should have prefix [Meta], and meta bugs are different than normal bugs.
Example of meta bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1751505

Fine, and if the issue won't be an issue, but question, then I will post it rather to “Discussions”.

And if two issues would be similar, then I will rather open a single issue.

Sounds good!

Maybe meta bug would be good. Meta bugs should have prefix [Meta], and meta bugs are different than normal bugs.

Example of meta bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1751505

Okay, I'll consider that.


WIFI-2671: Added codes 12 & 13 for rebbot & factory reset upgrade opt…

Signed-off-by: Owen Anderson<EMAIL_ADDRESS>We decided it would be easier for the AP to send up a different status code to the cloud instead of the cloud keeping track of what operation it started on the AP.
A change will need to be made on the cloud side to trigger a different message box on the UI

Can we rename the patch? 8)

Whoops 🤦

Merged 208b0165, thanks


ImportError: No module named 'django.utils.importlib'

Django 1.9.7 compatibility issue in following files:
Unhandled exception in thread started by <function check_errors..wrapper at 0x108d071e0>
Traceback (most recent call last):
File "/Users/admin/Desktop/django/da/lib/python3.5/site-packages/django/utils/autoreload.py", line 226, in wrapper
fn(_args, *_kwargs)
File "/Users/admin/Desktop/django/da/lib/python3.5/site-packages/django/core/management/commands/runserver.py", line 109, in inner_run
autoreload.raise_last_exception()
File "/Users/admin/Desktop/django/da/lib/python3.5/site-packages/django/utils/autoreload.py", line 249, in raise_last_exception
six.reraise(__exception)
File "/Users/admin/Desktop/django/da/lib/python3.5/site-packages/django/utils/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/Users/admin/Desktop/django/da/lib/python3.5/site-packages/django/utils/autoreload.py", line 226, in wrapper
fn(_args, **kwargs)
File "/Users/admin/Desktop/django/da/lib/python3.5/site-packages/django/init.py", line 18, in setup
apps.populate(settings.INSTALLED_APPS)
File "/Users/admin/Desktop/django/da/lib/python3.5/site-packages/django/apps/registry.py", line 108, in populate
app_config.import_models(all_models)
File "/Users/admin/Desktop/django/da/lib/python3.5/site-packages/django/apps/config.py", line 202, in import_models
self.models_module = import_module(models_module_name)
File "/Users/admin/Desktop/django/da/lib/python3.5/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 986, in _gcd_import
File "", line 969, in _find_and_load
File "", line 958, in _find_and_load_unlocked
File "", line 673, in _load_unlocked
File "", line 662, in exec_module
File "", line 222, in _call_with_frames_removed
File "/Users/admin/Desktop/django/da/lib/python3.5/site-packages/endless_pagination/models.py", line 11, in
from endless_pagination import (
File "/Users/admin/Desktop/django/da/lib/python3.5/site-packages/endless_pagination/loaders.py", line 6, in
from django.utils.importlib import import_module
ImportError: No module named 'django.utils.importlib'

Same issue, django = 1.9.7

Can anyone guide me how to solve this error i am getting same error in django = 1.10.8 ?


detect firefox developer edition

I am currently using Firefox Developer Edition (aka Quantum) in mac, and the path for the application is /Applications/FirefoxDeveloperEdition.app instead of just /Applications/Firefox.app.
So, here is my small contribution. I am new to GO so let me know if there is something I could do better.
Thank you for this awesome project

Hi! Thanks for contributing!
Before I can review this change, I'll need you to take care of the CLA. It's a quick document about what can be done with your code, fairly painless to sign.
https://github.com/FiloSottile/mkcert/blob/master/CONTRIBUTING.md#contributor-license-agreement
Thank you!

Hi @FiloSottile sorry for the very late response. We signed the CLA yesterday, I am glad you added these changes directly :)
Thanks again


Any click event on the gantt chart?

Do you have the click event on the bar of gantt chart? if I select one task and I would like to open another detail panel. thanks, George

Yes, there is a lot of events that you can configure on ngGantt
Check in this example by vEvents
https://github.com/jsGanttImproved/jsgantt-improved/blob/master/Documentation.md


App crash in SFO Lounge room

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Opened the lounge chat
Saw some messages

Expected Result:
Chat are loaded fine
Actual Result:
App crashes
Workaround:
Can the user still use Expensify without this being fixed? Have you informed them of the workaround?
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: v1.3.4-0
Reproducible in staging?: needs reproduction
Reproducible in production?: needs reproduction
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation

Expensify/Expensify Issue URL:
Issue reported by: @johnmlee101
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1682358290819999
View all open jobs on GitHub

Bug0 Triage Checklist (Main S/O)

[ ] This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
[ ] This bug is not a duplicate report (check E/App issues and #expensify-bugs)

If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead

[ ] This bug is reproducible using the reproduction steps in the OP. S/O

If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
If the reproduction steps aren't clear and you determine the correct steps, please update the OP.

[ ] This issue is filled out as thoroughly and clearly as possible

Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.

[ ] I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

Asking for a video to reproduce - https://expensify.slack.com/archives/C049HHMV9SM/p1682373378812259?thread_ts=1682358290.819999&cid=C049HHMV9SM

This just happened to me, @johnmlee101, and @neil-marcellini, all within a minute or two.

:wave: Friendly reminder that deploy blockers are time-sensitive ⏱ issues! Check out the open StagingDeployCash deploy checklist to see the list of PRs included in this release, then work quickly to do one of the following:

Identify the pull request that introduced this issue and revert it.
Find someone who can quickly fix the issue.
Fix the issue yourself.

My theory here is that after we stopped sending the participant list from the back-end to the front-end, personal details have to get up to date and during that delay, when someone that has never posted in a public room leaves a message, if their personal details are not stored in Onycx, it crashes.

We need to provide some reasonable defaults in the front end wherever we're passing personal details around.

taking a look at this right now.
    return (allPersonalDetails && allPersonalDetails[login]) || {
        login,
        displayName: Str.removeSMSDomain(login),
        avatar: getDefaultAvatar(login),
    };

this is called inside ReportUtils.getDisplayNameForParticipant() which is invoked in DetailsPage.js.
@jasperhuangg is my understanding correct that the login info of new users in the lounge chat room is not populated when the report chat is opened?

@hayata-suenaga No, the login should be populated, I think the issue is that the new user's entry in the personal details doesn't exist yet because the bedrock job hasn't run yet. Looking into this now

   ` let details = lodashGet(this.props.personalDetails, login);`

   I think we can add a default here after login?

It's confusing. We're already giving default after the above line...
        if (!details) {
            details = {
                login,
                displayName: ReportUtils.getDisplayNameForParticipant(login),
                avatar: ReportUtils.getAvatar(lodashGet(details, 'avatar', ''), login),
            };
        }

I think here, we should return early if the personalDetails is undefined or null.

This just happened to me, johnmlee101, and neil-marcellini, all within a minute or two.

@sakluger do you remember what you were trying to do when the app crashed?

Based on the stack traces provided (screenshot in OP, screenshot in Sasha's comment) we've opened two PRs that should prevent the crashes from occurring:

https://github.com/Expensify/App/pull/17984
https://github.com/Expensify/App/pull/17988

My theory here is that after we stopped sending the participant list from the back-end to the front-end, personal details have to get up to date and during that delay, when someone that has never posted in a public room leaves a message, if their personal details are not stored in Onycx, it crashes.

For posterity, I've tried this flow on both staging and production and haven't been able to trigger the crash.

Reviewing label has been removed, please complete the "BugZero Checklist".

The solution for this issue has been :rocket: deployed to production :rocket: in version 1.3.5-6 and is now subject to a 7-day regression period :calendar:. Here is the list of pull requests that resolve this issue:

https://github.com/Expensify/App/pull/17988

If no regressions arise, payment will be issued on 2023-05-03. :confetti_ball:
After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.

[ ] External issue reporter
[ ] Contributor that fixed the issue
[ ] Contributor+ that helped on the issue and/or PR

As a reminder, here are the bonuses/penalties that should be applied for any External issue:

Merged PR within 3 business days of assignment - 50% bonus
Merged PR more than 9 business days after assignment - 50% penalty

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

[ ] [@jasperhuangg / @hayata-suenaga] The PR that introduced the bug has been identified. Link to the PR:
[ ] [@jasperhuangg / @hayata-suenaga] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
[ ] [@jasperhuangg / @hayata-suenaga] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
[ ] [@jasperhuangg / @hayata-suenaga] Determine if we should create a regression test for this bug.
[ ] [@jasperhuangg / @hayata-suenaga] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.
[ ] [@Christinadobrzyn] Link the GH issue for creating/updating the regression test once above steps have been agreed upon:

The solution for this issue has been :rocket: deployed to production :rocket: in version 1.3.6-0 and is now subject to a 7-day regression period :calendar:. Here is the list of pull requests that resolve this issue:

https://github.com/Expensify/App/pull/17988

If no regressions arise, payment will be issued on 2023-05-04. :confetti_ball:
After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.

[ ] External issue reporter
[ ] Contributor that fixed the issue
[ ] Contributor+ that helped on the issue and/or PR

As a reminder, here are the bonuses/penalties that should be applied for any External issue:

Merged PR within 3 business days of assignment - 50% bonus
Merged PR more than 9 business days after assignment - 50% penalty

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

[ ] [@jasperhuangg / @hayata-suenaga] The PR that introduced the bug has been identified. Link to the PR:
[ ] [@jasperhuangg / @hayata-suenaga] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
[ ] [@jasperhuangg / @hayata-suenaga] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
[ ] [@jasperhuangg / @hayata-suenaga] Determine if we should create a regression test for this bug.
[ ] [@jasperhuangg / @hayata-suenaga] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.
[ ] [@Christinadobrzyn] Link the GH issue for creating/updating the regression test once above steps have been agreed upon:

It doesn't look like this issue was handled by an external contributor or C+. So nothing to pay in Upwork.
@jasperhuangg do you think we need a regression test for this?

I think we couldn't find clear reproduction steps for this issue. Has the root cause of this identified? I though an post mortem email about issues with SF Lounge #announce room. Is that one related to this one?

@Christinadobrzyn I don't believe we need regression tests for this.

Awesome! thanks @hayata-suenaga I'll close this one as complete since there's no regression test and no one external to pay.


Use ci.py explicitly in docs building instructions

This adds ci.py to the docs to make it more clear how to easily build the docs locally. This also re-arranges CI following the merging of all CI steps to run concurrently since there's no need to run the Sphinx precheck during GPU unit tests. This still preserves it though in the docs step as a way to quickly bail out if there are formatting errors so the full tutorials don't get built. Doing that surfaced a bunch of warnings in the full docs build that were previously just ignored (hence the extra warnings in the grep regex)
Ignored warnings from: https://ci.tlcpack.ai/blue/organizations/jenkins/tvm/detail/PR-9971/1/pipeline/
cc @areusch

Thanks @driazati :smile_cat:


Implement Team Manager to aid with team rules and balancing

This will replace the state-integrated team stuff with a proper team manager.

Distribution should allow for a percentage of players to be seekers on all team distribution modes.


change how mongodb stores image and thumbnail urls

Rather just change them to URI's...

this is comletly reworked, closing.


Milestones Ralesia MK2/3/4 entries broken after update.

Mod source
Factorio Mod Portal
Operating system

=Windows 10

What kind of issue is this?

[X] Compatibility
[ ] Locale (names, descriptions, unknown keys)
[ ] Graphical
[ ] Crash
[ ] Progression
[ ] Balance
[ ] Pypostprocessing failure
[X] Other

What is the problem?
Something updated in Milestones or Pyanodon regarding the Ralesia Mk2/3/4 recipes makes Milestones fail to recognize them.  Every other entry works fine, but if you open Milestones, you can find Ralesia Mk2/3/4 and see that they are now "missing item," basically.  It's probably an easy fix, but I can't find a single place where Pyanodon creates its Milestones preset to point out exactly where it is, or I'd just make a pull request.  Is it being made dynamically by a script?
Screenshots below.
Steps to reproduce

Install full Pyanodon's suite.
Install Milestones mod
Start a new game
You may need to change the milestones preset to "Full Pyanodon's."
The errors will print, and the three missing entries can be seen by opening Milestones interface and scrolling down.the

Additional context

Log file
No response

fixed


updating install.bat to allow installation from local path on windows

please take a look @smurawski
note: i couldn't make the powershell block pretty, seems as if tac (`) doesn't work as a carriage return escape with -command or perhaps ruby is eating.. dunno

Gents, let me know when you have something to test.
I'm trying to have this setup work in kitchen to avoid downloading the msi from the internet.
  name: chef_zero
  require_chef_omnibus: 12.0.3
  chef_omnibus_url: file:///c:/share/chef/chef-client-12.0.3-1.msi

Vagrant is not showing this 'vagrant-omnibus' plugin for me. Is it shipped with kitchen?
Vagrant 1.7.1
[10:10:17 /]$ vagrant plugin list
vagrant-share (1.1.4, system)
[10:10:44 /]$```

@alexpop I dont think vagrant-omnibus is shipps with chefdk or kitchen. I just add it via
vagrant plugin install vagrant-omnibus
The code in my original pull request works - though it's intended for windows only. It's actually what I'm using at the moment to get my 2k8 vagrant instances updated.
config.vm.synced_folder '~/Documents/ISO_BOX_etc/binaries', "c:/parent_binaries"
config.omnibus.install_url = 'c:\\parent_binaries\\chef_client'

I suspect for Linux instances below will work (it's what's in the read me right now)
config.omnibus.install_url = '/some/local/path'

No, vagrant-omnibus is not shipped nor used by Test Kitchen, see my comment here for more details.


Add upload of sourcemaps

This PR fixes #

Coverage decreased (-0.1%) to 34.15% when pulling dacd162c393ca32e7ad7dfa0497c22a361f2869b on sourcemaps into 65a31de166ae5ac3a3b34e84ac669d41b296f0fc on master.


Get-CsUserPolicyAssignment performance degradation

On the same machine this function takes 2-2.5 seconds to run with version 1.1.6, 14.5-15.5 seconds with version 2.0.0, and 4-4.5 seconds with version 2.3.0.
Since there is not currently a way to get group-based policy assignment data for multiple users using a single command, I am forced to query user by user. With 19,000 active users the speeds in 1.1.6 were barely fast enough to query all users within 12 hours. My specific use case could tolerate up to about 18 hours, but not the slower speeds in all 2.x versions.

Hello @gregg79 how are you? Hope you are doing good 👍
Here, we are limited to documentation fixes and improvements, for your particular inquiry the best would be to open a support ticket directly from your tenant.
Thank you

@get-itips Thank you very much for the contribution and sharing this explanation. @gregg79 Hope this comment is helpful for you. Considering the nature of the issue, as advised, please open a service ticket in your tenant and follow with them for the resolution. We proceed here to close it.
We will be very happy if you can share the outcome or resolution with us if you see documentation update is required. We will certainly update this documentation with that valuable information. Thanks again, for taking out some time to open the issue. Appreciate and encourage you to do the same in future also.


The result tip some FAILED SCHEDULE AUDIT.

When I excute sh benchmark.sh, the result tip some FAILED SCHEDULE AUDIT.
I want know, perhaps some reson.

To pass the schedule validation, the total number of queries that are executed on time need to be at least 95% [<IP_ADDRESS> of the spec].
To reduce the number of late operations, the time compression ratio needs to be increased.

David will get back to you regarding the technical details of this error. In the meantime, one comment from me:
Based on your issues, it seems that you are working towards getting an audited benchmark result. If this is the case, I strongly encourage you to reach out to<EMAIL_ADDRESS>to discuss this further – I'll be happy to provide assistance with the auditing process.


Building with verbose=no on Windows with Visual Studio duplicates lines in standard output

When building Godot (tested with https://github.com/godotengine/godot/commit/69bec86028f87307e549d7a2f49bbb7e2b1f3771 and Visual Studio 2017, can also be reproduced with VS 2015) with verbose=no passed to SCons, each compiled file will appear twice in the log. One of them will be colored and formatted, one of them won't. See below for screenshot.

This makes the console output less compact and readable than it ought to be.

I recently found out thanks to this article that Ninja filters those output messages out of the box: https://github.com/ninja-build/ninja/blob/master/src/clparser.cc#L67
I wonder if it's possible to do the same in SCons. Surely cl.exe's standard output should be redirected to a stream which can then be filtered?

Still valid in 84abf5a

That seems to be a SCons+cl.exe issue then and not something to fix in Godot. Since we haven't managed to work it around in 4 years I doubt it will happen now, so closing. Switching to Meson might solve it ;)


SubscribeF and parent components

@jonsterling
Trying to free up subscribe / EventSource so we can use it in parent components (see #234) is proving troublesome.
Currently EventSource can only be used in a useful way when g ~ Aff partially due to produce from Control.Coroutine.Aff, but really there's not much else that makes sense if we build the coroutine here. This doesn't work for parent components as they force g ~ QueryF.
I thought one fix would be just to capture the arguments we construct the EventSource with and defer the coroutine construction until we're interpreting it, as then we don't force Aff everywhere - it happens at the end, which is fine, as it has to be Aff there anyway for runUI. The problem with that is the row of effects need to go somewhere:
data EventSourceF f eff a
  = EventSourceF ((a -> Eff eff Unit) -> Eff eff Unit) (a -> Eff eff (f Unit))

-- Also need to hide the `a`, but there's no problem here
newtype EventSourceE f eff = EventSourceE (Exists (EventSourceF f eff))

We don't want to add an additional type variable to Component to capture these effects as it'll make the types even worse than they already are, and there will be duplication as the row will usually be present in a component's g as well.
So I figured I'd use existential hiding (and unsafeInterleaveEff later... :cry:), but I think this is safe, because we can ensure that the existential version it has to be constructed like this:
subscribe :: forall s f g eff. (MonadEff eff g) => EventSourceE f eff -> Free (HalogenF s f g) Unit
subscribe = ...
  -- there's a mkExists thing in here that disappears the `eff`, but because of
  -- the constraint we know that `g` must be able to deal with them

However, I'm not sure this is safe: we can re-interpret a component's g, so there's no guarantee that future gs will represent the effects we hid here. But then maybe it's not a problem as Eff's rows are additive (excluding ST and other FFI hacks to remove effects)?

Thanks for writing this up. I'm still trying to think through it, but when I glanced at your last sentence I became a little concerned. If safety relies on rows being strictly additive, I don't think that we can exclude things like ST or FFI hacks. Typing is a semantic / extrinsic property, and the particular set of rules which gives rise to the behavior of effect rows in the typechecker is a somewhat arbitrary approximation of the semantic typing relation. An FFI hack is a way to add a new rule to the syntax/proof theory which is sound wrt. the semantics; there are numerous cases where removing an item from an effect row is a valid thing to do, even if it's not treated in the built-in proof theory.
I guess my concern is whether your proposal is essentially predicated on the assumption that the inductive definition of effect typing which is in the type checker is actually complete for the intended semantics of the language (which it cannot be).

That's exactly my worry too, but I thought about it a bit more after writing that, I'm not sure considering the behaviour of effect rows is actually helpful here after all: we interpret a component by providing a natural transformation, therefore I if I understand correctly, it is definitely safe as any g' we end up with has to have a mapping from g.

OK, I think that makes sense.
Could you show me where you would have to unpack this existential and use unsafeInterleaveEff?

I haven't worked that bit out yet, I'll put something together now. I'm thinking it will be in the driver/runUI though, as that's where we'll build the coroutine based on the functions that the EventSource originally captured.

  CR.Consumer (f Unit) (Aff (avar :: AVAR | eff)) Unit
  -> EventSource2 f (Aff (avar :: AVAR | eff))
  -> Aff (avar :: AVAR | eff) Unit
makeItSo c es2 =
  SCR.runStallingProcess (runEventSourceE (runEventSource2 es2) $$? c) $> unit
  where
  runEventSourceE :: EventSourceE f (avar :: AVAR | eff) -> EventSource f (Aff (avar :: AVAR | eff))
  runEventSourceE (EventSourceE esf) = runExists runEventSourceF esf
  runEventSourceF :: forall a. EventSourceF f (avar :: AVAR | eff) a -> EventSource f (Aff (avar :: AVAR | eff))
  runEventSourceF (EventSource attach handle) = eventSource attach handle
  runEventSourceF (EventSource_ attach handle) = eventSource_ attach handle

Also, not that I'm necessarily recommending it, but you can always define IO in PureScript (possibly based on Aff), which would support lifting from any possible set of effects. Then Halogen could just expect an IO at the top level.

So, one question is—in this setting, if I've already got a coroutine (from some other source), I can't use the subscribe machinery anymore to consume it. However, I guess I should be able to just consume it in the normal way from inside eval so long as I have got the right effects. Does that sound right?

Yeah, that's right. I hadn't realised that earlier, but that's exactly what you're doing with the file input, isn't it? The case I was trying to solve with this (updating a tick on an interval) can probably also be done with the direct coroutine approach. This just gains us the ability to subscribe in parent components.

Cool; I think it sounds good then! Now that I think of it, directly using the coroutine makes more sense in the context I'm doing it anyway.

This got fixed at some point, it is now possible to subscribe in parent components.


constructor error expiration

delete:       new workbox.expiration.Plugin() and use     new ExpirationPlugin(config)
line 30
https://github.com/google-developer-training/pwa-training-labs/blob/master/sw-precache-workbox-lab/solution/app/sw.js
https://developers.google.com/web/tools/workbox/reference-docs/latest/module-workbox-expiration.ExpirationPlugin
best regards Tjark

Thanks Tjark. This lab is indeed a bit outdated - Workbox version 3 is used, while version 5 is current. I don't have time currently to update the API, but I will leave this issue as a reminder.
Thanks again!


Issue in Benchmark  example [4625]

I'm trying to run Benchmark example with following parameters：
./Benchmark subscriber tcp
./Benchmark publisher tcp
But I got errors in publisher terminal：
2019-01-31 14:27:49.871 [RTCP Error] Cannot find an available logical port. -> Function PrepareAndSendCheckLogicalPortsRequest
2019-01-31 14:27:50.392 [RTCP Error] Cannot find an available logical port. -> Function PrepareAndSendCheckLogicalPortsRequest
2019-01-31 14:27:50.835 [RTCP Error] Cannot find an available logical port. -> Function PrepareAndSendCheckLogicalPortsRequest
2019-01-31 14:27:51.892 [RTCP Error] Cannot find an available logical port. -> Function PrepareAndSendCheckLogicalPortsRequest
......

Hi @Yale323,
seems that you found a misconfiguration bug in this example.
I'll fix it cleaning it a bit and will prepare a PR that you will be able to test on your side too.
Sorry for the inconveniences, and thank you very much for opening the issue!

Here you have the fix: #408

Here you have the fix: #408

Thanks，I checkout branch release/1.7.1,   and run "cmake -DTHIRDPARTY=ON -DBUILD_JAVA=ON -DCOMPILE_EXAMPLES=ON -DPERFORMANCE_TESTS=ON -DSECURITY=ON .."
then make, I got following error:
[  0%] Generating Java application

Task :idl:compileJava
warning: [options] bootstrap class path not set in conjunction with -source 1.6
Note: /home/yale/Fast-RTPS/Fast-RTPS/thirdparty/idl/src/main/java/com/eprosima/idl/generator/manager/TemplateUtil.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

Task :compileJava
warning: [options] bootstrap class path not set in conjunction with -source 1.6
/home/yale/Fast-RTPS/Fast-RTPS/fastrtpsgen/src/com/eprosima/fastrtps/fastrtpsgen.java:32: error: cannot find symbol
import com.eprosima.idl.parser.typecode.Kind;
^
symbol:   class Kind
location: package com.eprosima.idl.parser.typecode
/home/yale/Fast-RTPS/Fast-RTPS/fastrtpsgen/src/com/eprosima/fastrtps/idl/grammar/Context.java:21: error: cannot find symbol
import com.eprosima.idl.parser.typecode.Kind;
^
symbol:   class Kind
location: package com.eprosima.idl.parser.typecode
/home/yale/Fast-RTPS/Fast-RTPS/thirdparty/fastcdr/src/java/com/eprosima/fastcdr/idl/generator/TypesGenerator.java:23: error: cannot find symbol
import com.eprosima.idl.parser.typecode.Kind;
^
symbol:   class Kind
location: package com.eprosima.idl.parser.typecode
3 errors

FAILURE: Build failed with an exception.

Yes, we detected the issue a moments ago.
The branch release/1.7.1 is going to update IDL-Parser, but the master branch doesn't have some develop's commits yet.
I created the PR to solve this issue, but meanwhile, you can try it switching manually your IDL submodule to develop (you may stage the modified submodule to avoid CMake revert the changes).
Best regards.

Hi @Yale323,
the fix is merged in the release/1.7.1 branch. Can you test it?

Hi @LuisGP
It works well，thanks for the bug fix.


Update to parse yaml

This is an update for the rename in https://github.com/web-platform-tests/wpt/pull/11201/files
@tobie PTAL. I might have missed something or need to test something that I removed from your old tests.

This LGTM.

Thanks!


Build fails on macOS 10.14.6 Mojave

Simplest program depending on libssh2-sys = "0.2.16" and doing libssh2_sys::init(); fails on macOS 10.14.6 Mojave.
Here is the project: https://github.com/idubrov/openssl-macos-issue
For convenience, this is all it is:
Cargo.toml
[package]
name = "openssl-macos-issue"
version = "0.1.0"
edition = "2018"

[dependencies]
libssh2-sys = "0.2.16"

src/main.rs:
fn main() {
    libssh2_sys::init();
}

Steps to reproduce: run cargo run on macOS
Results:
   Compiling openssl-macos-issue v0.1.0 (/Users/idubrov/Projects/openssl-macos-issue)
error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-m64" "-L" "/Users/idubrov/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib" "/Users/idubrov/Projects/openssl-macos-issue/target/debug/deps/openssl_macos_issue-2efbd7ed99d011e5.14oph2ryrcic2ns5.rcgu.o" "/Users/idubrov/Projects/openssl-macos-issue/target/debug/deps/openssl_macos_issue-2efbd7ed99d011e5.1dts6u7a2usqigcg.rcgu.o" "/Users/idubrov/Projects/openssl-macos-issue/target/debug/deps/openssl_macos_issue-2efbd7ed99d011e5.3ozgcq2xcp4pi75n.rcgu.o" "/Users/idubrov/Projects/openssl-macos-issue/target/debug/deps/openssl_macos_issue-2efbd7ed99d011e5.3x3q5atvk5pzpzxq.rcgu.o" "/Users/idubrov/Projects/openssl-macos-issue/target/debug/deps/openssl_macos_issue-2efbd7ed99d011e5.77err7e5ernzm9f.rcgu.o" "-o" "/Users/idubrov/Projects/openssl-macos-issue/target/debug/deps/openssl_macos_issue-2efbd7ed99d011e5" "/Users/idubrov/Projects/openssl-macos-issue/target/debug/deps/openssl_macos_issue-2efbd7ed99d011e5.2nymup00na9yzsl0.rcgu.o" "-Wl,-dead_strip" "-nodefaultlibs" "-L" "/Users/idubrov/Projects/openssl-macos-issue/target/debug/deps" "-L" "/usr/lib" "-L" "/Users/idubrov/Projects/openssl-macos-issue/target/debug/build/libssh2-sys-e12fc67748fd4726/out/build" "-L"<EMAIL_ADDRESS>"-L" "/Users/idubrov/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib" "/Users/idubrov/Projects/openssl-macos-issue/target/debug/deps/liblibssh2_sys-11046a02422a3a81.rlib" "/Users/idubrov/Projects/openssl-macos-issue/target/debug/deps/libopenssl_sys-5e28547054fc718d.rlib" "/Users/idubrov/Projects/openssl-macos-issue/target/debug/deps/liblibz_sys-f8ff374315e8a85c.rlib" "/Users/idubrov/Projects/openssl-macos-issue/target/debug/deps/liblibc-17765c000cbeb2f7.rlib" "/Users/idubrov/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libstd-b7d0d4c6440e3596.rlib" "/Users/idubrov/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libpanic_unwind-2fa12511d8ee5824.rlib" "/Users/idubrov/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libhashbrown-29d15f8ede79c49d.rlib" "/Users/idubrov/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_alloc-4f15d040f73e4150.rlib" "/Users/idubrov/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libbacktrace-4fcf576fa458096e.rlib" "/Users/idubrov/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libbacktrace_sys-2406d6e5b831dd2b.rlib" "/Users/idubrov/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/librustc_demangle-067abb57e8a16c4c.rlib" "/Users/idubrov/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libunwind-1e20b38d5a3d0391.rlib" "/Users/idubrov/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcfg_if-f2707e3d7ac3a5d7.rlib" "/Users/idubrov/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liblibc-ffa668c5204aaff0.rlib" "/Users/idubrov/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/liballoc-8baa8dc315a5da10.rlib" "/Users/idubrov/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_core-aa91dfbfa1d19d0e.rlib" "/Users/idubrov/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcore-6d8e604ca9ec56d7.rlib" "/Users/idubrov/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib/libcompiler_builtins-5eb0926740e75669.rlib" "-lz" "-lssl" "-lcrypto" "-lz" "-lSystem" "-lresolv" "-lc" "-lm"
  = note: Undefined symbols for architecture x86_64:
            "_OPENSSL_init_ssl", referenced from:
                openssl_sys::init::_$u7b$$u7b$closure$u7d$$u7d$::h50edd338b29a354d in libopenssl_sys-5e28547054fc718d.rlib(openssl_sys-5e28547054fc718d.openssl_sys.f1funnwq-cgu.8.rcgu.o)
          ld: symbol(s) not found for architecture x86_64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

error: aborting due to previous error

error: could not compile `openssl-macos-issue`.

To learn more, run the command again with --verbose.

Investigating the cause revealed that wrong openssl library is being linked.
Note that it passes "-L"<EMAIL_ADDRESS>which is the version I want to link to. However, in front of it, there is "-L" "/usr/lib", which makes linker to look in system directory and link to the version of OpenSSL shipped with macOS (which is 0.9).
Running cargo -vv run reveals the following piece of configuration emitted by libssh2-sys build:
[libssh2-sys 0.2.16] cargo:rustc-link-search=native=/usr/lib

Which, I think, is the cause for the error above.
To confirm, I added -Xlinker -v argument to the "cc" command above and got the following listing:
BUILD 18:57:17 Dec 13 2019
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
Library search paths:
	/Users/idubrov/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib
	/Users/idubrov/Projects/openssl-macos-issue/target/debug/deps
	/usr/lib
	/Users/idubrov/Projects/openssl-macos-issue/target/debug/build/libssh2-sys-e12fc67748fd4726/out/build
<EMAIL_ADDRESS>	/Users/idubrov/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/x86_64-apple-darwin/lib
	/usr/lib
	/usr/local/lib

Note how /usr/lib is included twice -- once before<EMAIL_ADDRESS>and once after. I think, the latter is added by default. However, the former one, I think, is caused by libssh2-sys build.
To summarize:

libssh2-sys build adds /usr/lib as a search path for libraries.
openssl-sys build adds<EMAIL_ADDRESS>as a search path for libraries. It, however, goes after /usr/lib.
openssl-sys uses includes from<EMAIL_ADDRESS>(correct), but when final binary is linked, it is linked against openssl in /usr/lib (incorrect).

I think this is an issue with libssh2-sys build (but could be caused by downstream libraries like, pkg_config).
(also, reported to openssl crate for visibility https://github.com/sfackler/rust-openssl/issues/1252)

This also happens on macOS 10.12 Sierra

This may be the same root cause as https://github.com/alexcrichton/ssh2-rs/issues/174
I've just published 0.2.17 to address that other issue; could you try with that and let me know if this is still a problem?

Assuming that this is resolved.

Oops, sorry, didn't notice previous notification. Yes, indeed it works now! Thanks a lot!


Add single function that kicks off all HIFLD Grid building

:rocket:

[x] Is your feature request essential for your project?

Describe the workflow you want to enable
I wish there was a single user-facing function that could orchestrate all lower-level functions to build HIFLD grid data.
Describe your proposed implementation
The functions inputs should only be pass-throughs to lower-level functions, e.g. the higher-level functions within prereise.gather.griddata.hiflddata.data_process.

Closed via #236.


the channel between teacher featuremap and student featuremap maybe different

Hello!  when trainning the Dnet,  the channel between teacher featuremap and student featuremap maybe different ,how to deal with it?

添加1*1卷积链接到GAN
---Original---
From<EMAIL_ADDRESS>Date: Mon, Jul 29, 2019 18:34 PM
To<EMAIL_ADDRESS>Cc<EMAIL_ADDRESS>Subject: [p517332051/GAN-Knowledge-Distillation-SSD] the channel between teacher featuremap and student featuremap maybe different  (#1)
Hello!  when trainning the Dnet,  the channel between teacher featuremap and student featuremap maybe different ,how to deal with it?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

谢谢！


Improved best-effort nature of connection warmup.

API change check
API changes are not detected in this pull request.


Improve key code to key name mapping.

Summary of the Pull Request
As a fallback, use the key detected by the WebUI.
Use the key detected by WebUI and our mapping only for from_settings call.
PR Checklist

[x] Applies to #783
[x] CLA signed. If not, go over here and sign the CLA

Validation Steps Performed
Manual

There is still an issue when using Alt + other keys. Those are not detected correctly by the WebView (and Edge) and are reported only by their key code. Unfortunately, there does not seem to be any way to get the key name from js. This will probably be solved when we move to WebView2, since it works correctly in Chrome.


[rv_core_ibex] Fix a fetch enable assertion

The SVA failed because it did not take the latched fetch disable state into account.
Fixes #15899
Signed-off-by: Michael Schaffner<EMAIL_ADDRESS>

i'm going to merge this first so we can get some results based on it tomorrow.


Clicking on "cluster explorer" hyperlink in CIS Scan page, user is redirected to logging in dashboard

What kind of request is this (question/bug/enhancement/feature request): bug
Steps to reproduce (least amount of steps as possible):

Navigate to Cluster --> Tools --> CIS Scans
Click on cluster explorer in the page

User is redirected to https://<rancher-server>/dashboard/c/c-cchgg/logging

Result:
Other details that may be helpful:
Environment information

Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI): master-head - commit id: 8fd81a9
Installation option (single install/HA): HA

Cluster information

Cluster type (Hosted/Infrastructure Provider/Custom/Imported): custom rke

Fixed the ember incorrect re-direct issue with https://github.com/rancher/ui/pull/4176, which I tested, but there are still dashboard changes that need to happen:
From the ember UI, the link goes to https:///dashboardc/local/cis it goes directly to the Cluster Explorer page instead of going to the 2 uses cases with this redirect.
It should mimic what is happening in monitoring: https://github.com/rancher/dashboard/blob/master/pages/c/_cluster/monitoring/index.vue
The test cases are:

Hit the link when backup/restore app hasn't been installed yet - direct to the app deployment page in dashboard to deploy backup and restore operator
Hit the link when backup/restore app has been installed - direct to the default product page in dashboard for backup and restore operator


Routing protobuf payloads to gRPC services

Title: Routing protobuf payloads to gRPC services
Description:
At Spotify we've been looking into allowing clients to call gRPC backend services by implementing something similar to gRPC Fallback. This would allow us to support unary calls from platforms where gRPC isn't available or for some reason suboptimal from the clients perspective, while still retaining type safety from Protobuf and its serialization semantics.
Initially we investigated using grpc_http1_bridge_filter for bridging HTTP 1.1 clients to native gRPC backends, but that would require every client application to prepend the gRPC frame as specified by the wire format, which isn't ideal organizationally. Additionally, the bridge does not the allow the usage google.api.http for mapping paths to RPC handlers (a functionality currently supported in the grpc_json_transcoder filter).
Our ideal scenario would be bypassing the transcoding semantics in grpc_json_transcoder  whenever Content-Type is set to application/x-protobuf or similar, and instead simply forwarding the request body (a protobuf message) to the designated gRPC method (based on path). Alternatively, the grpc_http1_bridge_filter could be extended to perform the gRPC framing internally depending on Content-Type, but that approach wouldn't allow us to map paths and verbs with google.api.http annotations.
This is something that would allow us to leverage Envoy more heavily in our backend systems, and we'd be happy do discuss contributing a patch if this feature request is sound from the project's perspective.

@mattklein123 @alyssawilk @euroelessar any opinions on this?

For the goal of skipping the transcoder filter based on content type, I thought snow did work here (https://github.com/envoyproxy/envoy/issues/12968) based on a similar desire to turn filters on and off by header value (https://github.com/envoyproxy/envoy/pull/14722)  but I didn't follow along the whole chain.  @snowp can you advise?

@alyssawilk I'm not sure I understand how that would help in our case, I think that PR tries to achieve something else if I understand correctly.
What we're trying to do is allowing HTTP 1.1 clients to call gRPC by sending the request protobuf as the payload as an alternative to JSON. For some of our applications serializing protobufs is not an issue, and their pain point is using gRPC libraries in their platforms, so something we can still leverage protobuf in these cases.
As far as I understand we can use the http1_bridge to solve one part of the problem, but each client would have to send a payload following the gRPC wiring format (0x0 + content-len + protobuf payload). What we'd like to do is moving the gRPC framing to the proxy layer, so that the client would only send the protobuf payload instead.
The JSON transcoding filter may also be of help in case this "bypassing" of the transcoder could also respect the same routes defined with google.api.http options in the protoset descriptor. If not that could be confusing to clients, as the path and verb mapping would only apply to the JSON-payloads but not protobuf.
Does that make sense? Please let me know if I can help clarifying anything else about the idea.

Re skipping filter execution we can do that with the matching API + SkipAction: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/matching/matching_api.html?highlight=extensionwithmatcher#filter-integration
re http1_bridge I don't see any problem with adding support to the http1_bridge filter to inject the gRPC frame (the same way the reverse bridge manages it) if someone is willing to do that work

@snowp glad to hear injecting the framing sounds reasonable to you.
re SkipAction: wouldn't that also bypass the google.api.http annotations? Consider this example:
service ConcatService {
    rpc Concat(ConcatRequest) returns (ConcatResponse) {
        option (google.api.http) = { get: "/v1/concat" };
    }
}

The json transcoder would map /v1/concat to the Concat RPC, but I believe skipping the filter means the URL wouldn't be mapped and the verb and path would be different depending on content-type (if that's how we determine the payload type). Is there any way to respect this mapping for protobuf as well? If not, would it make sense to add something for that?

It doesn't feel plausible to implement bypass-transcoding logic in json-grpc-transcoder as many of its features rely on a full ownership/control over the generated protobuf message.
For example, one can specify url as /v1/{a.b.message_id}, which assumes that a.b.message_id field is unconditionally set to the corresponding field, which is non-trivial to ensure if an original message body is supplied by the caller.
Arguably, I'd recommend to stick to gRPC call notation as close as possible, which is what http1_bridge does for you. It is understandable that managing frames can be a hustle if your language of choice does not have a proper grpc library support, in which case it's reasonable to add a logic to generate frame size for you for unary-unary calls from the content length.
Are there particular business/technological reasons which would require/encourage you to use non-grpc call notation?

That makes a lot of sense @euroelessar. I hadn't considered dynamic routes and was actually only considering static mapping. It shouldn't be a big problem to stick to gRPC notation, but what I was thinking about was the case where a grpc service is exposed both through JSON transcoding and the "protobuf over http1" approach I was proposing. It may be a bit confusing that the same RPC requires different paths depending on what serialization format you're invoking from the client, but it does sound unfeasible considering the example you mentioned (and presumably other generation issues).
I'll discuss this with my team next week and see when we can get started writing a patch for the http1_bridge , as I'm happy to hear it doesn't sound too crazy to introduce it.


Integration of RL

Too good amount of effort, tried using Q learning, tree searches, value and policy iterations but due to GPU and computational limitations, limited the project to only MinMax with a good evaluation function


DR2-1833 Remove department and series from output/sfn input

This is not causing any problems as such but we’re sending
department and series into the step function input and we’re
not using it. It’s safest to take it out so we’re not tempted
to use it and to make sure it’s not being picked up somewhere unexpected.

@MancunianSam, looks good. I left two comments on my review about the README ☝, you might have missed them.

@techncl I missed a bit off the json in the README. Could you check this one again please.


Bug: ScriptTask conversion uses scriptFormat header instead of language

Describe the bug
If a script task is converted, the header used to define the scripting language is scriptFormat, just like it was in Camunda 7.
This is not compatible with the zeebe-script-worker which requires language
Steps to reproduce

Define a Camunda 7 script task with inline script and script format
Convert it
The scriptFormat has become the header scriptFormat
Run the process with the zeebe-script-worker
it fails as no language is present

Expected Behaviour
The scriptFormat is translated to language
Library version
0.6.1
Camunda Platform 7 version
7.20.0
Camunda Platform 8 version
8.3.0

As a follow-up: I've seen the issue mentioned in the release 0.8.0, and I'm wondering if the dependency to the script-worker, that is introduced here, is documented anywhere?
Maybe it's worth to provide an example for script migration? And explain which docker contain to integrate to the cluster.


Sending cardinals

I have BTC non-inscripted I want to send to address
It's sitting at txid:1
This doesn't work:
ord wallet send address txid:1
and this just sends 10000 sats
ord wallet send address txid:1:0
Either there's a command I missed, or I think should be added.
Basically my hopes and dreams are a send-cardinal command which takes amount and uses non-inscripted utxos to send that amount to address

As far as i know, you can just use

ord wallet send   --fee-rate y
Xsats example is 100000sats (yes need to define unit)

And ord wallet will not send out any inscribed ordinals.
Havent tried with picking specific utxos through ord. (Can always import the wallet to sparrow and use careful utxo and control management to spend specific utxo)

That command looks incomplete, can you fix it? or point me to documentation, code, or tests, because I didn't see it anywhere

Sorry not sure why it cut off.
"Ord wallet send bc1pzzz 200000sats --fee-rate 10"
You can try
"Ord wallet send --help"

Be careful with this. It isn't documented and doesn't work very well.

It ignores the --fee-rate that you give it.

It leaves all your inscriptions locked so that ord wallet inscription doesn't list them any more

If you have any inscriptions in the wallet you can only send a cardinal once. When you try to send another you get the "output already locked" error.

I'm wondering whether this was left undocumented because it isn't ready for use.

Be careful with this. It isn't documented and doesn't work very well.

It ignores the --fee-rate that you give it.
It leaves all your inscriptions locked so that ord wallet inscription doesn't list them any more
If you have any inscriptions in the wallet you can only send a cardinal once. When you try to send another you get the "output already locked" error.

I'm wondering whether this was left undocumented because it isn't ready for use.

Yes, it ignores the fee rate.
Cardinals remained will be locked, and need to unlock them everytime u sent anything
$ bitcoin-cli -rpcwallet=ord lockunspent true

It uses the --fee-rate now and can be used.


Unable to load file

Maybe related to this.  When using in the context of a binary file an error is thrown.
Example:
with open ("./that.pdf", 'rb') as f:

    elements = partition_pdf(
            file=f,
            strategy='hi_res',
            is_image=False,
            include_page_breaks=True,
            analysis=True,
            infer_table_structure=True,

        )

Error:
PDFPageCountError: Unable to get page count.
I/O Error: Couldn't open file 'C:\Users\LavoriV\AppData\Local\Temp\tmp_fqqu798': No error.

Hi @vlavorini - could let use know what versions and operating system you're using and share an example file we could use to reproduce?

I'm on Windows, with Unstructured version 0.14.2. Here the file I use
population in EU.pdf

Thanks, @vlavorini !

I also encountered this issue.
Is there any progress or update on this matter?

@andy1213aa can you post the code you used and also mention whether you are on Windows?

I used the same code provided above by @vlavorini, and yes, I am also running it on Windows.
Is there currently a way to solve this problem on Windows? Thank you!

@MthwRobinson do you have any updates on the matter?

@simonschoe can you post a stack trace?

from unstructured.partition.pdf import partition_pdf

with open("testfile_with_images.pdf", 'rb') as f:
    base64str = base64.b64encode(f.read()).decode('utf-8')

file_bytes = base64.b64decode(base64str)
file_bytes = io.BytesIO(file_bytes)

doc_elements = partition_pdf(
    file=file_bytes,
    #filename="testfile_with_images.pdf",
    languages=['deu'],
    strategy="hi_res",
    hi_res_model_name="yolox",
)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
File ~\pdf2image\pdf2image.py:602, in pdfinfo_from_path(pdf_path, userpw, ownerpw, poppler_path, rawdates, timeout, first_page, last_page)
    [601](~/pdf2image/pdf2image.py:601) if "Pages" not in d:
--> [602](~/pdf2image/pdf2image.py:602)     raise ValueError
    [604](~/pdf2image/pdf2image.py:604) return d

ValueError:

During handling of the above exception, another exception occurred:

PDFPageCountError                         Traceback (most recent call last)
Cell In[6], [line 9](vscode-notebook-cell:?execution_count=6&line=9)
      [6](vscode-notebook-cell:?execution_count=6&line=6) file_bytes = base64.b64decode(base64str)
      [7](vscode-notebook-cell:?execution_count=6&line=7) file_bytes = io.BytesIO(file_bytes)
----> [9](vscode-notebook-cell:?execution_count=6&line=9) doc_elements = partition_pdf(
     [10](vscode-notebook-cell:?execution_count=6&line=10)     file=file_bytes,
     [11](vscode-notebook-cell:?execution_count=6&line=11)     #filename="tests/testfile_with_images.pdf",
     [12](vscode-notebook-cell:?execution_count=6&line=12)     #include_page_breaks=True,
     [13](vscode-notebook-cell:?execution_count=6&line=13)     languages=['deu'],
     [14](vscode-notebook-cell:?execution_count=6&line=14)     strategy="hi_res", # hi_res
     [15](vscode-notebook-cell:?execution_count=6&line=15)     hi_res_model_name="yolox", # "yolox_quantized", "yolox_tiny"
     [16](vscode-notebook-cell:?execution_count=6&line=16) )

File ~\unstructured\documents\elements.py:605, in process_metadata.<locals>.decorator.<locals>.wrapper(*args, **kwargs)
    [603](~/unstructured/documents/elements.py:603) @functools.wraps(func)
    [604](~/unstructured/documents/elements.py:604) def wrapper(*args: _P.args, **kwargs: _P.kwargs) -> list[Element]:
--> [605](~/unstructured/documents/elements.py:605)     elements = func(*args, **kwargs)
    [606](~/unstructured/documents/elements.py:606)     call_args = get_call_args_applying_defaults(func, *args, **kwargs)
    [608](~/unstructured/documents/elements.py:608)     regex_metadata: dict["str", "str"] = call_args.get("regex_metadata", {})

File ~\unstructured\file_utils\filetype.py:706, in add_filetype.<locals>.decorator.<locals>.wrapper(*args, **kwargs)
    [704](~/unstructured/file_utils/filetype.py:704) @functools.wraps(func)
    [705](~/unstructured/file_utils/filetype.py:705) def wrapper(*args: _P.args, **kwargs: _P.kwargs) -> list[Element]:
--> [706](~/unstructured/file_utils/filetype.py:706)     elements = func(*args, **kwargs)
    [707](~/unstructured/file_utils/filetype.py:707)     params = get_call_args_applying_defaults(func, *args, **kwargs)
    [708](~/unstructured/file_utils/filetype.py:708)     include_metadata = params.get("include_metadata", True)

File ~\unstructured\file_utils\filetype.py:662, in add_metadata.<locals>.wrapper(*args, **kwargs)
    [660](~/unstructured/file_utils/filetype.py:660) @functools.wraps(func)
    [661](~/unstructured/file_utils/filetype.py:661) def wrapper(*args: _P.args, **kwargs: _P.kwargs) -> list[Element]:
--> [662](~/unstructured/file_utils/filetype.py:662)     elements = func(*args, **kwargs)
    [663](~/unstructured/file_utils/filetype.py:663)     call_args = get_call_args_applying_defaults(func, *args, **kwargs)
    [664](~/unstructured/file_utils/filetype.py:664)     include_metadata = call_args.get("include_metadata", True)

File ~\unstructured\chunking\dispatch.py:74, in add_chunking_strategy.<locals>.wrapper(*args, **kwargs)
     [71](~/unstructured/chunking/dispatch.py:71) """The decorated function is replaced with this one."""
     [73](~/unstructured/chunking/dispatch.py:73) # -- call the partitioning function to get the elements --
---> [74](~/unstructured/chunking/dispatch.py:74) elements = func(*args, **kwargs)
     [76](~/unstructured/chunking/dispatch.py:76) # -- look for a chunking-strategy argument --
     [77](~/unstructured/chunking/dispatch.py:77) call_args = get_call_args_applying_defaults(func, *args, **kwargs)

File ~\unstructured\partition\pdf.py:210, in partition_pdf(filename, file, include_page_breaks, strategy, infer_table_structure, ocr_languages, languages, include_metadata, metadata_filename, metadata_last_modified, chunking_strategy, hi_res_model_name, extract_images_in_pdf, extract_image_block_types, extract_image_block_output_dir, extract_image_block_to_payload, date_from_file_object, starting_page_number, extract_forms, form_extraction_skip_tables, **kwargs)
    [206](~/unstructured/partition/pdf.py:206) exactly_one(filename=filename, file=file)
    [208](~/unstructured/partition/pdf.py:208) languages = check_language_args(languages or [], ocr_languages)
--> [210](~/unstructured/partition/pdf.py:210) return partition_pdf_or_image(
    [211](~/unstructured/partition/pdf.py:211)     filename=filename,
    [212](~/unstructured/partition/pdf.py:212)     file=file,
    [213](~/unstructured/partition/pdf.py:213)     include_page_breaks=include_page_breaks,
    [214](~/unstructured/partition/pdf.py:214)     strategy=strategy,
    [215](~/unstructured/partition/pdf.py:215)     infer_table_structure=infer_table_structure,
    [216](~/unstructured/partition/pdf.py:216)     languages=languages,
    [217](~/unstructured/partition/pdf.py:217)     metadata_last_modified=metadata_last_modified,
    [218](~/unstructured/partition/pdf.py:218)     hi_res_model_name=hi_res_model_name,
    [219](~/unstructured/partition/pdf.py:219)     extract_images_in_pdf=extract_images_in_pdf,
    [220](~/unstructured/partition/pdf.py:220)     extract_image_block_types=extract_image_block_types,
    [221](~/unstructured/partition/pdf.py:221)     extract_image_block_output_dir=extract_image_block_output_dir,
    [222](~/unstructured/partition/pdf.py:222)     extract_image_block_to_payload=extract_image_block_to_payload,
    [223](~/unstructured/partition/pdf.py:223)     date_from_file_object=date_from_file_object,
    [224](~/unstructured/partition/pdf.py:224)     starting_page_number=starting_page_number,
    [225](~/unstructured/partition/pdf.py:225)     extract_forms=extract_forms,
    [226](~/unstructured/partition/pdf.py:226)     form_extraction_skip_tables=form_extraction_skip_tables,
    [227](~/unstructured/partition/pdf.py:227)     **kwargs,
    [228](~/unstructured/partition/pdf.py:228) )

File ~\unstructured\partition\pdf.py:312, in partition_pdf_or_image(filename, file, is_image, include_page_breaks, strategy, infer_table_structure, languages, metadata_last_modified, hi_res_model_name, extract_images_in_pdf, extract_image_block_types, extract_image_block_output_dir, extract_image_block_to_payload, date_from_file_object, starting_page_number, extract_forms, form_extraction_skip_tables, **kwargs)
    [310](~/unstructured/partition/pdf.py:310)     with warnings.catch_warnings():
    [311](~/unstructured/partition/pdf.py:311)         warnings.simplefilter("ignore")
--> [312](~/unstructured/partition/pdf.py:312)         elements = _partition_pdf_or_image_local(
    [313](~/unstructured/partition/pdf.py:313)             filename=filename,
    [314](~/unstructured/partition/pdf.py:314)             file=spooled_to_bytes_io_if_needed(file),
    [315](~/unstructured/partition/pdf.py:315)             is_image=is_image,
    [316](~/unstructured/partition/pdf.py:316)             infer_table_structure=infer_table_structure,
    [317](~/unstructured/partition/pdf.py:317)             include_page_breaks=include_page_breaks,
    [318](~/unstructured/partition/pdf.py:318)             languages=languages,
    [319](~/unstructured/partition/pdf.py:319)             ocr_languages=ocr_languages,
    [320](~/unstructured/partition/pdf.py:320)             metadata_last_modified=metadata_last_modified or last_modification_date,
    [321](~/unstructured/partition/pdf.py:321)             hi_res_model_name=hi_res_model_name,
    [322](~/unstructured/partition/pdf.py:322)             pdf_text_extractable=pdf_text_extractable,
    [323](~/unstructured/partition/pdf.py:323)             extract_images_in_pdf=extract_images_in_pdf,
    [324](~/unstructured/partition/pdf.py:324)             extract_image_block_types=extract_image_block_types,
    [325](~/unstructured/partition/pdf.py:325)             extract_image_block_output_dir=extract_image_block_output_dir,
    [326](~/unstructured/partition/pdf.py:326)             extract_image_block_to_payload=extract_image_block_to_payload,
    [327](~/unstructured/partition/pdf.py:327)             starting_page_number=starting_page_number,
    [328](~/unstructured/partition/pdf.py:328)             extract_forms=extract_forms,
    [329](~/unstructured/partition/pdf.py:329)             form_extraction_skip_tables=form_extraction_skip_tables,
    [330](~/unstructured/partition/pdf.py:330)             **kwargs,
    [331](~/unstructured/partition/pdf.py:331)         )
    [332](~/unstructured/partition/pdf.py:332)         out_elements = _process_uncategorized_text_elements(elements)
    [334](~/unstructured/partition/pdf.py:334) elif strategy == PartitionStrategy.FAST:

File ~\unstructured\utils.py:217, in requires_dependencies.<locals>.decorator.<locals>.wrapper(*args, **kwargs)
    [214](~/unstructured/utils.py:214) @wraps(func)
    [215](~/unstructured/utils.py:215) def wrapper(*args: _P.args, **kwargs: _P.kwargs):
    [216](~/unstructured/utils.py:216)     run_check()
--> [217](~/unstructured/utils.py:217)     return func(*args, **kwargs)

File ~\unstructured\partition\pdf.py:659, in _partition_pdf_or_image_local(filename, file, is_image, infer_table_structure, include_page_breaks, languages, ocr_languages, ocr_mode, model_name, hi_res_model_name, pdf_image_dpi, metadata_last_modified, pdf_text_extractable, extract_images_in_pdf, extract_image_block_types, extract_image_block_output_dir, extract_image_block_to_payload, analysis, analyzed_image_output_dir_path, starting_page_number, extract_forms, form_extraction_skip_tables, pdf_hi_res_max_pages, **kwargs)
    [647](~/unstructured/partition/pdf.py:647)         final_document_layout = process_file_with_ocr(
    [648](~/unstructured/partition/pdf.py:648)             filename,
    [649](~/unstructured/partition/pdf.py:649)             merged_document_layout,
   (...)
    [656](~/unstructured/partition/pdf.py:656)             ocr_drawer=ocr_drawer,
    [657](~/unstructured/partition/pdf.py:657)         )
    [658](~/unstructured/partition/pdf.py:658) else:
--> [659](~/unstructured/partition/pdf.py:659)     inferred_document_layout = process_data_with_model(
    [660](~/unstructured/partition/pdf.py:660)         file,
    [661](~/unstructured/partition/pdf.py:661)         is_image=is_image,
    [662](~/unstructured/partition/pdf.py:662)         model_name=hi_res_model_name,
    [663](~/unstructured/partition/pdf.py:663)         pdf_image_dpi=pdf_image_dpi,
    [664](~/unstructured/partition/pdf.py:664)     )
    [666](~/unstructured/partition/pdf.py:666)     if hi_res_model_name.startswith("chipper"):
    [667](~/unstructured/partition/pdf.py:667)         # NOTE(alan): We shouldn't do OCR with chipper
    [668](~/unstructured/partition/pdf.py:668)         # NOTE(antonio): We shouldn't do PDFMiner with chipper
    [669](~/unstructured/partition/pdf.py:669)         final_document_layout = inferred_document_layout

File ~\unstructured_inference\inference\layout.py:333, in process_data_with_model(data, model_name, **kwargs)
    [331](~/unstructured_inference/inference/layout.py:331)     tmp_file.write(data.read())
    [332](~/unstructured_inference/inference/layout.py:332)     tmp_file.flush()  # Make sure the file is written out
--> [333](~/unstructured_inference/inference/layout.py:333)     layout = process_file_with_model(
    [334](~/unstructured_inference/inference/layout.py:334)         tmp_file.name,
    [335](~/unstructured_inference/inference/layout.py:335)         model_name,
    [336](~/unstructured_inference/inference/layout.py:336)         **kwargs,
    [337](~/unstructured_inference/inference/layout.py:337)     )
    [339](~/unstructured_inference/inference/layout.py:339) return layout

File ~\unstructured_inference\inference\layout.py:370, in process_file_with_model(filename, model_name, is_image, fixed_layouts, pdf_image_dpi, **kwargs)
    [360](~/unstructured_inference/inference/layout.py:360) else:
    [361](~/unstructured_inference/inference/layout.py:361)     raise ValueError(f"Unsupported model type: {type(model)}")
    [362](~/unstructured_inference/inference/layout.py:362) layout = (
    [363](~/unstructured_inference/inference/layout.py:363)     DocumentLayout.from_image_file(
    [364](~/unstructured_inference/inference/layout.py:364)         filename,
    [365](~/unstructured_inference/inference/layout.py:365)         detection_model=detection_model,
    [366](~/unstructured_inference/inference/layout.py:366)         element_extraction_model=element_extraction_model,
    [367](~/unstructured_inference/inference/layout.py:367)         **kwargs,
    [368](~/unstructured_inference/inference/layout.py:368)     )
    [369](~/unstructured_inference/inference/layout.py:369)     if is_image
--> [370](~/unstructured_inference/inference/layout.py:370)     else DocumentLayout.from_file(
    [371](~/unstructured_inference/inference/layout.py:371)         filename,
    [372](~/unstructured_inference/inference/layout.py:372)         detection_model=detection_model,
    [373](~/unstructured_inference/inference/layout.py:373)         element_extraction_model=element_extraction_model,
    [374](~/unstructured_inference/inference/layout.py:374)         fixed_layouts=fixed_layouts,
    [375](~/unstructured_inference/inference/layout.py:375)         pdf_image_dpi=pdf_image_dpi,
    [376](~/unstructured_inference/inference/layout.py:376)         **kwargs,
    [377](~/unstructured_inference/inference/layout.py:377)     )
    [378](~/unstructured_inference/inference/layout.py:378) )
    [379](~/unstructured_inference/inference/layout.py:379) return layout

File ~\unstructured_inference\inference\layout.py:62, in DocumentLayout.from_file(cls, filename, fixed_layouts, pdf_image_dpi, **kwargs)
     [59](~/unstructured_inference/inference/layout.py:59) logger.info(f"Reading PDF for file: {filename} ...")
     [61](~/unstructured_inference/inference/layout.py:61) with tempfile.TemporaryDirectory() as temp_dir:
---> [62](~/unstructured_inference/inference/layout.py:62)     _image_paths = convert_pdf_to_image(
     [63](~/unstructured_inference/inference/layout.py:63)         filename,
     [64](~/unstructured_inference/inference/layout.py:64)         pdf_image_dpi,
     [65](~/unstructured_inference/inference/layout.py:65)         output_folder=temp_dir,
     [66](~/unstructured_inference/inference/layout.py:66)         path_only=True,
     [67](~/unstructured_inference/inference/layout.py:67)     )
     [68](~/unstructured_inference/inference/layout.py:68)     image_paths = cast(List[str], _image_paths)
     [69](~/unstructured_inference/inference/layout.py:69)     number_of_pages = len(image_paths)

File ~\unstructured_inference\inference\layout.py:394, in convert_pdf_to_image(filename, dpi, output_folder, path_only)
    [391](~/unstructured_inference/inference/layout.py:391)     raise ValueError("output_folder must be specified if path_only is true")
    [393](~/unstructured_inference/inference/layout.py:393) if output_folder is not None:
--> [394](~/unstructured_inference/inference/layout.py:394)     images = pdf2image.convert_from_path(
    [395](~/unstructured_inference/inference/layout.py:395)         filename,
    [396](~/unstructured_inference/inference/layout.py:396)         dpi=dpi,
    [397](~/unstructured_inference/inference/layout.py:397)         output_folder=output_folder,
    [398](~/unstructured_inference/inference/layout.py:398)         paths_only=path_only,
    [399](~/unstructured_inference/inference/layout.py:399)     )
    [400](~/unstructured_inference/inference/layout.py:400) else:
    [401](~/unstructured_inference/inference/layout.py:401)     images = pdf2image.convert_from_path(
    [402](~/unstructured_inference/inference/layout.py:402)         filename,
    [403](~/unstructured_inference/inference/layout.py:403)         dpi=dpi,
    [404](~/unstructured_inference/inference/layout.py:404)         paths_only=path_only,
    [405](~/unstructured_inference/inference/layout.py:405)     )

File ~\pdf2image\pdf2image.py:127, in convert_from_path(pdf_path, dpi, output_folder, first_page, last_page, fmt, jpegopt, thread_count, userpw, ownerpw, use_cropbox, strict, transparent, single_file, output_file, poppler_path, grayscale, size, paths_only, use_pdftocairo, timeout, hide_annotations)
    [124](~/pdf2image/pdf2image.py:124) if isinstance(poppler_path, PurePath):
    [125](~/pdf2image/pdf2image.py:125)     poppler_path = poppler_path.as_posix()
--> [127](~/pdf2image/pdf2image.py:127) page_count = pdfinfo_from_path(
    [128](~/pdf2image/pdf2image.py:128)     pdf_path, userpw, ownerpw, poppler_path=poppler_path
    [129](~/pdf2image/pdf2image.py:129) )["Pages"]
    [131](~/pdf2image/pdf2image.py:131) # We start by getting the output format, the buffer processing function and if we need pdftocairo
    [132](~/pdf2image/pdf2image.py:132) parsed_fmt, final_extension, parse_buffer_func, use_pdfcairo_format = _parse_format(
    [133](~/pdf2image/pdf2image.py:133)     fmt, grayscale
    [134](~/pdf2image/pdf2image.py:134) )

File ~\pdf2image\pdf2image.py:611, in pdfinfo_from_path(pdf_path, userpw, ownerpw, poppler_path, rawdates, timeout, first_page, last_page)
    [607](~/pdf2image/pdf2image.py:607)     raise PDFInfoNotInstalledError(
    [608](~/pdf2image/pdf2image.py:608)         "Unable to get page count. Is poppler installed and in PATH?"
    [609](~/pdf2image/pdf2image.py:609)     )
    [610](~/pdf2image/pdf2image.py:610) except ValueError:
--> [611](~/pdf2image/pdf2image.py:611)     raise PDFPageCountError(
    [612](~/pdf2image/pdf2image.py:612)         f"Unable to get page count.\n{err.decode('utf8', 'ignore')}"
    [613](~/pdf2image/pdf2image.py:613)     )

PDFPageCountError: Unable to get page count.
I/O Error: Couldn't open file 'C:\Users\...\Temp\tmpzorb9m7z': No error.

The issue does not occur if I load the file using the filename arg though.

@simonschoe thanks for this :)
Okay, this looks like a bug that has been fixed on main but not released yet.
https://github.com/Unstructured-IO/unstructured-inference/commit/7804e0d7125dcfd8923fe7bbb1d255bfb1a96335
Can you try installing unstructured-inference from the main branch on GitHub? I think that's going to solve the problem. Something like this IIRC:
$ pip install -U unstructured-ingest @ git+https://github.com/Unstructured-IO/unstructured-ingest

I'll see about moving along a release.

Thanks for the feedback! Unfortunately, I have to resort to a stable release verion. I will look out for the upcoming unstructured-inference release

@simonschoe
I've faced the issue due to some dependencies not being installed. The unstructured version I used is 0.15.9.
sudo apt-get install poppler-utils # recommend by https://stackoverflow.com/questions/53481088/poppler-in-path-for-pdf2image
sudo apt install tesseract-ocr # recommended by https://tesseract-ocr.github.io/tessdoc/Installation.html
sudo apt install libtesseract-dev # recommended by https://tesseract-ocr.github.io/tessdoc/Installation.html
pip install tesseract # recommended by https://stackoverflow.com/a/52231794
pip install tesseract-ocr # recommended by https://stackoverflow.com/a/52231794

I've also found the command sudo apt-get install libleptonica-dev tesseract-ocr tesseract-ocr-dev libtesseract-dev python3-pil tesseract-ocr-eng tesseract-ocr-script-latn, but I thought it is legacy code since I will get the error E: Package 'tesseract-ocr-dev' has no installation candidate. You can further try the legacy code if you are still facing the PDFPageCountError: Unable to get page count. error after executing the above command.

Fixed by #3395.


README.md has messy table columns for input & output tables

I cannot seem to clearly make out the description column input & output tables in README.md. I tried with multiple browsers & font sizes to get it to work but no dice. It is a silly issue, but needs fixing.

sorry my bad. missed the horizontal scroll bar many pages below in the table. Pls ignore.


Update PCAP.h

I am trying the esp8266_pcap_serial.ino example but the coed doesn't compile without this change since i am not using a SD card, i am not very good in C so the change could be wrong

thanks


Make our vanity aspect.build go imports work with tooling

We should follow https://sagikazarmark.hu/blog/vanity-import-paths-in-go/
so that our savvy go imports for users actually work when they add a dependency on us and run go mod tidy

closing since I believe this is fixed, will reopen if and when someone outside our module tries to depend on it


feat: add llama2 colab notebook

preview in https://colab.research.google.com/drive/158zwSM__zs0caehysLinxLkjY7_naqcK?usp=sharing

@parano any comments on this when you have cycle?

@aarnphm @parano any new comments here?

finally the gptq we should also remove the outputs. LGTM from me, cc @parano
fix. tks~~

also what is this fLlama? should we just call it function-calling-llama-demo?

ok

all colab examples are available in our google drive https://drive.google.com/drive/folders/1xxpJJaKISS9lxOI4SjL8CeqF2Zt5Vf_Q

all colab examples are available in our google drive drive.google.com/drive/folders/1xxpJJaKISS9lxOI4SjL8CeqF2Zt5Vf_Q

Can we also make this publicly available to everyone? I believe currently this is only available internallly.

What is the status of this PR @xianml?

What is the status of this PR @xianml?

Sorry. will split this PR to multi part for a quicker feedback loop. will deprecate this one


SQS - Should the AWS4Signer take region from RegionEndpoint before attempting to look at query string

The Question
I was running some SQS code against a local API today and was setting the region endpoint up in a way that is consistent with S3:
var sqsClient = new AmazonSQSClient(new AmazonSQSConfig
{
    UseHttp = true,

    RegionEndpoint = RegionEndpoint.EUWest1,
    ServiceURL = "http://localhost:4566"
});

Because this API was respecting region implementation I was creating a queue via the SDK and attempting to see it in the code (or visa versa) and was seeing 0 queues being returned (but when I created the queue via cli and listed queues form the cli it was working) after some request inspection I saw that the region was actually set to us-east-1.
After a bit of digging around in the sdk I found you signer that does this logic:
string authenticationRegion = clientConfig.AuthenticationRegion;
if (request != null && request.AuthenticationRegion != null)
    authenticationRegion = request.AuthenticationRegion;

if (!string.IsNullOrEmpty(authenticationRegion))
    return authenticationRegion.ToLowerInvariant();

if (!string.IsNullOrEmpty(clientConfig.ServiceURL))
{
    var parsedRegion = AWSSDKUtils.DetermineRegion(clientConfig.ServiceURL);
    if (!string.IsNullOrEmpty(parsedRegion))
        return parsedRegion.ToLowerInvariant();
}

Which makes the correct way to change regions here look like this:
var sqsClient = new AmazonSQSClient(new AmazonSQSConfig
{
    UseHttp = true,
    AuthenticationRegion = "eu-west-1", // this is what controls regions here
    RegionEndpoint = RegionEndpoint.EUWest1,
    ServiceURL = "http://localhost:4576"
});

I'm wondering whether it would be a sensible idea to do a check on RegionEndpoint here as that is what usually changes regions for other clients, be it kinesis or S3 etc.
Environment

SDK Version:
Package Version: <IP_ADDRESS>
OS Info: Windows 10
Build Environment Visual Studio
Targeted .NET Platform:  .Net Core 3.1

This is a :question: general question

Hi @joro550,
Good morning.
Please advise if you are using community LocalStack emulation. I'm not quite sure if that would cause the issue and how credentials are configured in LocalStack. Based on the logic in DetermineSigningRegion(), here is the order in which region information takes precedence:

IRequest.AlternateEndpoint takes precedence over client config properties.
AuthenticationRegion
Region determined from Service URL
RegionEndpoint

RegionEndpoint would be the fallback case if no preceding sources are able to resolve the region. I will try to reproduce the issue. Could you please share the default configuration in credentials and config files (excluding the sensitive information like secret and access keys) and your application configuration which might have default profile set.
Thanks,
Ashish

Hey @ashishdhingra,
Yes I am using the community LocalStack emulation, I guess my confusion came in when the ServiceUrl was returning us-east-1, in the local stack instance (in my code) the AWS4Signer.DetermineSigningRegion would always fall into the following case unless you have the authentication region (or alternative region which I'm unsure how to set):
if (!string.IsNullOrEmpty(clientConfig.ServiceURL))
{
    var parsedRegion = AWSSDKUtils.DetermineRegion(clientConfig.ServiceURL);
    if (!string.IsNullOrEmpty(parsedRegion))
        return parsedRegion.ToLowerInvariant();
}

case, if it cannot retrieve a region out of the url it actually defaults to us-east-1. It makes sense that Service URL would come first in most cases, but looking at the AWSSDKUtils.DetermineRegion implementation:
public static string DetermineRegion(string url)
{
    int delimIndex = url.IndexOf("//", StringComparison.Ordinal);
    if (delimIndex >= 0)
        url = url.Substring(delimIndex + 2);

    if (url.EndsWith("/", StringComparison.Ordinal))
        url = url.Substring(0, url.Length - 1);

    int awsIndex = url.IndexOf(".amazonaws.com", StringComparison.Ordinal);
    if (awsIndex < 0)
        return DefaultRegion;

...... more code
}

It looks like it would return DefaultRegion which is set to us-east-1 because we are setting our url to "localhost" (or whatever your docker instance is called).
The confusion came because normally when I'm using localstack I set region endpoint and the service URL and it allows me to connect to the correct region.
I'll try and get a copy of my configuration and credentials and copy them into this issue ASAP, thank you 😄

Here is my credentials file with access and secret key taken out:
[default]
output=json
region = eu-west-1
aws_access_key_id =
toolkit_artifact_guid=8e972762-e8e3-47fd-8c0a-8aa9ffc2302b
aws_secret_access_key =

interestingly I don't actually have a configuration file

Oh - I have just figured out that there is intrinsic logic within the config object that set the service url to null when region endpoint is provided and visa versa, so this may be an easy answer to say that it wouldn't work in the way I want it to work :smile:

Thanks @joro550 for the input. Could you please paste the perm link to that logic here for reference. Also confirm if we could close this issue.

Sure thing, so as a result of my investigation you cannot set the service url and region endpoint to use to connect to localstack because the client config sets service url to null if you set region endpoint and visa versa which you can see here
What that means is that you need to set the region you are connecting to via the methods listed (in priority order):

IRequest.AlternateEndpoint takes precedence over client config properties.
AuthenticationRegion
Region determined from Service URL
RegionEndpoint

You can see that in this method here, all though I have suspicions that in the case where you don't have .amazonaws.com in your ServiceUrl (when set) that it will default to us-east-1 which you can see in this logic - which  seems to have changed in master.
Anyway - I confirm that you can close this issue, I'll do it myself with this comment, thank you for your time 😄


Add Verilator dependency

Thank or adding the link.

Can you please merge? I do not have write access to this repo (or freechipsproject)


Close restore and attach form right after receiving the job id

Description
When restoring and attaching a backup volume, the UI waits for the successfful response of the API to close the formulary. However, letting the form open after sending the command can confuse users, that might try to send the command again.
Note: other formularies (like volume or snapshot creation) always are closed after sending the command to the backend.
With this PR, we intend to close the formulary right after sending the command, as done with others.
Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[ ] Bug fix (non-breaking change which fixes an issue)
[x] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale

[ ] Major
[x] Minor

How Has This Been Tested?
It was tested in a local lab.
Before the changes, I tried to restore and attach a backup volume. After clicking the button Ok, the command was sent, but the formulary was kept open. After the changes, when clicking the button Ok, the command is sent and the formulary is closed.

@acs-robot a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

UI build: :heavy_check_mark:
Live QA URL: http://qa.cloudstack.cloud:8080/client/pr/6603 (SL-JID-2057)


TY GP sound @ ge

When u sold/bought something at the G.E could u get oda to say "thank you for your gp" just like in the yt clip of him scamming dclaws
https://discord.com/channels/425480079368257546/1202779793750499349/1217631865255628940
https://www.youtube.com/watch?v=IXwd6JJXRhk&ab_channel=OdablockClips

After careful consideration we've decided not to implement this because it will get old really fast.


Add livy_session_startup_timeout_seconds config back in

Should fix #9155 - @chlafreniere @corivera is there an easy way to check that this is actually fixed?

@Charles-Gagnon I think you can query the config from a cell after the session starts.


Open MainViewModelTest.kt in the androidTest folder -> in the test folder

At step 6, there is an instruction that says:
Open MainViewModelTest.kt in the androidTest folder.
But the MainViewModeTest.kt is inside the test folder.

Still there, not fixed yet


REPL for Ergo

We should provide a read-eval-print-loop for Ergo. This is useful for learning the language.

Some ideas for what the REPL could provide:

Load existing Ergo files and CTOs and provide interactive loop
Interactive execution:

Execute expressions and return results
Execute/test contracts with multiple parameters
Instantiate contract with its variables
Initialize a contract state
Calling a contract clause
Send a request to a contract
Respond with Error or Response + State + Emit

Batch execution

Execute from a file
Use as test harness (e.g., as series of enforce?) to replace the smoke test (examples/smoke)

Contract composition

Create a contract from multiple stand-alone clauses (e.g., new Contract{ } with clause1, clause 2


Unable to install dependencies

Hi all,
Great work! I'm going through the paper and was trying to run the notebooks on my machine. For some reason I was not able to install the dependencies following instructions from here. I'm getting the following error while trying step 3.
I'm running this on a Windows 11 machine. Seems like uvloop doesn't support Windows. Is there any way to get around that?
error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [18 lines of output]
      Traceback (most recent call last):
        File "C:\Users\hsayeed\miniconda3\envs\text_to_insight_book_8\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "C:\Users\hsayeed\miniconda3\envs\text_to_insight_book_8\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\hsayeed\miniconda3\envs\text_to_insight_book_8\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\hsayeed\AppData\Local\Temp\pip-build-env-ui2_p946\overlay\Lib\site-packages\setuptools\build_meta.py", line 327, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=[])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\hsayeed\AppData\Local\Temp\pip-build-env-ui2_p946\overlay\Lib\site-packages\setuptools\build_meta.py", line 297, in _get_build_requires
          self.run_setup()
        File "C:\Users\hsayeed\AppData\Local\Temp\pip-build-env-ui2_p946\overlay\Lib\site-packages\setuptools\build_meta.py", line 313, in run_setup
          exec(code, locals())
        File "<string>", line 8, in <module>
      RuntimeError: uvloop does not support Windows at the moment
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.```

Thanks for the report, I think it is caused by the dependency updates that I did and also caused #118. I'll give it a look

I now dropped the last two dependency updates and the installation works again on GitHub (https://github.com/lamalab-org/matextract-book/actions/runs/10193759745/job/28198981323), but I'll now also verify locally

And - then there is a different problem with the operating system.
A quick fix would be to use WSL on Windows. I'll check if we can also remove uvloop

@hasan-sayeed can you check after the latest updates? We now also did some checks on Windows and it seems to work on our side

Awesome, thanks for confirming! I'll now close this issue, but feel free to re-open it if you run into something like this again!
Thanks also for trying the notebooks in the first place!


define dependencies not to wrap globally

I often use lodash. When I have a file with 5 dependencies where I want to mock everything but lodash I currently need to specify all other 4 dependencies explicitly: inject?lib1&lib2&lib3&lib4!testable. Therefore it would be great if the plugin could have some kind of global configuration to always exclude libraries like lodash.

Yeah, that would be neat—thanks for the suggestion.

+1

This is resolved by #7.
All dependencies of a module are still injected by default, however if a replacement is no provided it will roll back to the original dependency.


Code Prime Factor problem :infinity: in any programming language you like :computer:

Code Prime Factor problem :infinity: in the relevant language folder :file_folder: in the main branch.
if language folder doesn't exist you can make folder :file_folder: and add your program in it.
You can find new languages in this Programming Language List.
Give star :star: to this repository.

hello sir, assign me this issue please

Hello @iParadocx , I would like to contribute to this using C++, Thank You.

I would like to contribute to this issue in Java. Please assign this task to me.


Setup file watching for dev server

File watching has been added an properly documented in the readme.
However, a file change does not refresh the browser. Will add browser refresh as another build issue.


Add pytest-xdist in generated projects.

It's super useful to have tests running in parallel.
And of course, for each tests runner we need to create separate database.

Maybe it's not necessary.


contact count and matching keys doesn't match, 5 contacts

The CWA shows  a red status and 5 contacts -> high risk.
The logs are showing matchcount: 1. This means i had 5 contacts with a single person on one day? I'm asking because on the day in question i didn't had contact with anyone i don't know (also know their green app status). So i'm assuming receiving beacon signals from one of my neighbours through the wall/ceiling.
Attached Screenshot and Log from the CWA App (iOS 14.1 und CWA 1.5.1).

@patrick-werner thanks for posting this… Just to mirror some information from over at the community Slack:
There is only one check which shows one match in the log entries for today:
 {
          "Hash" : "4CF0C497455611780A6E99FE12B16AC70DD135BB6D2769AFB0FB10A56F425722",
          "MatchCount" : 1,
          "KeyCount" : 14309,
          "AppBundleIdentifier" : "de.rki.coronawarnapp",
          "Timestamp" : "2020-10-24 17:13:04 +0200"
        }

This usually would mean 1 encounter/exposure in CWA, but not in this case for some reason 😕.
Here are all the log entries for today in a table:

Details

Timestamp
Hash
MatchCount
KeyCount
Timestamp.1

24.10.2020   17:12
064B723E5EDD2EB8CD4C36FAA67FE839A486451748A5521B447606795867A061
0
18645
24.10.2020 17:13

24.10.2020   17:12
D6C9ADEF176488BB7BCFA9F3A01EB5A9F692817441B777A630A1F74131A03CDF
0
16122
24.10.2020 17:13

24.10.2020   17:12
4CF0C497455611780A6E99FE12B16AC70DD135BB6D2769AFB0FB10A56F425722
1
14309
24.10.2020 17:13

24.10.2020   17:12
D63C8A25912DA34CD9A266967B854BB98BC09290BFE12B804C1FD796AE64BBD3
0
10876
24.10.2020 17:13

24.10.2020   17:12
F049E50F243C7B991867B229668A44CBCA6E63846EBECA0A2B8E3C0BA8B7ACAC
0
6386
24.10.2020 17:13

24.10.2020   17:12
3908E8EF3DE421C31CD585B8CBAFCEA7D54C21BF94E5E690A477AEE217E8B95A
0
4814
24.10.2020 17:13

24.10.2020   17:12
F9073A1268C72E6C93884F761E27066E9455BCFC0EFDB5068EA8C0BAAF1E098E
0
16674
24.10.2020 17:12

24.10.2020   17:12
41444FD9FFAEFDD39A78B11E59DD58EF2146D5F65D39E7607C0E1997712B737F
0
43934
24.10.2020 17:12

24.10.2020   17:12
1A7FA72A1A077EBD7EC11F0F8B3BB482A22865B133E9B45D66EA7AF11D5AAC53
0
41646
24.10.2020 17:12

24.10.2020   17:12
8F2EA05416F4175417980FFEC401506135A9591AC6A6F715CB94BF6A92723AA4
0
34105
24.10.2020 17:12

24.10.2020   17:12
DB52431B890F3BC23B5C1BF902CC6EB2EDE0ABD827ABD58C207CA20E525B1008
0
28218
24.10.2020 17:12

24.10.2020   17:12
35FC7AAA7F90C04599FE355EA2AFEDC18A93F74E2AFB8E152C138EF0704E171C
0
18134
24.10.2020 17:12

24.10.2020   17:12
9CCB79003AEC7BAAAA5F2733453AF4DAE414919FECF388344F8C4F7F21C47581
0
14649
24.10.2020 17:12

24.10.2020   17:12
970614A4D5AED5E567649FC508C6013847B446E03881A69647986950B497F227
0
23657
24.10.2020 17:12

24.10.2020   14:58

There seemingly was an attempt to perform checks at 14:58 (but the entry is empty, so seemingly it didn't work) and then the actual checks were performed at 17:12 which is also the timestamp which shows up in CWA… but even there, only one match in one of the packages is found 😕.

One question @Ein-Tim @ndegendogo: you also have encounters/exposures under iOS 14.1 and CWA 1.5.1, right? Is your EN log showing the same number of matches as your CWA?

@daimpi yes, always. And I am verifying this daily.
I could not check during 5 weeks with iOS 13.7 because of a bug (#1106), but before and afterwards I did never see a deviation.

@patrick-werner your ENF log file shows a match since 3 days.
Did you check cwa the days before? Are you aware if this strange mismatch was there from the beginning (all these three days)?
If you checked cwa: was it always red, or was it green (green with encounters) before?

One crazy (?) idea on this: could it be that Apple fixed their bug (#1106) only partially in 14.1, and still suppress some of the encounters in their ENF log file??

@patrick-werner which iOS version exactly do you have? Is it the "official" version, or some preliminary / special version for developers?
I noticed that the "build" number in the second line of your ENF log differs from mine.
I had always taken this number as indication of iOS version.
My number is (with iOS 14.1)
"Build" : "18A8395",
Your ENF log shows:
"Build" : "18A393",
@Ein-Tim could you please check your number?

@daimpi yes, always. And I am verifying this daily.
I could not check during 5 weeks with iOS 13.7 because of a bug (#1106), but before and afterwards I did never see a deviation.

Can verify this, always had same amount of "matched Keys" any Encounters in CWA @daimpi

@Ein-Tim could you please check your number?

Yes, my Build number is 18A8396 (like the Umber from you)

Your ENF log shows:
"Build" : "18A393",

This is the Build Number from iOS 14.0.1 the version which fixed this is iOS 14.1 so I think he had 1 Red Encounter, this one got logged but the other 4 low Risk Encounters did because of #1106 not get logged...
@patrick-werner Please update to iOS 14.1

oh wow. I was 100% sure beeing in iOS 14.1, which i wasn't. Thanks for the explanation, sorry for not being aware of my device s/w version.
Thanks @Ein-Tim & @daimpi  !!!

@patrick-werner no worries and thanks to @ndegendogo and @Ein-Tim for solving this riddle ❤.
This was really very curious, as seemingly Apple went all over the place with what they showed in their log… I was under the impression that #1106 meant full suppression of all the matches, but seemingly that was only true for some versions. E.g. we had someone with red exposures on "iOS 14 public beta 8" ("Build" : "18A5373a") reporting here that they had 0 matches in their EN log with 14 (red) exposures shown in CWA. But this behavior was then seemingly changed again for iOS 14.0.1 and finally really fixed with 14.1. Fun times ^^.

is there some place where one can look up which build number corresponds to which iOS version?

I am not aware of any "official" list of this kind.
I have only an (incomplete) private list compiled of my own version history, plus a few from users ENF logs posted here.

Wikipedia lists them (~semi-official)


Subarrays as second argument

From a matrix A
A = rand(10, 10)

I can interpolate with respect to the first dimension
 CoordInterpGrid(1:10, A[:, 1], BCnearest, InterpLinear)

However, I can't interpolate with respect to the second dimension, i.e. do something like
 CoordInterpGrid(1:10, A[1, :], BCnearest, InterpLinear)
 CoordInterpGrid(1:10, slice(A, 1, :), BCnearest, InterpLinear)

The only solution is to transpose A so that the order of dimensions is inverted. However, adding a support for subarray would probably be faster, no?

Even faster would be to use Interpolations :smile:. Your timing is impeccable; I just pushed the new README, with benchmarks, a few minutes ago.
Interpolations already supports SubArrays  (or should).

It's even better - you implemented lookup wrt some dimension. Exactly what I was looking for.

:v: