refactor(store): store core gets field layout from Tables table [n-08]

going to pass this to @alvrs for review as its unclear why we're seeing a bump in various gas tests (esp deletes)

Closing since this creates more gas in the hot code path than is worth it. Let's note this in the audit issue.


MM-31518 Cloud Integrations updates

Documentation ticket: https://mattermost.atlassian.net/browse/MM-31518
Updated:

Cloud Admin Guide > Integrations > Bot Accounts

Replaced CLI details with mmctl option
Cloud documentation Mattermost server references updated to Mattermost workspaces

Hello @cwarnermm,
Thanks for your pull request! A Core Committer will review your pull request soon. For code contributions, you can learn more about the review process here.

Hello @cwarnermm,
Thanks for your pull request! A Core Committer will review your pull request soon. For code contributions, you can learn more about the review process here.

@jwilander - Can you assist with the outstanding question here re: data retention, please?
@mgdelacroix - Can you assist with the outstanding question here regarding mmctl user convert?

@cwarnermm can you connect with @sbishel or @mkraft about that question? I'm not sure on the answer


Prevent Login with Incorrect Credentials

Description: The system should block login attempts with invalid credentials.
Acceptance Criteria:
If the email or password is incorrect, the system displays: "Incorrect email or password."
The system prevents access until the correct details are provided.

provided


fix: stop the progress ticker after build completes

Fixes: https://github.com/knative-sandbox/kn-plugin-func/issues/542

Note that removing the line that prints "Deploying function" results in nicer output.
> kn func deploy
   🙌 Function image built: docker.io/lanceball/foobar:latest
   Function deployed at URL: http://foobar-default.lball-cluster-0c576f1a70d464f092d8591997631748-0000.us-east.containers.appdomain.cloud
>

vs.
> kn func deploy
   Deploying function
   Function updated at URL: http://foobar-default.lball-cluster-0c576f1a70d464f092d8591997631748-0000.us-east.containers.appdomain.cloud


Produced expression is incorrect?

I've tried the following input, based on your example on the github page:
w :: Double -> Double -> Double
w far near = -(2 * far * near) / (far - near)

Running Herbie, I get the following output:
Compiling with Herbie floating point stabilization
Found math expression within binding w :: Double -> Double -> Double
  original expression = (2 * far * near) / (far - near)
  improved expression = if far < (negate 1.7210442634149447e81)
    then (far / (far - near)) * 2 * near
    else if far < 8.364504563556443e16
        then (far * 2) / (far - near) / near
        else (far / (far - near)) * 2 * near
  original error = 13.82279321294945 bits
  improved error = 2.34375e-2 bits
  WARNING: Not substituting the improved expression into your code
  WARNING: Cannot satisfy the constraint: Fractional Double

Based on this output, I created the following variant of w:
w' :: Double -> Double -> Double
w' far near = -(if far < (negate 1.7210442634149447e81)
    then (far / (far - near)) * 2 * near
    else if far < 8.364504563556443e16
        then (far * 2) / (far - near) / near
        else (far / (far - near)) * 2 * near)

Note that I put an extra negation around the Herbie suggested improvement, since it said the matched expression was without the negation in the original function w.
But these expressions wildly differ. Consider:
far, near :: Double
far = -1.4999999999999998
near = -1.25

We get:
ghci> w far near
15.000000000000012
ghci> w' far near
9.600000000000007

This is radically different output; and indeed if you do the "math" w' is way off. The correct result is much closer to what's computed by the original function w.
Am I reading Herbie's output incorrectly?

Having taken another look at it, I think the expression in the second then should be:
((far * 2) / (far - near)) * near

instead of:
(far * 2) / (far - near) / near

That would make way more sense, but perhaps something else is going on here.

This related to issue #8.  I believe it's a parsing bug in the plugin code I wrote.  It'll probably be a few days before I get to take a look at this in more detail.

This should be fixed now.


db.seek only finding exact matches

Me again :)
I'm having trouble with db.seek only finding exact matches, and not (as per the documentation) "first key greater than or equal to specified key".
I believe the culprit is this, from EntryIterator.java:
 public boolean hasNext() {
    if (first) {
      if (key != null) {
        this.entry = cursor.seek(SeekOp.KEY, key);
      }

shouldn't that be SeekOp.RANGE rather than .KEY ?

FWIW, I've just made this change locally and tested it and am now getting perfect results with my custom comparator :)  Thanks loads for the help!

Great! If you send me a pull request i'll merge it in.


Update README.md

Issue #, if available:
Description of changes:
Refactor README to briefly go over Homebrew, Taps, etc and also update with information about packages i.e. ec2-macos-utils and amazon-ena-ethernet-dext
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Fixed typo, should be good.


Bugfix: when db_type is set to postgres, tailscale client will be hang up for a long time

[x] read the CONTRIBUTING guidelines
[ ] raised a GitHub issue or discussed it on the projects chat beforehand
[ ] added unit tests
[ ] added integration tests
[ ] updated documentation if needed
[ ] updated CHANGELOG.md

Fixes #764
When db_type is set to postgres, tailscale login and tailscale up can hang up for a long time. because expiry time is 0001-01-01 00:00:00.
I'm not sure this change is make sense, but it‘s worked.

Hi! as part of #1473, we have reorganised a lot of the code.
To clear PRs that needs to be rebased or redone, we are closing open PRs that will require significant code change to be merged.
In addition, the issue of the PR might in some cases have been fixed, change or no longer relevant, so it would be great if this is considered as well.
Thank you for your contribution!
If it is still relevant and the PR is reopened, we will aim at getting the changes into the next release after the reorg if accepted.


Enemy color swaps

Exclude:

Beavers
Klaptraps
Klobbers and Kabooms
Kremlings

https://discord.com/channels/463917049782075395/1190389279789174946


Can't build stripped down S3 flavor of cachepot

I want a no-frills cachepot for my S3-powered CI environment but it won't build. My environment:
% rustc --version
rustc 1.61.0-nightly (52b34550a 2022-03-15)
% cargo --version
cargo 1.61.0-nightly (65c826642 2022-03-09)

The error:
% cargo install cachepot --version 0.1.0-rc.1 --no-default-features --features=s3
[[[ SNIP ]]]
error[E0432]: unresolved import `crate::config::WorkerUrl`
  --> /Users/xlange/.cargo/registry/src/github.com-1ecc6299db9ec823/cachepot-0.1.0-rc.1/src/dist/mod.rs:16:5
   |
16 | use crate::config::WorkerUrl;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^ no `WorkerUrl` in `config`

error[E0412]: cannot find type `WorkerUrl` in module `crate::config`
   --> /Users/xlange/.cargo/registry/src/github.com-1ecc6299db9ec823/cachepot-0.1.0-rc.1/src/compiler/compiler.rs:725:23
    |
725 |     Ok(crate::config::WorkerUrl),
    |                       ^^^^^^^^^ not found in `crate::config`

Some errors have detailed explanations: E0412, E0432.
For more information about an error, try `rustc --explain E0412`.
error: could not compile `cachepot` due to 2 previous errors
warning: build failed, waiting for other jobs to finish...
error: failed to compile `cachepot v0.1.0-rc.1`, intermediate artifacts can be found at `/var/folders/mc/3pj1vp8s239fn610rbv56f440000gp/T/cargo-installTYYXXo`

Caused by:
  build failed

Hey @xrl, thanks for reporting. This is definitely a bug, possible introduced with #134


Improve IUnknown Unsafe Annotations

This PR does two things:

Makes the QueryInterface and Release methods on IUknownImpl unsafe.

These methods require ensuring certain variants for them to be called safely. For example, Release can free memory so the user might be sure that once they call Release the no longer use self.

Makes QueryInterface, AddRef, and Release take non-exclusive references since they are effectively methods on aliased references and thus cannot be &mut self.

As a side effect I needed to change QueryInterface's out param to be *mut *const c_void instead of *mut *mut c_void. I believe this more closely models what an interface out param is.

cc @eholk who has expressed interest in helping review safety related PRs

I'm just going to check with @eholk before merging.

I'm just going to check with @eholk before merging.

Looks good to me!


Loss of focus is observed after tabbing from "No, I would not to participate" radio button

Steps to Reproduce

Launch Xamarin Workbooks application. This will invokes "Create a new Workbook" window.
Navigate to Tools -> Preferences and invoke it. This will invoke general tab of preferences window.
Invoke Feedback tab using Mouse (As it is not accessible by keyboard).
Navigate to "No, I would not to participate" radio button and press tab.

Expected Behavior
Keyboard focus should move to next interactive element (Feedback Tab).
Actual Behavior
1)Loss of focus is observed after pressing tab from "No, I would not to participate" radio button.
2) If user press the tab(2nd tab) again it sets focus on "Yes, I would participate (Recommended)" radio button.
Inspect Behavior:
After pressing tab from "No, I would not to participate" radio button, focus is going to content area ("Help Improve Xamarin Workbooks") of preferences. Please refer below screenshot.

User Impact:
People with visual impairments can become disoriented when keyboard navigation takes focus somewhere unexpected or when they cannot easily find the content surrounding an interactive element.
Screenshots:

Environment
Windows : Version 1803 (OS Build : 17134.81)
Xamarin Version : 1.4(master/b3f92f9)
Log File

Please add the following labels to the bug:
A11yMAS
Accessibility
Xamarin-Workbooks-Windows
Wipro
MAS2.4.3

Exemption Approved - 1813


Cuda gpu

Hello,
I have a RAM with 13GB, I activated cuda in the params.py file but the training and test are still slow comparing to the capacity of my machine.
It's supposed to run quickly I mean.
I'm wondering if cuda is actually working or not in reality?

@mariembenslama since our displayInterval is set to 100 and batchSize is set to 64, it means the training loss will be printed every 6400 samples. You can set the displayInterval to 10 then it may seems ten times quickly. Of course, you can use the command watch -n 1 nvidia-smi to checkout the using case of GPU, it will show something like
Sun Apr 28 17:44:06 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 410.104	  Driver Version: 410.104      CUDA Version: 10.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla P40           Off  | 00000000:3B:00.0 Off |                    0 |
| N/A   36C    P0   128W / 250W |   1063MiB / 22919MiB |     66%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0    249645      C   python                                      1053MiB |
+-----------------------------------------------------------------------------+

If there are still problems, please let me know, thank you~

Thanks! I guess it's good now :) it jumped quickly to a good intervalle of accuracy and loss ^-^
Thank youuu~~~
Is the model (DL in general) always taking too long to grasp all the alphabets?

@mariembenslama I believe you can do it! But you can add more training data to it to let it know the right decline direction. If you have more training data, after 9/1000 epoch, the loss can be lower! Good luck!

Thank you! I will do as you say 😊
Thanks a lot again 😀😀😀


Support gettimeofday on more than macos

This appears to be in linux and in openbsd as well:

https://github.com/torvalds/linux/blob/master/lib/vdso/gettimeofday.c
https://github.com/openbsd/src/blob/master/sys/sys/time.h#L439

Note that std currently says different syscalls are used on mac vs linux but this is not part of std's contract and third party code could call the syscall directly on different platforms.

Yeah, seems to be a POSIX thing:

POSIX.1-2001 describes gettimeofday() but not settimeofday().  POSIX.1-2008 marks gettimeofday() as obsolete, recommending the use of clock_gettime(2) instead.

However, we would then need a test. Could you add one? tests/pass/libc.rs might be a reasonable location.

Ah, I didn't know it was obsolete. If that's the case, do you want to leave it as-is? I was just reading some code for a different fix and noticed this was restricted to macos when technically it doesn't need to be. Also happy to push this through add a test. Whatever you want / think is more correct!

Updated and added tests

obsolete

I meant in posix FWIW, as you pointed out.

CI is not happy, can you take care of that?

Yep, didn't see that libc exposed a different signature depending on platform.

Looks good. :) Can you squash this all into a single commit?

Done. Sorry for the back and forth, I've never really done anything with pointers in rust before. Appreciate the fast reviews!

No worries, thanks for sticking with me through the rounds of review! And congrats on your first Miri contribution. :)
@bors r+

@bors r+

:pushpin: Commit 73a1a27a450c0a7c0bfcab258230bb3165d9c05c has been approved by RalfJung

:hourglass: Testing commit 73a1a27a450c0a7c0bfcab258230bb3165d9c05c with merge 1b3332a0a68dc5e06bdaf7615d4205d75935914e...

:sunny: Test successful - checks-actions
Approved by: RalfJung
Pushing 1b3332a0a68dc5e06bdaf7615d4205d75935914e to master...


Update Android emulator for seamless integration, fixes #380

I followed steps from these pages

https://learn.microsoft.com/en-us/dotnet/maui/data-cloud/local-web-services?view=net-maui-8.0
https://learn.microsoft.com/en-us/dotnet/maui/platform-integration/communication/authentication?view=net-maui-8.0
https://stackoverflow.com/questions/67156558/webauthenticator-you-must-subclass-the-webauthenticatorcallbackactivity-and-c

It runs without error now in the MAUI App in the android emulator

@dotnet-policy-service agree

@michaelstonis  to review here please

@jamesmontemagno @normandev92 I like the changes here, but I wanted to break them out a bit to separate the functionality and incorporate it a little better with the overall app design.
I have kicked up two PRs that bring in the overall idea behind this PR.
https://github.com/dotnet/eShop/pull/399
https://github.com/dotnet/eShop/pull/401

Got final tweaks here: https://github.com/dotnet/eShop/pull/402 will close this out.
Thank you so much for this initial commit that led the way here, can you checkout all the updates on your side.


Is there a file size limit?

When uploading a 15 megabytes large eml file, I'm getting an empty response from the API.
The POST request to /api/analyze/file is sent, but after some time 0 B are returned and the loading circle in the GUI just disappears.
Please let me know if you need further details.

@andreas5232 Cloud you share the file with me, please?
Here is my contact address.
<EMAIL_ADDRESS>You can use PGP if you want to use it.

I checked the eml file and it is possible to analyze the file in my local env.
Note that eml-analyzer.herokuapp.com uses a Heroku's free dyno. So a CPU/memory heavy task is cannot executed in it.
Please consider to use the app in your local env. It is the straightforward solution for the issue.

That's strange, I'm already using a local docker env deployed using the guide specified in the readme. Do I have to increase any limits?

Sorry I tested it in the my laptop without docker and it did not work in the docker container as you said.
I cannot understand the root cause but it seems like an issue is lying on a combination of supervisor and gunicorn.
So I replaced supervisor + gunicorn with circus + uvicorn and the combination works fine.

@andreas5232 Could you git pull the repository again and test it, please?

Thank you for the fast response!
I have pulled and redeployed the docker container.
Now it loads the large eml file fine. 🎉 👍
One thing I'm still wondering about are the displayed attachment file sizes.
My computer says the eml has: 16.198.857 Bytes.
However the eml analyzer displays 17 attachments for this email with 16 times 15.4 Mb and one with 11.4 Mb.
That doesn't seem realistic to me.

@andreas5232 Just a confirmation, is your eml file a real world one? Or is it manipulated/crafted one?

The eml file is a real world scenario, which I found during my daily work as spamfilter admin.

Fixed. #59


jami-ontology-index module, problem with pom?

[INFO] Scanning for projects...
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR]   The project psidev.psi.mi.jami.ontology:jami-ontology-index:3.0.0-SNAPSHOT (/home/colin/psi-jami/jami-ontology-index/pom.xml) has 1 error
[ERROR]     Non-resolvable parent POM: Could not find artifact psidev.psi.mi:psimi-master:pom:1.8.6-SNAPSHOT and 'parent.relativePath' points at wrong local POM @ line 5, column 13 -> [Help 2]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException

that seems to have gone away


Update 002-search.md

@bradh good looking out. Thank you!
If you have any other feedback on the new docs as they are still coming together, we'd love to hear them.


POC extensions registration

POC extensions registration

Closed in favor of https://github.com/rancher/dashboard/pull/8460


[Release v2.8.2] Merge into Develop

Release Notes: v2.8.1 (3)
This release is due to a requirement to App Store to remove references to donations.  In it’s replacement, a reference to the Litecoin Foundation support page where the user can elect to send LTC if they so choose.

[Warmfix] Issue:146 Remove direct donation: support litecoin foundati…

Other Improvements

Fix crash in Amount formatter (#140)
[Bugfix] Renamed enum from State (#147) (#150)

Bugfixes

Renaming the enum caused a conflict with SwiftUI @State

Authors
Kerry Washington<EMAIL_ADDRESS>Mohamed Barry<EMAIL_ADDRESS>

Need to increase V number


MultiThreadedWallet emits events

Description
Workers forward events emitted from their wallet, on to the main thread. The main thread re-emits the event from their wallet.
Fixes #3262.
Fixes #3228.
To do this, I needed to make a significant change to the MultiThreadedWallet:

before, the WorkerManager would send an operation to the worker, listen for one 'message' event, and assume that the event contains the result of API call
after, the WorkerManager will send an operation to the worker, listen to 'message' events until the event contains the result of the API call

This allows the worker to also forward wallet events to the main thread.
With @lalexgap, I considered changing the type of the value that's sent back by the worker. This was ruled out, as it would have required a large change to the SingleThreadedWallet. It also would have been awkward to deal with the special case 'objectiveStarted' event emitted by _createChannel.
I also considered adding a messageId to the operation sent to the worker, so that we can listen for messages with the correct messageId. I judged this as overkill, as we are still restricting workers to API call at a time. We may wish to allow workers to handle concurrent API calls, depending on where the bottleneck lies, so this strategy may be employed in the future. This possibility is another reason why I chose this implementation.
To test this effectively, I had to make some changes to the PayerClient, to be able to configure it to use the MultiThreadedWallet.
I snuck in a bug fix where events were not properly emitted during pushMessage. This arose naturally while doing this work, which is a bit of scope creep. ⚠ This will result in multiple 'objectiveStarted' events being emitted by the wallet, exacerbated by apps using syncObjectives from #3275. I'd prefer to tackle this shortcoming separately.
Changes [Optional]
In addition to the main change outlined above,

Against my own recommendation, I included two additional bugfixes

pushMessage trigers wallet events
MultiThreadedWallet throws an error when passed an invalid configuration

PayerClient can now be configured with a partial wallet config
tests are added for the MultiThreadedWallet in two places

How Has This Been Tested? [Optional]

The e2e test now runs against both a single- and multi-threaded wallet
A test was added that checks that events are emitted during pushMessage. This test runs against both single- and multi-threaded wallets

:warning: Does this require multiple approvals? [Optional]
Please explain which reason, if any, why this requires more than one approval.

[ ] Is it security related?
[ ] Is it a significant process change?
[x] Is it a significant change to architectural, design?

This is a significant change to the MultiThreadedWallet. Considering there were previously no tests of the MultiThreadedWallet, it is probably worthwhile to get two reviewers.

Checklist:
Code quality

[x] I have written clear commit messages
[x] I have performed a self-review of my own code
[ ] This change does not have an unduly wide scope
* explained above
[x] I have separated logic changes from refactor changes (formatting, renames, etc.)
[x] I have commented my code wherever necessary (can be 0)
[x] I have added tests that prove my fix is effective or that my feature works, if necessary

Project management

[x] I have applied the appropriate labels
[x] I have linked to all relevant issues (can be 0)
[x] I have added all dependent tickets (can be 0)
[x] I have assigned myself to this PR
[x] I have chosen the appropriate pipeline on zenhub

Slightly related: one of the proposals I wrote last week was about wallet architecture (link), and would impact this work. The proposal is to split the majority of the wallet logic out into an engine, that is strictly request-response, and have a small "comms" wrapper that emits events:

I believe that if we did this it would make this change unnecessary, as we could set it up so that the comms code only runs in the main thread, and there would be no need for workers to submit events. I feel like this would probably be a less complex and more robust solution.

Slightly related: one of the proposals I wrote last week was about wallet architecture (link), and would impact this work. The proposal is to split the majority of the wallet logic out into an engine, that is strictly request-response, and have a small "comms" wrapper that emits events:

I believe that if we did this it would make this change unnecessary, as we could set it up so that the comms code only runs in the main thread, and there would be no need for workers to submit events. I feel like this would probably be a less complex and more robust solution.

While I'm not opposed to that, that work has not yet been planned, while this PR is resolving a high priority issue that blocks current sprint goals. I think the functional changes in this PR are easy to reverse, should we adopt your architecture.


Metric Relabeling and Values

Hello Everyone,
I am using Prometheus to monitor a RabbitMQ cluster and currently working on relabeling on some dynamic queues. I am grouping a bunch of queues into one generic queue label.
My question here is what happens to the values of those metrics that are getting relabeled?
Are there any options like average/sum for the values in the metrics while relabeling them?
Thank you!

The github issues are focused on bug reports :beetle: and this looks like more like a support request :+1:
I would recommend looking through the official docs and examples as well as searching in the google groups or asking in the irc channel and I am sure there will be someone to help you out.
irc: #prometheus
group: https://groups.google.com/forum/#!forum/prometheus-users
docs: https://prometheus.io/docs/introduction/overview/
Feel free to reopen if you are 100% convinced that is not a support request and a bug in Prometheus.


Uniform language highlight on helm chart README

This is just a formatting change, removed some white spaces and changed the language from console/bash to sh on all code blocks.

Thanks for the PR!


Add base class documentation to theme type variations

Fix functionality that when a theme type variation is added and a new base class is set, the documentation from the associated base class is added to the type variation. Associated with https://github.com/godotengine/godot/issues/94790

Not sure on who could be reviewing this @Calinou , got any suggestions?

@NicoCobb You rebased this PR incorrectly.

@NicoCobb You rebased this PR incorrectly. See Pull request workflow for more information.

Oh my bad, thank you for the notice. I didn't catch previously that I should be squashing everything into one commit and doing using amend rather than making additional commits, as well as I'd accidentally skipped fetch from master properly.
I think this should be now properly addressed, let me know if there's anything else to resolve or process steps I missed!
Also, this commit is still not ready for re-review as there's a caching issue I'm working on resolving related to this. Is there anything I should do to mark a PR as still not set for re-review, seeing that when I incorrectly rebased it automatically created a new review request?

I looked closely at the solution and it doesn't seem correct. It adds a temporary DocData when you change type variation. It only fixes the issue for the current editor session. After restarting, the descriptions are empty again.
Not sure what would be the correct solution though. The inspector should be somehow able to look up the base type of theme type variation.

I looked closely at the solution and it doesn't seem correct. It adds a temporary DocData when you change type variation. It only fixes the issue for the current editor session. After restarting, the descriptions are empty again.
Not sure what would be the correct solution though. The inspector should be somehow able to look up the base type of theme type variation.

Oh you're totally right that it does remove it on a new editor session, I'm looking to work on this the rest of today to try and wrap this up since I've taken a bit too long on this overall so I'll look into if there's a good way to maintain it between editor sessions.
I'll have the godot contributor chat open all day where I'm Rakali3 so feel free to ping me here or there if you maybe have any suggestions on how to alter my approach to this


Update s.js

Corrected a company name aSfeboda to the right spelling safeboda
Kindly merge @gray-adeyi @Chaos10001 @OSCA-Ado-Ekiti

Thanks for that.


How to hide #search-hits until a keypress is detected by #search-searchbar?

Hi everyone,
What's a good way to hide search results until either the first keypress is detected, or a return keypress is detected, by the searchbar?
I'm using Jekyll on GitHub pages with the Minima theme. Thank you!

Hi @bjpcjp it seems like you found the answer to your issue 😄


Add Fail2ban and PASD

I'm looking into replacing fail2ban with crowdsec. https://github.com/crowdsecurity/crowdsec


[POAE7-2928] Add simple join benchmark to tpch framework

What changes were proposed in this pull request?
Add join query based on tpc-h, nation.n_nationkey ⋈ customer.c_nationkey.
We can run this query by following steps:

Turn ENABLE_BENCHMARK on under cpp/src/cider-velox/CMakeLists.txt
make release
Cider: /TpchInMemBenchmark -nopartial_agg_pattern -left_deep_join_pattern -run_query_verbose=23 -scale_factor=1.
Velox: ./TpchInMemBenchmark -nopartial_agg_pattern -run_query_verbose=23 -scale_factor=1.

Why are the changes needed?
Improvement.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
UTs.
Which label does this PR belong to?

Gentle ping @harborn for review, thanks!


When .from_hash argument has a key with -, an error occurs

I have issues
When .from_hash argument has a key with -, an error occurs.
$ bundle exec irb
irb(main):001:0> require 'type_struct'
=> true
irb(main):002:0> Type = TypeStruct.new('hoge-id': String)
=> Type
irb(main):003:0> Type.from_hash('hoge-id': '123456')
Traceback (most recent call last):
       16: from /Users/tsub/.asdf/installs/ruby/2.6.3/lib/ruby/gems/2.6.0/gems/bundler-2.0.2/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
       15: from /Users/tsub/.asdf/installs/ruby/2.6.3/lib/ruby/gems/2.6.0/gems/bundler-2.0.2/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
       14: from /Users/tsub/.asdf/installs/ruby/2.6.3/lib/ruby/gems/2.6.0/gems/bundler-2.0.2/lib/bundler/cli.rb:465:in `exec'
       13: from /Users/tsub/.asdf/installs/ruby/2.6.3/lib/ruby/gems/2.6.0/gems/bundler-2.0.2/lib/bundler/cli/exec.rb:28:in `run'
       12: from /Users/tsub/.asdf/installs/ruby/2.6.3/lib/ruby/gems/2.6.0/gems/bundler-2.0.2/lib/bundler/cli/exec.rb:74:in `kernel_load'
       11: from /Users/tsub/.asdf/installs/ruby/2.6.3/lib/ruby/gems/2.6.0/gems/bundler-2.0.2/lib/bundler/cli/exec.rb:74:in `load'
       10: from /Users/tsub/.asdf/installs/ruby/2.6.3/bin/irb:23:in `<top (required)>'
        9: from /Users/tsub/.asdf/installs/ruby/2.6.3/bin/irb:23:in `load'
        8: from /Users/tsub/.asdf/installs/ruby/2.6.3/lib/ruby/gems/2.6.0/gems/irb-1.0.0/exe/irb:11:in `<top (required)>'
        7: from (irb):3
        6: from /Users/tsub/sandbox/reproduce_typestruct_error/vendor/bundle/gems/type_struct-0.7.4/lib/type_struct.rb:77:in `from_hash'
        5: from /Users/tsub/sandbox/reproduce_typestruct_error/vendor/bundle/gems/type_struct-0.7.4/lib/type_struct.rb:77:in `new'
        4: from /Users/tsub/sandbox/reproduce_typestruct_error/vendor/bundle/gems/type_struct-0.7.4/lib/type_struct.rb:21:in `initialize'
        3: from /Users/tsub/sandbox/reproduce_typestruct_error/vendor/bundle/gems/type_struct-0.7.4/lib/type_struct.rb:21:in `each'
        2: from /Users/tsub/sandbox/reproduce_typestruct_error/vendor/bundle/gems/type_struct-0.7.4/lib/type_struct.rb:29:in `block in initialize'
        1: from /Users/tsub/sandbox/reproduce_typestruct_error/vendor/bundle/gems/type_struct-0.7.4/lib/type_struct.rb:29:in `instance_variable_set'
NameError (`@hoge-id' is not allowed as an instance variable name)

NameError (@hoge-id' is not allowed as an instance variable name) is raised.
By the way, as an example of Hash key containing -, AWS Lambda event.

Environment details

Ruby: 2.6.3
type_struct: 0.7.4

Thank you for reporting!
I'll fix it later.

@tsub v0.8.0 released!
Please try again!

I was able to confirm that it came to work as intended.
$ bundle update type_struct
Fetching gem metadata from https://rubygems.org/.
Resolving dependencies...
Using bundler 2.0.2
Fetching type_struct 0.8.0 (was 0.7.4)
Installing type_struct 0.8.0 (was 0.7.4)
Bundle updated!

$ bundle exec irb
irb(main):001:0> require 'type_struct'
=> true
irb(main):002:0> Type = TypeStruct.new('hoge-id': String)
=> Type
irb(main):003:0> Type.from_hash('hoge-id': '123456')
=> #<Type hoge-id="123456">
irb(main):004:0> Type.from_hash('hoge-id': '123456')['hoge-id']
=> "123456"

@ksss Many thanks!


custom server side domain name for the Google Tag Manager

We can change server side domain name for the Google Tag Manager to circumvent adblockers. So the new tracking server is located at some custom domain.
Background to above: when adblockers or browsers see googletagmanager.com (or other domains or code refering to tracking, they will block it). Having custom domain means that no blockers will match on this url and it will approvedpproved.

+1

Also transport_url option would be great to add because of the new google server side proxy feature.
Nevertheless +1 here.

@alinemorelli any timeline to merge this change. Its been while since google had added option for server side GTM.

@alinemorelli could you merge this? It would help a lot

@alinemorelli I also faced the problem with the server-based tagging and need this functionality. How can I help you to merge this PR?

I needed to get a solution working quickly, so I created a fork that allows you to customize the url:
https://github.com/darjama/react-gtm-custom-domain
https://www.npmjs.com/package/react-gtm-module-custom-domain

Yeah I need this too now, thanks @darjama for your fork!

+1


How to custom ios dialog share like Skype or Facebook

Hi thanks for your library
I want to custom ios share dialogs like Skype or FB. How can I do it?
Thanks

You may be best looking for an iOS dev, it looks like it's a separate Xcode project (as an "extension"), it may be a good starting point, but prob not easy to extend or customise directly.

I'd be interested to see/hear if you stumble across a simple way to bypass any dialog - I just want to send the current url from the browser into my app.
Works flawlessly in Android, but iOS seems to have this pesky dialog with a thumbnail.

I'm trying to custom it but I'm not swift developer. And I want to check loggedIn in your ShareViewControllerSwift but I can't
How I can do that? I search gg and it not working  with library

You might be able to reference the code from https://github.com/ajith-ab/react-native-file-share-intent#readme. It is the precursor to react-native-receive-sharing-intent and it set up the share extension as a react-native app.


[Feature Request]: apply kubernetes the app

Is there an existing issue for this?

[X] I have searched the existing issues

Feature Description
as this app is dockerize then we can add kubernetes into this after this your website do self healing when any error occur and also do load balancing

i make a docker image on docker hub
i write a deployment.yaml file which deploy your app
i write a service.yaml file which help in applying load balancing the website

Use Case
it is used when we want to handle more traffic and we want that without shutdown we can update our app
Benefits
No response
Add ScreenShots
No response
Priority
High
Record

[X] I have read the Contributing Guidelines
[X] I'm a GSSOC'24 contributor
[X] I have starred the repository

Is there an existing issue for this?

[x]  I have searched the existing issues

Feature Description
as this app is dockerize then we can add kubernetes into this after this your website do self healing when any error occur and also do load balancing

i make a docker image on docker hub
i write a deployment.yaml file which deploy your app
i write a service.yaml file which help in applying load balancing the website

Use Case
it is used when we want to handle more traffic and we want that without shutdown we can update our app
Benefits
No response
Add ScreenShots
No response
Priority
High
Record

[x]  I have read the Contributing Guidelines
[x]  I'm a GSSOC'24 contributor
[x]  I have starred the repository

Hello @agarwalhimanshugaya , Go for it
All the Best


Weird error notification when checking for updates when clock is out of sync

I opened my parallels Windows VM that hasn't been turned on in a month and when I tried to check for updates i kept getting this weird error. Turns out my clock had reset back to the last time I had used it and after syncing the time that update service started working. However, I had never seen this error message before and it would be great if this was more human and less 🤖 .

If this is really related to the clock, the issue is somewhere deep in Chrome or Window's network stack, since none of our code depends on the clock. Closing it as it seems a very infrequent occurrence.


Suggession for Dynamic Information View

Hi,
Is there any variable to hide the top view where the info written. I don't want to crossed view.
Please see the image.

The view you marked crossed would be hidden when the hint variable is set to nil.


Fedora 42 dropped JDK 11 and 17 support

JDK 11 and 17 are removed from the Fedora 42 repo:
The java-17-openjdk package is deprecated and may no longer receive updates. Since f42 install adoptium-temurin-java-repository and install temurin-17-jre
https://fedoraproject.org/wiki/Changes/ThirdPartyLegacyJdks#adoptium-temurin-java-repository

The java-11-openjdk package is deprecated and may no longer receive updates. Since f42 install adoptium-temurin-java-repository and install temurin-11-jre
https://fedoraproject.org/wiki/Changes/ThirdPartyLegacyJdks#adoptium-temurin-java-repository

Even though the base image is still Fedora 40, the JDK 17 build fails. — temporary build error.

JDK 11 EOL'd. JDK 17 is relevant once we update to Fedora 42 – we might deprecate / EOL it then too.


split controllers in more file

Hi
im using silex php framework,
i have 2 file:
the first is the app.php with this:
<?php
require_once __DIR__.'/../vendor/autoload.php';
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\HttpFoundation\ParameterBag;
use Silex\ControllerProviderInterface;
$app = new Silex\Application();
$app['debug'] = true;
$app->register(new Silex\Provider\DoctrineServiceProvider(), array(
'db.options' => array(
    'driver'    => 'pdo_mysql',
    'host'      => 'localhost',
    'dbname'    => 'testapp',
    'user'      => 'root',
    'password'  => 'turoke55u',
    ),
));
$app->before(function (Request $request) {
     if (0 === strpos($request->headers->get('Content-Type'), 'application/json')) {
        $data = json_decode($request->getContent(), true);
        $request->request->replace(is_array($data) ? $data : array());
    }
});
$app->mount('/', include 'login.php');
$app->run();

and the second is the login.php:
<?php
use Symfony\Component\HttpFoundation\Request;
$login = $app['controllers_factory'];
$login->post('/apiv1/user/login', function (Request $request) use ($login) {
    $userinfo = array(
        'email'  => $request->request->get('email'),
        'mode'  => $request->request->get('mode'),
        'password'  => $request->request->get('password'),
    );
    $passwordcoding = sha1($userinfo['email']."66643asd");
    $emailverification = "SELECT email,password FROM user WHERE email='".$userinfo['email']."'";
    $selectemail = $login['db']->$fetchAll('SELECT * FROM user');
    var_dump($emailverification);
});
return $login;

when i run the select on db i receive this error:
Cannot use object of type Silex\ControllerCollection as array in /mnt/hgfs/Share_Folder/frontend/src/login.php on line 13

solutions?
and the second questions is why if i change in app.php from this
$app->mount('/', include 'login.php');

to this:
$app->mount('/apiv1/user/login', include 'login.php');

and in the login.php
$login->post('/',

like the silex documentation, the framework doesnt works??
thanks.

Hello, your $login variable is not the same as your $app variable.
As the error message suggests, your login var is an instance ControllerCollection class, it is not an Application or a Pimple one.
You have to use $app instead of $login in your controller function
$login->post('/apiv1/user/login', function (Request $request) use ($app) {
    $userinfo = array(
        'email'  => $request->request->get('email'),
        'mode'  => $request->request->get('mode'),
        'password'  => $request->request->get('password'),
    );
    $passwordcoding = sha1($userinfo['email']."66643asd");
    $emailverification = "SELECT email,password FROM user WHERE email='".$userinfo['email']."'";
    $selectemail = $app['db']->$fetchAll('SELECT * FROM user');
});

i have changed the code like this:
<?php
$app = $app['controllers_factory'];
$app->post('/', function (Request $request) use ($app) {
$userinfo = array(
'email'  => $request->request->get('email'),
'mode'  => $request->request->get('mode'),
'password'  => $request->request->get('password'),
);
$passwordcoding = sha1($userinfo['email']."66643asd");
$emailverification = "SELECT email,password FROM user WHERE email='".$userinfo['email']."'";
$selectemail = $app['db']->$fetchAll('SELECT * FROM user');
var_dump($emailverification);
});
return $app;
and now i receive this error:
[Sun Jan 17 05:01:27 2016] [error] [client <IP_ADDRESS>] PHP Warning:  Missing argument 1 for Silex\\Route::run() in /mnt/hgfs/Share_Folder/frontend/vendor/silex/silex/src/Silex/Route.php on line 51
[Sun Jan 17 05:01:27 2016] [error] [client <IP_ADDRESS>] PHP Notice:  Undefined variable: to in /mnt/hgfs/Share_Folder/frontend/vendor/silex/silex/src/Silex/Route.php on line 53
[Sun Jan 17 05:01:27 2016] [error] [client <IP_ADDRESS>] PHP Warning:  Missing argument 1 for Silex\\Route::run() in /mnt/hgfs/Share_Folder/frontend/vendor/silex/silex/src/Silex/Route.php on line 51
[Sun Jan 17 05:01:27 2016] [error] [client <IP_ADDRESS>] PHP Notice:  Undefined variable: to in /mnt/hgfs/Share_Folder/frontend/vendor/silex/silex/src/Silex/Route.php on line 53

$app = $app['controllers_factory'];

This is wrong, you are overriding your app variable.
Use $app variable only inside your controller function (use $app as a container), and for your factory creation. Use $login to create your route.
$login= $app['controllers_factory'];
$login->post('/', function (Request $request) use ($app) {
    $userinfo = array(
        'email' => $request->request->get('email'),
        'mode' => $request->request->get('mode'),
        'password' => $request->request->get('password'),
    );
    $passwordcoding = sha1($userinfo['email']."66643asd");
    $emailverification = "SELECT email,password FROM user WHERE email='".$userinfo['email']."'";
    $selectemail = $app['db']->$fetchAll('SELECT * FROM user');
    var_dump($emailverification);
});
return $login;

ok works but, now the app.php have this code:
$app->mount('/', include __DIR__.'/login.php');

and the in the login.php
use Symfony\Component\HttpFoundation\Request;<-- i needed to add this
$login->post('/apiv1/user/login', function (Request $request) use ($app) {

in the documentation it's write like this:
app.php
 $app->mount('/apiv1/user/login', include __DIR__.'/login.php');

login.php
 $login->post('/', function (Request $request) use ($app) {

but if i write like the documentation i recieve a 404 only on a post request the get works fine.why?

Because the "post" in "$login->post()" sets the allowed method for that route to POST. The documentation explains this pretty well: http://silex.sensiolabs.org/doc/usage.html#routing

ok thanks. you can close this issue

Can someone close this issue?


markdown: adopt code to pass newly generated tests

Signed-off-by: Jasstkn<EMAIL_ADDRESS>
[x] fix tests for markdown.go (students should refactor this code so it should be as bad as possible)
[x] copied markdown.go to examplar.go to pass CI tests

@andrerfcsantos Hello. Could you help me with tests in CI? locally everything pass:
go test .
ok      markdown        0.183s

Hey, I have also found a solution passing the tests (#2341), maybe we can work together providing the stub markdown.go that has to be refactored.

Hey, I have also found a solution passing the tests (#2341), maybe we can work together providing the stub markdown.go that has to be refactored.
In CI the solution in example.go is used and it should show that the problem is solvable (does not matter how clean). The markdown.go is only the file that is the starting point for the student. In this case the starting point is a existing code that passes all tests and that should be refactored.

Yes, I understand the difference between those 2 files. I think the examplar solution should somehow implement the language's idiomatic principles that's why I didn't want to just copy Python's code approach.
Also I checked your markdown.go and code there seems to be too good. I think this exercise should contain only one function and be badly written.

What about using your markdown.go and my example.go?

What about using your markdown.go and my example.go?

Yep, good idea. Do you want to push it to my branch? So you input represented as well or otherwise?

It would be nice if you could fix your CI, so your PR can be merged (I think you only need to copy the content of markdown.go to example.go). After your PR is merged, mine can be merged

It would be nice if you could fix your CI, so your PR can be merged (I think you only need to copy the content of markdown.go to example.go). After your PR is merged, mine can be merged

Okay, I'll copy your example.go then.

Why did you copy my version? My version will be merged here: #2341

@eklatzer done.
@andrerfcsantos Could you review what we have here?

Why did you copy my version? My version will be merged here: #2341

My bad. I misunderstood what you mean.

Thanks, why did you delete gen.go?

Thanks, why did you delete gen.go?

I guess it caused some issues for me to test it locally. Restored.
I don't know if I should fix linter's errors regarding style. What do you think?

I don't know if I should fix linter's errors regarding style. What do you think?

I think only changes passing CI are merged, so I think you have to

I don't know if I should fix linter's errors regarding style. What do you think?

I think only changes passing CI are merged, so I think you have to

it's green now ✅

Thanks


New Shapefile dbf much larger than original

This issue was imported from CodePlex
rajs wrote 2011-01-28 at 23:00
As part of my application I have to create new shapefiles representing the features that interest a user specified extent. One of my use cases includes all the features. In this case the new and original shapefiles appear be identical as far as number
of shapes and content of the dbf. The issue I am seeing is that the new dbf is significantly larger than the original. Why is this? Is there a work around?
Here is a snippet of the code used to create the shapefile:
var envelope = new Envelope( -180.0, 180.0, -90.0, 90.0);
var extent = new Extent(envelope);

// Set up the source featureset
var source = Path.Combine(sourceDirectory, layer);
var sourceFs = new FeatureSet();
//sourceFs.FillAttributes();
sourceFs.Open(source);

if (sourceFs.IndexMode)
{
    // Set up the destination featureset
    var destination = Path.Combine(destinationDirectory, layer);
    var destinationFs = new FeatureSet(sourceFs.FeatureType) { Projection = sourceFs.Projection };
    destinationFs.CopyTableSchema(sourceFs.DataTable);

    var indicies = sourceFs.SelectIndices(extent);

    foreach (int index in indicies)
    {
        var feature = sourceFs.GetFeature(index);
        var newfeature = destinationFs.AddFeature(feature);
        newfeature.CopyAttributes(feature);
    }

    sourceFs.Close();
    sourceFs.Dispose();

    destinationFs.SaveAs(destination, true);
    destinationFs.Close();
    destinationFs.Dispose();

    return;
}
// Should not get here, check logs for this entry
Log.WarnFormat("SourceFeatureSet not opened in IndexMode");

Shade1974 wrote 2011-01-28 at 23:31
The only thing that I can think of right off the top of my head is that the copy table schema is not capturing the original "Length" of each field. So string fields might be defaulting to 255 characters or something. That could cause a lot of
empty space. An easy way to check this is to simply open up the two DBF files using something like OpenOffice. Microsoft Excel can technically open them, but won't show you what the length of each field is. Open office shows the length of each field when you
open the document. It also lets you change that pretty easily and actually save to the dbf format. I use it a lot when working with shapefiles.
Ted
rajs wrote 2011-01-29 at 00:44
This is exactly what is going on. All 'C' or character fields are now 255 in length. In addition several of the 'N' or number fields have their precision changed for example area was N 19.11 and is now N 18.6. Based on this it looks like their is an issue
with the CopyTableSchema method.
Is this approach I have correct for copying features (geometry + data) to a new shapefile?
-Raj
Shade1974 wrote 2011-01-29 at 01:31
If you want to wrest more control in the process, check to see if you can cast the newly created "DataColumn" as a DotSpatial.Data.Field. If you can, then you can specify the Length and the Decimal for the number format. It appears that the original
copyschema is only copying the .Net aspects like data type.
Ted
rajs wrote 2011-01-29 at 06:48
Your suggestion was spot on. I have created a custom copytableschema method that handles this issue as opposed to using the FeatureSet.CopySchema method. My method is as follows:
    private void CopyTableSchema(FeatureSet sourceFs, FeatureSet destinationFs)
    {
        int cols = sourceFs.DataTable.Columns.Count;
        for (int i = 0; i < cols; i++)
        {
            Field srcField = (Field)sourceFs.DataTable.Columns[i];
            DataColumn dataColumn = new DataColumn(srcField.ColumnName, srcField.DataType);
            Field field = new Field(dataColumn) { Length = srcField.Length, DecimalCount = srcField.DecimalCount };
            destinationFs.DataTable.Columns.Add(field);
            field.Dispose();
            dataColumn.Dispose();
            srcField.Dispose();

        }
    }

I look forward to seeing this fixed in the FeatureSet.CopySchema at some point.
-Raj
Shade1974 wrote 2011-01-31 at 18:12
I think we can do something like this, but we must account for the situation where the columns are not also Fields. For instance, copy table schema can be used on a standard DataTable that is created in .Net through some other method. In such a case, it
would break when you cast it to a Field. But we can use a safe cast + null check approach to incorporate this extra condition testing into the program. Thanks for posting your solution.
Ted

Not reproduced.


Fix/158 is array

Fixed proxying so that shared values uses proxies, while all other wrapping doesn't. This solves issues described in #158
Closes #158

Is that PR ready?


When a node connects multiple child nodes, all child nodes will execute repeatedly

Suppose I have a node A, and A have a out port connected to child nodes B1,B2,B3,B4.
If I disconnect and reconnect A and B1 (or add a child node and connect to A), node A will triggered the onDataUpdated() event, then all child nodes who connect to A will all be executed, even I just reconnect A and B1, but B2,B3,B4 will all be executed too.
If I restore from .flow file, B1 will execute one time, B2 will execute two times, B3 will execute 3 times, B4 will execute 4 times! Normally, B1~B4 will all execute just one time!

That's a good observation, I'll check and address it. Thanks

Fixed in master.


feat(issue): paginate issue comments and events

Question
Response

Version?
v1.6.0

Devices tested?
iPhone 6 Plus Simulator

Bug fix?
no

New feature?
yes

Includes tests?
no

All Tests pass?
yes

Related ticket?
#751

Screenshots
There are several screens that have been refactored.

[x] edit-issue-comment
[x] issue-settings
[x] issue
[x] new-issue
[x] pull-merge

Before
After

Description

This PR loads paginated issue comments and events together by the new API client. Feeling hard to keep sorting by time with two pagination APIs, so I choose the issue timeline API, even though it is still in preview version.
To make this PR as small as possible, feature improvements can include but do not limit to,

fetching pull request information by GraphQL
fetching repository labels by the new API client
designing and controlling navigation params better

Coverage decreased (-0.09%) to 51.104% when pulling 2335a645c9545013f538e202d55dbf2a4109b0e4 on chinesedfan:paginate-issue-comments into a0d6b06fedfc8812ae6fce642cfa8f8f6d314668 on gitpoint:master.

This is so so great, thank you so much. Something we've needed in the app for some time for these screens :)
Will take a look by tonight 🙌

@chinesedfan thank you for the excellent work, but I do not yet have the opportunity to check it.


make vmcache can work with vsock

I think you may re-title the issue as "enable vsock for vmcache" because you just need to turn on the switch 😂


--cleanup option deletes files just added by export/--update. Maybe related with accented album names?

SUPER WORK! EXCELLENT TOOL!!!
Before submitting a bug report, please ensure you are running the most recent version of osxphotos and that the bug is reproducible on the latest version

Running version 0.43.8

Describe the bug
Following an export and update of the albums; it then goes into the cleanup activity and deletes files that where just loaded.
It only removes the files and then album folder (when they become empty) for some Albums. Only thing I can note is that the Album's names are accented.
To Reproduce
Steps to reproduce the behavior:

What' the full command line you used with osxphotos?

osxphotos export /Volumes/photo-1/Family.Photos --db "/Users/Shared/Pictures/iPhoto Shared Library.photoslibrary" --load-config colibri.toml  > so.log 2>se.log &
colibri.toml is as follows

[export]
cleanup = true
directory = "{album[/,.|:,.]}"
exiftool = true
exiftool_merge_keywords = true
exiftool_merge_persons = true
from_date = 2021-01-01T00:00:00
keyword_template = [ "{keyword}",]
report = "colibri.csv"
skip_original_if_edited = true
strip = true
update = true
verbose = true

What was the error output?

Expected behavior
Would expect the just uploaded files not to be deleted.
Screenshots
Please note the sample std output I get. You can see the pic DSC03660.JPG being exported and then being deleted.
Loaded options from file colibri.toml
osxphotos version 0.43.8
exiftool path: /usr/local/bin/exiftool
Using export database /Volumes/photo-1/Family.Photos/.osxphotos_export.db
Processing database /Users/Shared/Pictures/iPhoto Shared Library.photoslibrary/database/photos.db
Processing database /Users/Shared/Pictures/iPhoto Shared Library.photoslibrary/database/Photos.sqlite
Database locked, creating temporary copy.
Processing database.
Database version: 6000, 7.
Processing persons in photos.
Processing detected faces in photos.
Processing albums.
Processing keywords.
Processing photo details.
Processing import sessions.
Processing additional photo details.
Processing face details.
Processing photo labels.
Processing EXIF details.
Processing computed aesthetic scores.
Processing comments and likes for shared photos.
Processing moments.
Done processing details from Photos library.
Exporting 3099 photos to /Volumes/photo-1/Family.Photos...

(...)

Exporting DSC03660.JPG (DSC03660.JPG) as DSC03660.JPG
Writing metadata with exiftool for /Volumes/photo-1/Family.Photos/2021-04-09 Tróia/DSC03660.JPG
Exported new file /Volumes/photo-1/Family.Photos/2021-04-09 Tróia/DSC03660.JPG

(...)

Processed: 3099 photos, exported: 622, updated: 0, skipped: 2477, updated EXIF data: 622, missing: 0, error: 0
Elapsed time: 2712.842 seconds
Cleaning up /Volumes/photo-1/Family.Photos
Deleting /Volumes/photo-1/Family.Photos/2021-04-09 Tróia/DSC03660.JPG
Deleting /Volumes/photo-1/Family.Photos/2021-04-09 Tróia/DSC03628.JPG
Deleting /Volumes/photo-1/Family.Photos/2021-04-09 Tróia/DSC03663.JPG
(...)

Desktop (please complete the following information):

OS: [e.g. which version of macOS]: macOS Monterey 12.1
osxphotos version (osxphotos --version): 0.43.8

Additional context

I'm exporting into a mounted volume off a Synology DSM 7 NAS Server.
The locale settings are as follows

LANG="en_GB.UTF-8"
LC_COLLATE="en_GB.UTF-8"
LC_CTYPE="en_GB.UTF-8"
LC_MESSAGES="en_GB.UTF-8"
LC_MONETARY="en_GB.UTF-8"
LC_NUMERIC="en_GB.UTF-8"
LC_TIME="en_GB.UTF-8"
LC_ALL=

Tried to reproduce with just a single Album (accented) but was not exactly able to. Will run more tests!
Also note all my files is Photos are "Referenced" files and the library was migrated from iPhoto.
Looking for file DSC03660 on dates file .osxphotos_export.db I get:

5874|2021-04-09 Tróia/DSC03660.JPG|2021-04-09 tróia/dsc03660.jpg|48624BED-877B-46BF-B03A-B91D762EE371|32768|4548022|1618136601.0|32768|4493835|1618136601.0

Thanks for the detailed bug report. It's quite possible this has to do with accented characters -- Unicode characters have been at the root of several osxphotos issues, partly because the representation used by Photos is at times different than that used by the filesystem.  The test suite already uses albums with accented characters to catch this sort of thing but it's possible there are still edge cases. I'll see if I can reproduce this.

I was able to replicate this in an album with accented characters.  Now I just need to track down why this is happening...

@all-contributors add @oPromessa for bug

I think this is fixed in v0.43.9

Super. Will upgrade and try it out!

Seems to have done the trick! Thanks for the prompt reply!

Hi there. Sorry to bother.
Still related to this I have an album:

with accented characters Água
but not only that but it also contains multiple pics (3x) with the same name, e.g. AAF035.jpg
My files are all reference files. So their original location is in different folders, or course!

I'm testing exporting to the MAC local disk to avoid conflicts on accessing remote folders with other issues
First run I make it correctly generates new names for the 3 files.
Água/AAF035 (1).jpg
Água/AAF035 (2).jpg
Água/AAF035.jpg

Second time around two new files are created and the previous two are deleted.
Água/AAF035 (3).jpg
Água/AAF035 (4).jpg
Água/AAF035.jpg

Deleting /Users/(...)/Água/AAF035 (1).jpg
Deleting /Users/(...)/Água/AAF035 (2).jpg

Third run it does a similar process.
Água/AAF035 (1).jpg
Água/AAF035 (2).jpg
Água/AAF035.jpg

Deleting /Users/(...)/Água/AAF035 (3).jpg
Deleting /Users/(...)/Água/AAF035 (4).jpg

Let me know If I can run it in debug mode to collect more traces to facilitate troubleshooting

No worries. No stress here.
Would the option --overwrite  help?

I think 0.45.4 fixes the problem with files and albums with accented characters.  I tested the exact situation you described with accented album name and photos containing the same name and it works as expected. Please upgrade and give that a try.

I've ran some tests and limited the export to just the 3 files (with --regex on template  "{album}/{name}" -- which works like a charm. YAY!!)
With 0.45.4 problem still seems to persist.
2022-02-05 10:29:39.666436 -- Loaded options from file colibri.toml
2022-02-05 10:29:39.666522 -- osxphotos version 0.45.4
2022-02-05 10:29:39.670758 -- Using export database /Users/Shared/Pictures/Export/Photos.2021/Test.Album/.osxphotos_export.db
(...)
2022-02-05 10:30:04.980072 -- Done processing details from Photos library.
Exporting 3 photos to /Users/Shared/Pictures/Export/Photos.2021/Test.Album...
2022-02-05 10:30:52.499233 -- Exporting AAF035.jpg (AAF035.jpg) as AAF035.jpg (1/3)
2022-02-05 10:30:52.505697 -- Skipped up to date file /Users/Shared/Pictures/Export/Photos.2021/Test.Album/Água/AAF035.jpg
2022-02-05 10:30:52.506755 -- Exporting AAF035.jpg (AAF035.jpg) as AAF035.jpg (2/3)
2022-02-05 10:30:52.816523 -- Writing metadata with exiftool for AAF035 (3).jpg
2022-02-05 10:30:53.027365 -- Exported new file /Users/Shared/Pictures/Export/Photos.2021/Test.Album/Água/AAF035 (3).jpg
2022-02-05 10:30:53.027687 -- Touched date on file /Users/Shared/Pictures/Export/Photos.2021/Test.Album/Água/AAF035 (3).jpg
2022-02-05 10:30:53.029392 -- Exporting AAF035.jpg (AAF035.jpg) as AAF035.jpg (3/3)
2022-02-05 10:30:53.133867 -- Writing metadata with exiftool for AAF035 (4).jpg
2022-02-05 10:30:53.254227 -- Exported new file /Users/Shared/Pictures/Export/Photos.2021/Test.Album/Água/AAF035 (4).jpg
2022-02-05 10:30:53.254284 -- Touched date on file /Users/Shared/Pictures/Export/Photos.2021/Test.Album/Água/AAF035 (4).jpg
Processed: 3 photos, exported: 2, updated: 0, skipped: 1, updated EXIF data: 2, missing: 0, error: 0, touched date: 2
Elapsed time: 0:00:00
Cleaning up /Users/Shared/Pictures/Export/Photos.2021/Test.Album
2022-02-05 10:30:53.254845 -- Deleting /Users/Shared/Pictures/Export/Photos.2021/Test.Album/Água/AAF035 (1).jpg
2022-02-05 10:30:53.255010 -- Deleting /Users/Shared/Pictures/Export/Photos.2021/Test.Album/Água/AAF035 (2).jpg
Deleted: 2 files, 0 directories

This was a hard one to track down!  I think I've finally fixed it.  Working on cleaning up the test cases since I had to modify the test Photos libraries.  I created a special test case using the settings from your .toml and the same album/filenames just to be sure this worked.
tests/test_cli.py::test_export_cleanup_exiftool_accented_album_name_same_filenames PASSED                                            [100%]

The problem was unicode representation in the export database--sqlite wasn't using the same representation as the file system. I've normalized unicode everywhere I can think of now.
But...this means your export db will be invalid and you'll still have the issue when you update until the second update which will have the fixed filenames recorded in the export db.

But...this means your export db will be invalid and you'll still have the issue when you update until the second update which will have the fixed filenames recorded in the export db.

Never mind....I built a migration into the ExportDB() class so it'll automatically normalize all the old filenames if needed :-)

Cool. Sorry for that!

For "update until the second updated which"... it-s okay. It-a about 160 or so pics in that situation. That's fine.
For the export db will be invalid...

invalid only for the same name pics

This case would be fine as I can run the --update a couple of times and get the 160 or so pics fixed!

of invalid for all the pics

In this case, if I delete the .osxphotos_export.db file and keep the files already copied on the destination:

Would osphotos simply rebuild the database without copying the files?
Or I'll simply have to delete everything and restart the whole copy?

FYI, as you mentioned the .toml file. I've added a few flags which I feel would not impact the scenario but  just in case here are the additional fields/options.
touch_file = true
not_hidden = true

FYI, as you mentioned the .toml file. I've added a few flags which I feel would not impact the scenario but just in case here are the additional fields/options.

Thanks -- I'll add those just in case!
With the automatic migration, yes, everything should "just work" with the next update.

I think it's fixed for real in v0.45.5.  Give that a try and let me know if it works!

On it...

On my small test with previously exported pics he kept the numbering that was already there (3) and (4).
Ran it three times with same result.

So we have he issue solved.
The naming is not as per the minimum which should have been Água/AAF035 (1).jpg and Água/AAF035 (1).jpg but on a fresh copy it works as expected. Don't believe this to be an issue.
Will run it on the larger set...

Água/AAF035 (3).jpg
Água/AAF035 (4).jpg
Água/AAF035.jpg

The naming is not as per the minimum which should have been Água/AAF035 (1).jpg and Água/AAF035 (1).jpg but on a fresh copy it works as expected. Don't believe this to be an issue.

Yeah, I can't really resolve that easily without really complicating the code that matches existing files to photos.

No worries. It's just great as it is!


[Docs]: Documentation to support datadog for logging in appsmith

Is there an existing issue for this?

[X] I have searched the existing issues

Summary
Given the log management datadog provides. It would be great to integrate and write documentation around how to integrate datadog to analyze appsmith logs
Why should this be worked on?
Good to have feature

Any updates on this feature request?
we also would like to have this!


Add Mac OSX testing for Travis CI

The failure is due to https://github.com/rust-lang/rust/issues/42422. Until it gets resolved I think we should allow failures for the beta channel as well.

I agree, also could you create a .editorconfig file (http://editorconfig.org/) with 4 spaces for indentation and final newline as a part of the project bootstrap issue. Will help keep consistency across editors and prevent accidental diffs.


Add Heap analytics tracking

Overview
Adds Heap analytics tracking to all pages on the app. Also includes a badge on the landing page to allow us to track 50,000 sessions per month for free.
Checklist

[x] PR has a name that won't get you publicly shamed for vagueness
[x] Files changed in the PR have been yapf-ed for style violations

Demo

Testing Instructions

Browse to the web app
Ensure the Heap badge is visible on the landing page
Open up your network tab, filter by 'heap'
Reload the page and ensure there's some Heap network activity
Login, browse around the app and ensure there's Heap network activity throughout

Closes #157646460

This seems to work well, but I think the light badge would look better. The dark one is too visually overpowering for my taste.
Here's a screenshot with the light badge:

If you want to change it you just have to change the image file name to badgeLight.png.

@tgilcrest: You specifically requested the dark badge. I agree that the light badge looks better, but this page styling will be fully revamped, so maybe the point is moot. Let me know which you'd prefer and I can make the change if desired.

They requested the Dark badge because they felt it worked better with the aesthetic that they are moving towards with the Draw the Lines site. We still have some outstanding work to re-style the landing page to match DTL once we get those assets from P'unk Ave, so what it looks like currently will change.


Reproduction of MS-MARCO passage and document ranking experiments

Reproduced on ubuntu 20.04 using WSL2 on Windows 10, no issues

Codecov Report

Merging #1737 (0d6c388) into master (a62aca0) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master    #1737   +/-   ##
=========================================
  Coverage     57.58%   57.58%
  Complexity     1032     1032
=========================================
  Files           173      173
  Lines          9781     9781
  Branches       1344     1344
=========================================
  Hits           5632     5632
  Misses         3675     3675
  Partials        474      474

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a62aca0...0d6c388. Read the comment docs.


IO Exception when uploading big files with Drive resumable upload

The following error happens after several GB's of uploading the file we're trying to backup in Drive. Happens at different stages. Already did the Steam.php fix mentioned and the MediaFileUpload.php fix (this one made the error prompt even faster).
[09-Jun-2015 11:02:41] PHP Fatal error:  Uncaught exception 'Google_IO_Exception' with message 'HTTP Error: Unable to connect: 'fopen(https://www.googleapis.com/upload/drive/v2/files?uploadType=resumable&upload_id=AEnB2Urc2_53dFy5ypIcncNv6OZWSIeEz3x3ZdaaOupBfBLyB4K4gRw7TAeoCs-Zov2Yjlj8Ws6pUqobagjeC4y_bUJ06Xf-kA): failed to open stream: operation failed'' in /root/scriptEnCrontab/libs/src/Google/IO/Stream.php:141
Stack trace:
#0 /root/scriptEnCrontab/libs/src/Google/IO/Abstract.php(136): Google_IO_Stream->executeRequest(Object(Google_Http_Request))
#1 /root/scriptEnCrontab/libs/src/Google/Http/MediaFileUpload.php(164): Google_IO_Abstract->makeRequest(Object(Google_Http_Request))
#2 /root/scriptEnCrontab/respaldoDrive.php(169): Google_Http_MediaFileUpload->nextChunk('?M??d????????...')
#3 /root/scriptEnCrontab/respaldoDrive.php(85): insertFile(Object(Google_Client), Object(Google_Service_Drive), '/root/ITbk/www_...', 'bck21-html_4.tg...', '0B2Xiur2QhdC6fl...')
#4 {main}
thrown in /root/scriptEnCrontab/libs/src/Google/IO/Stream.php on line 141

If its unable to connect, could you try catching and retrying? There isn't any automatic retry in the library unfortunately.

Oh, i thought that that's why the status flag was for, let me try it. I'll come back with what happened.

This is no longer valid as v1-master is no longer maintained.


Automated Package Upgrade

:tada: This PR is included in version 12.0.39 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Optimize Known Methods and Known Versions

Optimize TakeStartLine function. TakeStartLine is now 2 times faster then the previous one

Hi @Anderman, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by .NET Foundation and real humans are currently evaluating your PR.
TTYL, DNFBOT;

How did you measure that it's 2 times faster?

I did the measurement  on the function TakeStartLine with an input string GET / HTTP/1.1\r\n. on Core CLR X64 Release version. At the moment it is 1.3 times faster.
The previous test I didn't realize that the different versions has an big impact on the relative test
For the test I created an version with both versions of TakeStartLine (Org and new). The test looked something like
public static void perf_TakeStartLine()
        {
            var socketInput = new SocketInput(new MemoryPool2(), null);
            var firstLine = Encoding.ASCII.GetBytes("GET / HTTP/1.1\r\n");

            var inputBuffer = socketInput.IncomingStart(128);
            Buffer.BlockCopy(firstLine, 0, inputBuffer.Data.Array, inputBuffer.Data.Offset, firstLine.Length);
            socketInput.IncomingComplete(firstLine.Length, null);

            var maxRequest = 10000000;
            var connectionContext = new ConnectionContext()
            {
                SocketInput = socketInput,
                ServerAddress = new ServerAddress(),
                DateHeaderValueManager = new DateHeaderValueManager(),
            };
            var frame = new Frame<object>(application: null, context: connectionContext);
            var sw = new Stopwatch();
            sw.Start();
            for (int i = 0; i < maxRequest; i++)
            {
                var success = frame.TakeStartLine_org(socketInput);
            }
            sw.Stop();

            var sw2 = new Stopwatch();
            sw2.Start();
            for (int i = 0; i < maxRequest; i++)
            {
                var success = frame.TakeStartLine(socketInput);
            }
            sw2.Stop();
            var org = (int)(((maxRequest)) / (sw.Elapsed.TotalMilliseconds));
            var improved = (int)(((maxRequest)) / (sw2.Elapsed.TotalMilliseconds));
            var factor = (1.0 * improved) / org;
            Console.WriteLine($"org:{org:D4} new:{improved:D4} req/ms {factor.ToString("0.0")}x ");
        }

I did some tests of my own with your change and it went from 1,000,000 requests @ ~1200ms to ~900ms.

Thx for the comment. The aspnet team has the best repro's. Corefx has not that quality and coreclr is no fun at all. but maybe that's because my lack of knowlegde of c++. (I did a JIT change for an unsafe vector copy (-; and tried an intrinsic version of buffer.blockcopy )

A few more comments. You'll need to rebase since there are conflicts with the dev branch.

Hi @Anderman, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by .NET Foundation and real humans are currently evaluating your PR.
TTYL, DNFBOT;

Reopened.


add subscriptions create method

Given the changes to core, we have to implement the new subscriptions instead of relying on the old webhooks.
I plan to add tests later, and I just want to be sure that I am following the guidelines of the project.
Please let me know any recommendations or changes you would like me to implement in this PR.

Hey Miguel,
Thanks for the input, see the changes here: https://github.com/synapsepay/SynapsePayRest-Ruby/tree/subscription_updates
Thanks!


Fix: Tween always return true for _get(), _set()

Fix: #38473

Thanks!

Cherry-picked for 3.2.2.


getting error with Dapr container and pod is crashing in AKS

here is the error from POD logs.
info: Microsoft.DurableTask[3]
The sidecar at address localhost:50001 is unavailable. Will continue retrying.
info: Microsoft.DurableTask[3]
The sidecar at address localhost:50001 is unavailable. Will continue retrying.
info: Microsoft.DurableTask[3]
The sidecar at address localhost:50001 is unavailable. Will continue retrying.
info: Microsoft.DurableTask[3]
The sidecar at address localhost:50001 is unavailable. Will continue retrying.
info: Microsoft.DurableTask[3]
Note: If you have a general support question and are looking for a quicker response, please checkout our discord channel for answers from the community:
https://aka.ms/dapr-discord
In what area(s)?

/area runtime

/area operator

/area placement

/area docs

/area test-and-release

Ask your question here

AKS and Dapr 1.13.2


fix autobahn test app to be an app

also remove net46 version of autobahn conformance test, the autobahn
doesn't need to be run twice...

I'm going to add code to appveyor and travis to install autobahn so they can run those tests


Merge Vue CLI updates & upgrade to OpenLayers v6.15.7

What is included:

First I have merged all Vue CLI PRs sequentially and resolved conflicts (#298, #297, #312, #314), great work @sronveaux! I hope this helps to save time in getting these great additions merged.
Then I solved the remaining linting errors and upgraded to OpenLayers 6.7.0. I did some basic testing and couldn't find errors, so this could close #261.
There are no breaking changes I can see in OpenLayers until v7, so I upgraded to v6.15.1, the last version prior.
Upgrading to v7 would require some refactoring for removed methods: https://github.com/openlayers/openlayers/blob/main/changelog/upgrade-notes.md#700
After merging this PR, implementing #290 would be possible.

What I missed:

I couldn't get the Karma ChromeHeadless tests to work, as I run my coding environment inside a docker container.
I get (expected) errors related to system dbus and WebGL:

Failed to connect to the bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory
Passthrough is not supported, GL is disabled

Hi @spwoodcock,
great to see you already got going and thanks a lot for providing this so quickly! I`m currently a bit limited on time but I hope to have some time for testing here in the upcoming weeks.
Just for clarification: The vue-cli-5 upgrade has to go into Wegue v2, as it comes with new major versions of several dependencies like Webpack etc. So by incorporating the vue-cli upgrade PR, it has to go into v2 too obviously.

Ah my bad! I thought it made sense to build this on the Vue CLI upgrades, as #261 was having errors due to outdated dependencies.
I don't think there is another solution due to conflicts with the geotiff library otherwise.
I hope this PR was helpful and will simplify the review for the combined Vue CLI changes.

Hi @spwoodcock. I finally found some time to test the OL v6.15.7 migration in Wegue and in a custom application. Everything works out of the box as intended. Looks like the OL guys did a clean job in semantic versioning, so I couldn`t find any breaking changes either. And a big thanks again to you for giving us a merged PR, so we can fiddle around with the upcoming changes much easier!
Regarding your comments above:

The OL 7 migration seems to be a more complex job. For the very least I know of the tooltip functionality, which makes use of Map::forEachLayerAtPixel, which will eventually break and be cumbersome to work around. If you are interested to work on that (as part of a seperate PR) it`s obviously very much appreciated. :)
I have no experience in running the IDE inside a docker container and accessing ChromeHeadless tests from there. Did you install it properly in the respective image? Here`s some info on StackOverflow, not sure if it could be helpful?!

@chrismayer: I have to point out that I only use a small subset of OL functionality. It would be desirable if you could verify it in one of your applications too, so we have a little broader coverage.
One final comment:
For the sake of a clean commit history, I propose that we rebase this one on master once the time has arrived - or just redo it by adjusting the OL version if simpler. Anyway, very nice to have this PR around for testing!
Cheers,
Felix

Hi @fschmenger, thanks for getting back to me.
Glad the PR was useful for testing. Like you say, feel free to rebase and merge, or close and re-create, as you prefer.
I'm made 4 commits, two related to fixing lint errors (preventing the app starting), two for a progressive ol upgrade.

Hi @spwoodcock,
I missed running the karma unit tests yesterday. Currently they don`t work for me either - however they do work in the underlying #314.
Uncaught ReferenceError: require is not defined
  at webpack://wegue/./node_modules/ol/Map.js?:4:30

  ReferenceError: require is not defined
      at eval (webpack://wegue/./node_modules/ol/Map.js?:4:30)
      at Module.<anonymous> (index.2854134847.js:755:1)
      at __webpack_require__ (index.2854134847.js:5023:42)
      at eval (webpack://wegue/./src/components/ol/Map.vue?./node_modules/babel-loader/lib/index.js??clonedRuleSet-40.use%5B0%5D!./node_modules/@vue/cli-service/node_modules/@vue/vue-loader-v15/lib/index.js??vue-loader-options:10:35)
      at Object.<anonymous> (index.2854134847.js:748:1)
      at __webpack_require__ (index.2854134847.js:5023:42)
      at eval (webpack://wegue/./src/components/ol/Map.vue?:6:246)
      at Module.<anonymous> (index.2854134847.js:741:1)
      at __webpack_require__ (index.2854134847.js:5023:42)
      at eval (webpack://wegue/./src/components/ol/Map.vue?:7:91)

I assume there is some problem with OL using commonJS imports. However I`m not sure why this hits us in the unit tests and not on production build, and why it should be a problem with OL 6.15.7 but not with OL 6.4.3.
@sronveaux Do you have any idea here? I tried to set transpileDependencies like I proposed in #314, that got rid of a bunch of warnings but at the end runs into the same problem.

Another way to debug this kind of laborious issue is to find the exact version of OL where the problem starts to occur, see what's new from the previous version and add the potential new dependency in the tests/unit/index.js as explained above... very frustrating to do...

@chrismayer: I have to point out that I only use a small subset of OL functionality. It would be desirable if you could verify it in one of your applications too, so we have a little broader coverage.

I just tested this PR with our latest customer app, which uses some more OL functionality besides the common Wegue stuff. It really looks amazing and works quite well. I tested dev mode and production build without any problems. And upgrading was quite smooth, only npm install, making eslint happy due to some new rules and this thing is ready to fly :rocket: .
Great work everybody :muscle:

Based on the discussions around fixing the unit tests, what are the two constants that require updating here?
Feel free to update my branch, or I can update in a few days when I'm back.

Hi @spwoodcock, I couldn`t push the fixes for the unit tests to your branch,
so I temporarily parked them in #331 2a451d9a960d458c52be51cab34821951ab32417.
Could you just pull it from there?

@fschmenger done 👍

I updated this to include the additional commits to #314 and the latest commit on master as of 26/04/2023 (c549872209600eb380fc46cf0bdec9ed53bb8ecf).

I will close this PR in favour of #333.

This works quite well. Just tested again.
The conflict is just a removed trailing whitespace in  src/components/attributeTable/AttributeTableWin.vue, so is easily fixable.
I am going to approve now.

Thanks for the PR @chrismayer 👍

Just in time for FOSS4G 😉

Just in time for FOSS4G wink

Total coincidence, but certainly not bad at all :wink:


Execution of Quickstart Tutorial fails because of missing 2.root.json

Description of issue or feature request:
Execution of the "Quickstart Tutorial" fails (with Tag v0.12.2 and current develop HEAD c63b7f3). It reports that the file 2.root.json cannot be found.
Current behavior:
Execution of the "Quickstart Tutorial" fails (with Tag v0.12.2 and current develop HEAD c63b7f3) with following message:
Could not download URL: 'http://localhost:8001/metadata/2.root.json'
Traceback (most recent call last):
  File "/path/to/tuf/download.py", line 264, in _download_file
    response.raise_for_status()
  File "/usr/lib/python3/dist-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: File not found for url: http://localhost:8001/metadata/2.root.json
Update failed from http://localhost:8001/metadata/2.root.json.
Traceback (most recent call last):
  File "/path/to/tuf/client/updater.py", line 1507, in _get_metadata_file
    upperbound_filelength)
  File "/path/to/tuf/download.py", line 150, in unsafe_download
    return _download_file(url, required_length, STRICT_REQUIRED_LENGTH=False)
  File "/path/to/tuf/download.py", line 264, in _download_file
    response.raise_for_status()
  File "/usr/lib/python3/dist-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: File not found for url: http://localhost:8001/metadata/2.root.json
Failed to update '2.root.json' from all mirrors: {'http://localhost:8001/metadata/2.root.json': HTTPError('404 Client Error: File not found for url: http://localhost:8001/metadata/2.root.json')}
ERROR:root:HTTP error for root version 2
Traceback (most recent call last):
  File "/path/to/tuf/client/updater.py", line 1150, in _update_root_metadata
    version=next_version)
  File "/path/to/tuf/client/updater.py", line 1786, in _update_metadata
    upperbound_filelength, version)
  File "/path/to/tuf/client/updater.py", line 1602, in _get_metadata_file
    raise tuf.exceptions.NoWorkingMirrorError(file_mirror_errors)
tuf.exceptions.NoWorkingMirrorError: No working mirror was found:
  'localhost:8001': HTTPError('404 Client Error: File not found for url: http://localhost:8001/metadata/2.root.json')
INFO:tuf.download:Downloading: 'http://localhost:8001/metadata/timestamp.json'
INFO:tuf.download:Downloaded 557 bytes out of an upper limit of 16384 bytes.
INFO:tuf.download:Downloading: 'http://localhost:8001/metadata/snapshot.json'
INFO:tuf.download:Downloaded 472 bytes out of an upper limit of 2000000 bytes.
INFO:tuf.download:Downloading: 'http://localhost:8001/metadata/targets.json'
INFO:tuf.download:Downloaded 735 bytes out of an upper limit of 5000000 bytes.
INFO:tuf.client.updater:'targets.json' up-to-date.
INFO:tuf.client.updater:'targets.json' up-to-date.
INFO:tuf.download:Downloading: 'http://localhost:8001/targets/testfile'
INFO:tuf.download:Downloaded 10 bytes out of the expected 10 bytes.
INFO:tuf.client.updater:The file's sha256 hash is correct: 114811b0b8998cb9853a5379598021410feddf69bb2ee7b7145d052a7e9b5d45
INFO:tuf.client.updater:The file's sha512 hash is correct: 9484ec641bd36078436a993000045bbc380f77d8897e9dcc1b784c10642abe31f5469b9b6c31afa3a286abb4d0ce618468043883bce231c99aa984ea9ff14a64

Expected behavior:
The Quickstart Tutorial works without errors.

Thanks for the report @Silvanoc. This is a known issue (see #983) where the logging in TUF is reporting errors despite things proceeding as expected.
You can see from the last four lines of INFO output that the file was downloaded and verified as matching the hashes in the targets metadata:
INFO:tuf.download:Downloading: 'http://localhost:8001/targets/testfile'
INFO:tuf.download:Downloaded 10 bytes out of the expected 10 bytes.
INFO:tuf.client.updater:The file's sha256 hash is correct: 114811b0b8998cb9853a5379598021410feddf69bb2ee7b7145d052a7e9b5d45
INFO:tuf.client.updater:The file's sha512 hash is correct: 9484ec641bd36078436a993000045bbc380f77d8897e9dcc1b784c10642abe31f5469b9b6c31afa3a286abb4d0ce618468043883bce231c99aa984ea9ff14a64

Ups, sorry for the duplicate. Closing it.


fix: GIT_TAG not parsed inside CPMFindPackage

When using CPMFindPackage, if no VERSION is given then GIT_TAG is used as fallback if defined, which is a good idea.
The issue is that GIT_TAG is missing inside the oneValueArgs variable therefore GIT_TAG is not parsed. This is resolved by this little change.

Good catch, thanks for the fix!

The fix is live in v0.27.5! 🎉


Revert "Fix phone number regex"

Reverts IEEEKeralaSection/rescuekerala#657
Merging that PR seemed to introduce some changes to the admin CSV page that we can't locate:
diff --git a/mainapp/admin.py b/mainapp/admin.py
index 4decaad..1bbf055 100644
--- a/mainapp/admin.py
+++ b/mainapp/admin.py
@@ -88,14 +88,13 @@ class NGOAdmin(admin.ModelAdmin):

     def download_csv(self, request, queryset):
         header_row = [f.name for f in NGO._meta.get_fields()]
-        body_rows = []
-        for ngo in NGO.objects.all():
-            row = [
-                getattr(ngo, key) if key != 'district' else ngo.get_district_display()
-                for key in header_row
-            ]
-            body_rows.append(row)
-
+        body_rows = queryset.values_list()
+        # for ngo in NGO.objects.all():
+        #     row = [
+        #         getattr(ngo, key) if key != 'district' else ngo.get_district_display()
+        #         for key in header_row
+        #     ]
+        #     body_rows.append(row)
         response = create_csv_response('NGOs', header_row, body_rows)
         return response

@@ -128,7 +127,7 @@ class RescueCampAdmin(admin.ModelAdmin):
     list_filter = ('district','status')

     def download_csv(self, request, queryset):
-        header_row = ('district', 'name', 'location', 'taluk' , 'village' ,  'status', 'contacts', 'facilities_available', 'total_people',
+        header_row = ('district', 'name', 'location', 'status', 'contacts', 'facilities_available', 'total_people',
                       'total_males', 'total_females', 'total_infants', 'food_req',
                       'clothing_req', 'sanitary_req', 'medical_req', 'other_req')
         body_rows = []

going to revert and reapply the patch
cc @cibingeorge  @Kavit900

update: we went with force-pushing the last known state to master before the history got messed up. @Kavit900  will recreate the patch on top of it.


Enable new shortcut creation in context menu

What feature or improvement do you think would benefit Files?

The Add New Shortcut option is missing from context menu. Rt click >New > Shortcut

We can currently create file shortcuts by clicking on file or folder but not for web addresses (URL).

Also for some reason we can't click the context menu > New > Folder option when the context menu touches the top edge of screen

Requirements

The proposal to add the option to create new shortcuts from context menu, both for files and URLs.
To enable clicking on menu items when context menu reaches top edge of screen

Files Version
Version: <IP_ADDRESS> OS Version: 10.0.19044.2075
Windows Version
Edition Windows 10 Home Single Language Version 21H2 Installed on 31-03-2021 OS build 19044.2075 Experience Windows Feature Experience Pack 120.2212.4180.0
Comments
No response

Thank you for your feedback, merging with #1671


Drawer does not overlap header

I have a question, I can not find a solution.
I have to create a layout where the drawer and header are fixed and visible on the desktop. The drawer also can not overlap the header (figure 1). In Mobile the drawer starts invisible.

On the desktop, the drawer will never be invisible. Instead I would like it to be reduced only. (Figure 2)

The drawer will only be invisible on the mobile.

This kind of thing is not supported in MDL. Nor will it be supported here. Material Components for Web should eventually allow for this type of thing to happen.
Since this isn't something we'll be allowing for within this project, it falls as a support question. Which are best left to StackOverflow. But really, you're looking at quite a bit of manual code/overriding to handle this. You'd probably be better off if you need to stick with MDL writing your own application bar and drawer implementations.
Thank you.

With MDL I can not make a drawing screen and a drawing in those that in the Material Design specification?
Navigation drawer clipped under the app bar: Apps focused on productivity that require balance across the screen


Update bug fix

fix a bug ,when updates with member, that have primary key and other Struct member

does test pass for you?

Hi, I'm wondering can you help me with this? https://github.com/jinzhu/gorm/issues/1659


Generic type bounds of IsIterableContaining are inconsistent with those of IsMapContaining

Hello all! Thanks a lot for the new Hamcrest 2.1 library, it's a pleasure to work with and I can also use it easily in my Java 9 modules.
The IsIterableContaining matcher extends TypeSafeDiagnosingMatcher<Iterable<? super T>>. I understand that the goal is to, for instance, match a Set<Number> when it has an Integer.
Why then does IsMapContaining extend TypeSafeMatcher<Map<? extends K, ? extends V>>? Wouldn't it be just as reasonable to match a Map<Object, Object> when it has a String key? Or a Long value?

Writing library code with Java generics is a bit like (╯ರ ~ ರ）╯︵ ┻━┻
Perhaps @sf105 might have some insights as to how the current implementation came to be the way it is?

Collection<Banana> bananas = singleton(banana);
Collection<Fruit>  fruits  = singleton(banana);

assertThat(fruits,  hasItem(banana));
assertThat(bananas, hasItem(fruit) );  // this one is currently invalid

I think this makes sense: Why would I want to check if a collection of some specific type contains some less specific type? Usually I already know the actual runtime type of the value I use to create the matcher with, because I create it myself, whereas the collection being examined is passed in and could be anything.
The same is true for maps. Therefore I would propose to make the hasKey() and hasValue() matchers contravariant.

The IsIterableContaining matcher extends TypeSafeDiagnosingMatcher<Iterable<? super T>>.

The IsIterableContaining  have to extend TypeSafeDiagnosingMatcher<Iterable<? extends T>>.
Usualy you have a collection of entities (e.g. Car). This list can contain any object that extends Car (List<? extent Car>). And you want to apply to each element the Matcher that can check the Car or it's parent (Matcher<? super Car>).
So, you want to construct Matcher that take the Iterable<? extends Car> and apply to it given Matcher<? super Car>

I've been reading up on poloymorphic collections in Java, and the PECS rule (producer extends, consumer super). I've created #422 as an attempt to address this issue. There's a new test in IsIterableContainingTest that demonstrates the problem and the fix.
I'm a bit worried that this change might impact a lot of existing code. I'd appreciate if folks could check out that fork and give it a test


fixed prefix mask

python evaluation/run_eval.py \
    --dataset_name super_glue \
    --dataset_config_name copa \
    --template_name "best_option" \
    --model_name_or_path ../bloom-ckpt/tr13f-6b3-ml-t0-lmtoks168b-t0toks13b-prefix \
    --output_dir ./prefix/copa-best-option-prefix \
    --prefixlm

should result in {'accuracy': 0.63} as oppose to previous 0.53

Which transformers version are you using?
I am using this one, which incorporates the changes from this PR + allows for prefixlm.
My masks are -inf where the model should not pay attention and 0 elsewhere. Note padding is on the left for inputs & right for targets.
Bidirectional input
model_inputs["causal_mask"][0, :, :10, :10]
tensor([[[-inf, -inf, -inf, -inf, -inf, -inf, 0., 0., 0., 0.],
         [-inf, -inf, -inf, -inf, -inf, -inf, 0., 0., 0., 0.],
         [-inf, -inf, -inf, -inf, -inf, -inf, 0., 0., 0., 0.],
         [-inf, -inf, -inf, -inf, -inf, -inf, 0., 0., 0., 0.],
         [-inf, -inf, -inf, -inf, -inf, -inf, 0., 0., 0., 0.],
         [-inf, -inf, -inf, -inf, -inf, -inf, 0., 0., 0., 0.],
         [-inf, -inf, -inf, -inf, -inf, -inf, 0., 0., 0., 0.],
         [-inf, -inf, -inf, -inf, -inf, -inf, 0., 0., 0., 0.],
         [-inf, -inf, -inf, -inf, -inf, -inf, 0., 0., 0., 0.],
         [-inf, -inf, -inf, -inf, -inf, -inf, 0., 0., 0., 0.]]])

Causal target
model_inputs["causal_mask"][0, :, -10:, -10:]
tensor([[[0., -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf],
         [0., 0., -inf, -inf, -inf, -inf, -inf, -inf, -inf, -inf],
         [0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf, -inf],
         [0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf],
         [0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf],
         [0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf],
         [0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf],
         [0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf],
         [0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf],
         [0., 0., 0., 0., -inf, -inf, -inf, -inf, -inf, -inf]]])

This is then added to the attention scores here thus masking out the values we do not want to attend to. The transformation here does not change the causal mask as it is already applied.
Your masks are first True where it should attend and False where not. After the combination with the attention mask here assuming you use that version, your masks look like the below I think:
out_mask_l[0, :, :10, :10]
tensor([[[-3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38,
          -3.4028e+38,  1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00],
         [-3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38,
          -3.4028e+38,  1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00],
         [-3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38,
          -3.4028e+38,  1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00],
         [-3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38,
          -3.4028e+38,  1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00],
         [-3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38,
          -3.4028e+38,  1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00],
         [-3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38,
          -3.4028e+38,  1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00],
         [-3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38,
          -3.4028e+38,  1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00],
         [-3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38,
          -3.4028e+38,  1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00],
         [-3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38,
          -3.4028e+38,  1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00],
         [-3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38,
          -3.4028e+38,  1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00]]])

out_mask_l[0, :, -10:, -10:]
tensor([[[ 1.0000e+00,  0.0000e+00,  0.0000e+00,  0.0000e+00, -3.4028e+38,
          -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38],
         [ 1.0000e+00,  1.0000e+00,  0.0000e+00,  0.0000e+00, -3.4028e+38,
          -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38],
         [ 1.0000e+00,  1.0000e+00,  1.0000e+00,  0.0000e+00, -3.4028e+38,
          -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38],
         [ 1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00, -3.4028e+38,
          -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38],
         [ 1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00, -3.4028e+38,
          -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38],
         [ 1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00, -3.4028e+38,
          -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38],
         [ 1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00, -3.4028e+38,
          -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38],
         [ 1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00, -3.4028e+38,
          -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38],
         [ 1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00, -3.4028e+38,
          -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38],
         [ 1.0000e+00,  1.0000e+00,  1.0000e+00,  1.0000e+00, -3.4028e+38,
          -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38]]])

After the addition the -3.4 masks out the padding, but there are some arbitrary 1s added I think?
I'm probably missing sth here, do you know where I am wrong?

I was basing the implementation from Hailey's PR. I didn't know there was another PR. Looking again at your method, it should be correct.

Yes, I forked transformers instead of Hailey's PR because I wanted the changes the PR I mentioned.
Looking forward to your thoughts!