helm chart upgrade

This PR makes the following changes:

Fixes #1232
Upgrade Prometheus version to 2.1.0
Upgrade Grafana version to 4.6.3

Tested on SPC, successfully deployed Postgres using v 0.5.2 image

@muratkars Can you try to resend the PR with git commit -s command.
This will set your signature in your commit message.


Make question waiting for data persist accross views

http://manage.dimagi.com/default.asp?185734

@dannyroberts I'm using your pending intent widget infrastructure on all questions that use any sort of callout

@phillipm Left a couple of very minor, optionally actionable comments. Other than that, can you just update the class-level comment for PendingCalloutInterface now that its usage is more broad than what Danny had there? Good to merge after that.

@amstone326 thanks, updated the PR based on your feedback


Bar hit detection in simulation

Detect bar contact based on joint velocity
Adds a node detect_bar_contact and a launchfile to run RViz, Gazebo, detect_bar_contact, and plotjuggler:
roslaunch marimbabot_simulation marimbabot_contact_debug.launch

The node detect_bar_contact detects which bars (notes) were hit based on joint_states and publishes a note string (e.g. b5, dis5) to topic sim_notes. The node play_notes then plays the detected notes using a crude (for now) approach with real WAV files recorded in a very noisy room.
A MIDI file is generated and saved to marimbabot_simulation/midi when the nodes are stopped.
Closes #163
Closes #41
Closes #104

I cannot see the launch file you mention
"marimbabot_contact_debug.launch"
@iibrahimli

PR description is outdated, should be marimbabot_simulation/launch/marimbabot_full_sim.launch as shown in the Readme

@iibrahimli
I had to add
sudo apt-get install libportaudio2  (additionally to the python package)
and
chomp_motion_planner
in the package.xml
and install ros-noetic-moveit-planners-chomp
Please consider adding them :)

@iibrahimli I had to add sudo apt-get install libportaudio2 (additionally to the python package) and chomp_motion_planner in the package.xml and install ros-noetic-moveit-planners-chomp
Please consider adding them :)

Good catch! thanks, will add now.

overall LGTM but i am not sure if it makes sense to keep all the audio files in the repo. There are so many wav files.

Agreed. Now that you mention it, I will remove the notes for now, we generate a MIDI file and can play that anyways.

Regarding chomp stuff, no motion planning is done in the simulation package. I'd guess that is a dependency from another package.


Add in and length for inclusion and iteration

Welcome to Codecov :tada:
Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.
Thanks for integrating Codecov - We've got you covered :open_umbrella:


Fix: Remove old flow run infra override access checks

https://github.com/PrefectHQ/prefect/pull/12742 removed experimental flags for flow run infra overrides.  However the UI still contains access checks meaning that users in OSS can not access or view the flow run infra overrides section of the custom run form (and elsewhere).
This PR removes the access checks. Accompanying PR in https://github.com/PrefectHQ/prefect/pull/13401

@zhen0 Yes, this looks right, we should have removed this alongside the other PR.

I think we need to handle/remove the deploymentScheduleFlowRunInfraOverrides flag also.
I split what-was-originally 1 flag for flow-run-infra-overrides into 2 as the server-side only had partial support so we could roll out some of the functionality but hold on the scheduling updates (which needed server-side support).
I'm guessing the current state now though is that we want both on everywhere - in which case, we should update  usages of deploymentScheduleFlowRunInfraOverrides to default to "on" behavior.


Clean version of wnordmann/sampleapp

Complete rebase of the wnordmann/sampleapp
which should have a clean merge atop ultimateredux.
(List below is wnordmann's work...)

Adds Feature table example.

Added util function to reproject geojson from one projection
to another.

Tweaks to Sass/CSS handling.

Coverage increased (+0.1%) to 92.54% when pulling c3b03e253de00f91552bec737e0df11a3cffe545 on theduckylittle:wnordmann-sampleapp-clean into 8e3d62db991732cd665b8fe296271fabf15f2459 on boundlessgeo:ultimateredux.

Coverage increased (+2.01%) to 94.419% when pulling 7394f8ca7201ece271096def0bb97624ad68dbae on theduckylittle:wnordmann-sampleapp-clean into 8e3d62db991732cd665b8fe296271fabf15f2459 on boundlessgeo:ultimateredux.


cannelloni: init at 1.0.0

Description of changes
Add cannelloni, a linux SocketCAN over ethernet tunnel.
Things done

Built on platform(s)

[x] x86_64-linux
[X] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[X] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[X] Fits CONTRIBUTING.md.

Hello,
What's the status of this PR ?

Addressed comments and updated to 1.1.0.
@ocfox @drupol should now be ready to merge.


RunnerDeployment Repository Var Bug

Describe the bug
The regex check for the repoisitoy variable in Kind: RunnerDeployment, checks for '^[^/]+/[^/]+$' which requires a '/' between 2 strings. For my case I am using a repository inside of the my company organization, so I am also defining the organization var. The issue is the repo var should take only the name of the repo (when inside an org), with no '/' denoting who owns it. Then because the org var is also defined I would think the helm chart would concat those 2 vars together with the '/', but I get the following error:
│ Error: RunnerDeployment.actions.summerwind.dev "example" is invalid: spec.template.spec.repository: Invalid value: "": spec.template.spec.repository in body should match '^[^/]+/[^/]+$'
So I also tried to define org var as the org value, but then put in the repo var as org/repo but then get the following error:
| Error: cannot patch "example" with kind RunnerDeployment: admission webhook "validate.runnerdeployment.actions.summerwind.dev" denied the request: RunnerDeployment.actions.summerwind.dev "example" is invalid: spec.template.spec.repository: Invalid value: "/": Spec cannot have many fields defined enterprise, organization, and repository.
So to finish it off, I concluded by trying to set the repo to / again, but without defining the organization var this time. Finally it deployed in terraform.
Note I have yet to check and see if the runner is working, so I wouldn't consider this a viable workaround.
Checks
Versionz: actions.summerwind.dev/v1alpha1

[x] My actions-runner-controller version (v0.x.y) does support the feature
[ ] I'm using an unreleased version of the controller I built from HEAD of the default branch

To Reproduce
Steps to reproduce the behavior:

See description above

Expected behavior
I expect a conditional to exist within the code that says when organization is defined, rgex excludes the need for the '/', and when organization is NOT defined, then to keep the regex that is currently being used.
Environment (please complete the following information):

Kubernetes Version: 1.18.17
Deployment Method: Terraform Helm Release
Helm Chart Version: 0.0.1

Additional context
I also tried to deploy these helm charts locally outside of terraform and got the same errors. So I feel very confident the issue is unrelated to deploying with terraform.

I expect a conditional to exist within the code that says when organization is defined, rgex excludes the need for the '/', and when organization is NOT defined,

Nope. organization is for defining organizational runners and repository is for repository runners`. If it worked like you expected, there will be no way to distinguish the two :)
All the explanations and logs you've shared seem to explain this. So to me, it looks like you're just explaining how the runner deployments are supposed to work (and it seems to be working as expected to me).
Also, we don't have a chart that creates RunnerDeployment. So it should have nothing to do with Helm.

I expect a conditional to exist within the code that says when organization is defined, rgex excludes the need for the '/', and when organization is NOT defined,

Nope. organization is for defining organizational runners and repository is for repository runners`. If it worked like you expected, there will be no way to distinguish the two :)
All the explanations and logs you've shared seem to explain this. So to me, it looks like you're just explaining how the runner deployments are supposed to work (and it seems to be working as expected to me).
Also, we don't have a chart that creates RunnerDeployment. So it should have nothing to do with Helm.

Gotcha, thank you for the explanation, that makes more since.


README: Add show-results + get-test example

I lost more time than I care to admit trying to find a built-in way to do this, so documenting it seems beneficial.

Nice bash-foo. :)


Allow before_child_exit block to be optional

Rather than re-implement hook registration, lean on the existing methods
in Resque.
For example, here is how the after_fork hook is registered: https://github.com/resque/resque/blob/85706d536287910c53e0eb14e98c612b03dbea20/lib/resque.rb#L259
This fixes a bug where resque will error if a before_child_exit hook
has not been registered (https://github.com/stulentsev/resque-multi-job-forks/issues/19).

Released 0.5.1


Add question to CLI that asks which storage onramp people would like to use (storacha, lighthouse, or akave)

@xBalbinus I think we can close this now


M3-1128 Linode IP Refresh Fixes

A recent refactor that turned state into props caused multiple regressions related to the Linode > Networking > IP Addres Tables not refreshing upon modification.
This addresses those issues.

Coverage decreased (-0.01%) to 26.355% when pulling e5ed60604b179cdbf48adf6b95be7afc5744032b on asauber:M3-1128 into 9e0213a44ca45acd7d992a61cb467ffa1c3df4fb on linode:develop.


Your pre-trained model on lossy JPEG images

Thanks for sharing your code and approach.
But testing your approach using everyday images is a hard task since the images are mostly JPEGs and have noisy artifact blocks and in 99% of cases we don't have access to the RAW noiseless version of the images.
Can you provide a pre-trained model on noisy images? I'm curious to see if the SRGAN-based methods only do nicely on the noiseless raw images or if they can do both DeNoising + SuperResolution too?
Thanks again.

Hi, @Neltherion
Yes, the problem you mentioned is very common in the real-world scenarios.
A very simple way is to train (or fine-tune) the models to noisy images. But I may not have time to prepare the data and train these models in the next few months.
You may have a try using the codes :smiley:


In the paged attention mode, whether the kcache space must be malloced in a continuous space?

I'm confused by the gmem malloced for the paged attention.
In paged attention mode, two K cache_block for the same query,  do the gmem need to be malloced in continous space?
like nheads=1, headdim=128,page_block_size=64, let's say the address for one query's first cache-block is __half* p_block0,
and the address for the second block is __half* p_block1.
does p_block1 = p_block0 + page_block_size * headdim * nheads?

Not sure I understand the question but the function docstring should tell you the shapes of the tensors and whether they need to be contiguous:
        k_cache: (batch_size_cache, seqlen_cache, nheads_k, headdim) if there's no block_table,
            or (num_blocks, page_block_size, nheads_k, headdim) if there's a block_table (i.e. paged KV cache)
            page_block_size must be a multiple of 256.
        v_cache: (batch_size_cache, seqlen_cache, nheads_k, headdim) if there's no block_table,
            or (num_blocks, page_block_size, nheads_k, headdim) if there's a block_table (i.e. paged KV cache)
        block_table [optional]: (batch_size, max_num_blocks_per_seq), dtype torch.int32.


Combine cases crashes when case generates simulation error

Describe the bug
For the Generic System model with Calculate generation profiles and nameplate capacity from open cases on the Power Plant page, when one of the cases being combined generates an SSC simulation error, clicking Select cases causes SAM to crash.
To Reproduce
Steps to reproduce the behavior:

Download and open the the "st helena.sam" file attachment to https://sam.nrel.gov/forum/forum-general/4066-combine-cases-shuts-down-sam.html.
Go to the "New WT" case and run a simulation to see the SSC error "exec fail(windpower): failed to read local weather file".
Go to the "All Wind" case.
On the Power Plant page, click Select cases.
SAM crashes.

Update: SAM crashes for this file even when there is no SSC simulation error.
Expected behavior
Clicking Select cases should not cause SAM to crash.
If one or more of the cases being combined generates an error, clicking Select Cases should inform the user there is a problem with one of the cases, perhaps in a message box, like "Simulation for case [case name] failed with error: [SSC error]".

The issue with this file is that the 2 wind cases were built with LCOE calculator financial model, which forgoes the assignment of system_use_lifetime_output. The combine cases function doesn't know what to do when this variable is not assigned. What is the intended behavior @Matthew-Boyd @cpaulgilman ?

@mjprilliman Thanks for investigating. Would it work to assume one year of gen data if system_use_lifetime_output is not assigned? Or, perhaps it should be assigned in the LCOE calculator model. The intended behavior is that it should be possible to combine cases that use the LCOE calculator financial model.


Git diff utilizes older Razor editor resulting in C# background color showing through

Ultimately this issue embodies a bigger ask to allow "misc" Razor files in the new Razor editor.
Picture from VS issue:

VS Issue where this was originally reported: https://developercommunity.visualstudio.com/t/Razor-editor-not-respecting-code-backgro/1574754

More reports:

https://developercommunity.visualstudio.com/t/Razor-syntax-highlighting-does-not-work-/1576862

@NTaylorMullen judging by your first line you've already imagined a fix here, can you share?

@NTaylorMullen judging by your first line you've already imagined a fix here, can you share?

So it'd be a bigger task where we'd hijack "loose" Razor files (ones not associated with a project) and allow our editor to run on those with a reasonable set of defaults. This would in turn result in consistent colorization among both sides. Alternatively we could also add "git diff" aware logic where we only swap the left hand side of the diff to the new editor if the right hand side is the new editor, in such an instance we'd also inherit the project information over. Latter would be a harder implementation most likely where the prior would be a bigger discussion on implications (for instance we'd indirectly change the default editor for .NET Framework Razor files).

Of note: This is not a Razor or LSP specific thing. Roslyn (with LSP off) also experiences color differences:

Does that change your prioritization preferences @NTaylorMullen?

Does that change your prioritization preferences @NTaylorMullen?

Ah as from being a 'candidate' to backlog? If so, I don't think so only because the old/new Razor editor are a bit more jarring


Still can not easily register Class randomizer programatically

Hi,
I'm still struggling with registering Randomizer for a certain Type programatically.
Wiki describes randomizer registration for: field... named..of class.
While I need: just class.
Please, help. Did I miss the right line in a manual?

Hi,
The short answer is : You can provide a custom registry in which you associate a randomizer to a type.
Now this is a design question: As a user of the library, I want to be able to register a custom randomizer if the field:

has a given type
has a given name
is declared in a given class
is annotated with a given annotation
(two, three or) All of the above

From an API design point of view, there should be a way to define how to identify a field in the process of bean introspection and this is the role of the FieldDefinition class.
Field name, type and declaring class is the common denominator to identify a field. You can not have two fields with the same name, same type and same declaring class. Field definition are unique according to these three criteria. For the moment, FieldDefinition and its builder are designed to cover these criteria.
You can still use registries to provide custom randomizers by Field or by Type. When you implement a custom RandomizerRegistry, you get access to the Field being introspected and here you can get all informations above: field name, type, annotations etc and hence provide your custom field -> randomizer mapping.
Hope this helps
Kind regards
Mahmoud

Everything sounds right.
I'd only emphasize the existence of 2 API use-cases:

Building a specific randomizer - while specifying what registries to use, etc.
Invoking (a previously built) randomizer on a Class<?> - while modifying the randomization behavior during this particular randomization invocation only.

Case 1 is well covered.
Case 2 - there we can modify using FieldDefinition, which covers most cases.
Still, I had this edge case, when ALL the fields of a particular Type would have to be generated differently from Randomizer default. And FieldDefinition couldn't do the job for me. I wonder if there're should be a way - or is it just too much (for example, because the modification logic becomes too complex.)

Possible fix: https://github.com/benas/random-beans/pull/97

Hi @wheredevel
@PascalSchumacher has kindly added support for registering a randomizer for a given type. This removes the need to implement a custom registry for that.
Would you please give it a try and see if everything is ok for you? I've just published an updated version of the 3.0.0-SNAPSHOT.
In the same way, do you think random beans would also add support to register a randomizer by field name in addition to field type? For example, this can be used like: For every field named "name", use the NameRandomizer. Does this make sense?

Hi Pavel,
Your feature request will be released in v3.0.0 soon.
If you have any problem, don't hesitate to file a bug.
Kind regards
Mahmoud


Bump fastify to version 4.3.0

Hey, I noticed this hasn't been updated to support Fastify 4, I tried bumping the version and fixing any errors that popped up.

There were indeed a few typing conflicts with 4.x, but also and more importantly register() is now async and must be awaited.
1.1.0 should be 4.x-compatible.


What is the resolution it uses on each image?

What is the resolution it uses on each image to extract data? Can we change this setting if needed in container deployments to get better OCR results?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 439c70ec-4484-25de-4942-939112188d58
Version Independent ID: 08f0bb9d-c6a0-d35b-8b3e-e46cca613487
Content: How to install and run containers - Computer Vision
Content Source: articles/cognitive-services/Computer-vision/computer-vision-how-to-install-containers.md
Service: cognitive-services
Sub-service: computer-vision
GitHub Login: @diberry
Microsoft Alias: diberry

@anilvem1
Thanks for the feedback! We are currently investigating and will update you shortly.

@anilvem1 Hi, to extract data from a image Computer Vision returns tags based on thousands of recognizable objects, living beings, scenery, and actions. When tags are ambiguous or not common knowledge, the API response provides 'hints' to clarify the meaning of the tag in context of a known setting. Tags are not organized as a taxonomy and no inheritance hierarchies exist. A collection of content tags forms the foundation for an image 'description' displayed as human readable language formatted in complete sentences. Note, that at this point English is the only supported language for image description.
After uploading an image or specifying an image URL, Computer Vision algorithms output tags based on the objects, living beings, and actions identified in the image. Tagging is not limited to the main subject, such as a person in the foreground, but also includes the setting (indoor or outdoor), furniture, tools, plants, animals, accessories, gadgets etc.
More information could be found at https://docs.microsoft.com/en-us/azure/cognitive-services/Computer-vision/concept-tagging-images
For now, I don't think you can change the settings of the API.

@anilvem1
We will now proceed to close this thread. If there are further questions regarding this matter, please respond here and @YutongTie-MSFT and we will gladly continue the discussion.


feat x/bridge: keeper tests

Epic: #7643
What is the purpose of the change
Added x/bridge keeper tests
Testing and Verifying
Bridge's keeper is covered with tests.
Documentation and Release Note

[ ] Does this pull request introduce a new feature or user-facing behavior changes?
[ ] Changelogs

Where is the change documented?

[ ] Specification (x/{module}/README.md)
[ ] Osmosis documentation site
[x] Code comments
[ ] N/A

Summary by CodeRabbit

New Features

Introduced a new BridgeKeeper for managing bridge functionalities within the app.
Added a bridge module to enhance app capabilities.

Refactor

Updated the Keeper struct and related functions for better management of bridge module parameters.

Tests

Added comprehensive tests for the bridge module's genesis state, parameter updates, and keeper functionalities.

Bug Fixes

Fixed parameter initialization logic in the bridge module to correctly handle default values.

I think we are missing param store wiring, like so https://github.com/osmosis-labs/osmosis/pull/7855/files#diff-1ac782d2fab1d48ad7f7ef06cf2d58b37c99b6b4c98879a5ad86a985b73fad58R755-R757


v5 Grid has odd horizontal spacing when placed alongside other components in a Stack

[x] The issue is present in the latest release.
[x] I have searched the issues of this repository and believe that this is not a duplicate.

Attempting to nest a Grid container inside a Stack results in the container being offset horizontally from the rest of the components. When no other components are in the Stack besides the Grid, it works as expected.
<Container component="main" maxWidth="sm">
  <Stack spacing={8}>
    {/* Removing this text component corrects the alignment. */}
    <Typography>test</Typography>
    <Grid container spacing={8}>
      {Array.from(Array(6)).map((_, index) => (
        <Grid item xs={12} key={index}>
          <Paper>offset</Paper>
        </Grid>
      ))}
    </Grid>
  </Stack>
</Container>

Sandbox: https://codesandbox.io/s/nifty-sanderson-i5te7
Might be related to #21507

@QuaternionMark you can specify only rowSpacing and it should fix the issue - https://codesandbox.io/s/xenodochial-morning-13m8n?file=/src/Demo.js The spacing prop is shorthand for both rowSpacing and columnSpacing that is why you had the spacing at the start.

The spacing prop is shorthand for both rowSpacing and columnSpacing that is why you had the spacing at the start.

@mnajdova That's not the issue. Using the  columnSpacing prop without the Typography component (or any element) being above the Grid in the Stack doesn't cause any spacing issues
After looking into it, it seems that the issue occurs due to the Grid container wanting to apply a negative margin, but then it gets overridden by Stack's spacing prop and it results in a gutter being visible from the nested items.
I could just avoid setting a columnSpacing prop on the Grid, but then that makes doing layouts where it changes from row-spanned to col-spanned more cumbersome by having to do <Grid container rowSpacing={{ xs: 4 }} columnSpacing={{ md: 4 }}>. It also prevents someone from doing any horizontal spacing on vertically-aligned grids without applying margins to the components nested in the Grid.
Is this behavior documented somewhere or is it a bug?


v2.1 and v2.0

https://github.com/snowplow/snowplow-golang-tracker/blob/v2.1.0/go.mod#L13
Does this prevent v2.1 to be installed?

It does - this is a bug introduced by the addition of the stress testing tool.  Will be fixed in v2.2.0.

Hi @fi0 this has now been fixed - as per your other ticket please change the import line to:

"github.com/snowplow/snowplow-golang-tracker/v2/tracker"


Manually update actions cache if build fails

https://stackoverflow.com/a/64819132

Closed in #108


HttpWriter including SalesForceRestWriter, ThrottleWriter, etc

Added:

SalesForceRestWriter

insert and upsert
supports batch insert and upsert

ThrottleWriter (by QPS and bytes)
RetryWriter (default: exponential backoff)
AvroToRestJsonEntryConverter (Converts Avro to Json where Json is based on user provided HOCON template so that Json can be any structure).

Unit test included. Passed unit test and integration test.
Design doc(Linkedin internal. Will update gobblin wiki once pull request is completed.): https://docs.google.com/a/linkedin.com/document/d/105PhMzM3O_FeQmrD5-IFhbK24dSbnUeqoM3T2pNKlnM/edit?usp=sharing
Will pull from latest master branch once reviewed.

@chavdar Could you take a look? Thanks.

@ydai1124 can you review?

@ydai1124 Thanks for the review. I've addressed PR comments and will be working on merging it from master branch's latest code.

@ydai1124 After merging it from master branch, there's a exception I am getting after successfully completed the task and when it's writing task state into FileSystem, it's complaining the FileSystem is already closed. Have you seen this problem? (I am not sure if my change brought this problem as my change didn't touch this part and other JDBC writer is having same failure as well.)
2016-08-19 14:13:25,021 INFO [TaskExecutor-0] gobblin.runtime.Task: publish.data.at.job.level is true. Will publish data at the job level.
2016-08-19 14:13:25,022 INFO [main] gobblin.runtime.mapreduce.MRJobLauncher: All assigned tasks of job job_connectorJobTypeTests_gobblinTest-mySqlToHdfs_1471641185829 have completed in container attempt_1461347166530_0447_m_000000_0
2016-08-19 14:13:25,022 INFO [main] gobblin.runtime.mapreduce.MRJobLauncher: Writing task state for task task_connectorJobTypeTests_gobblinTest-mySqlToHdfs_1471641185829_0
2016-08-19 14:13:25,022 INFO [TaskExecutor-0] gobblin.runtime.mapreduce.MRTaskStateTracker: Task task_connectorJobTypeTests_gobblinTest-mySqlToHdfs_1471641185829_0 completed in 3164ms with state SUCCESSFUL
2016-08-19 14:13:25,025 INFO [TaskExecutor STOPPING] gobblin.runtime.TaskExecutor: Stopping the task executor
2016-08-19 14:13:25,025 INFO [TaskExecutor STOPPING] gobblin.runtime.TaskExecutor: Attempting to shutdown ExecutorService: java.util.concurrent.ThreadPoolExecutor@25277b25[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 1]
2016-08-19 14:13:25,025 INFO [TaskExecutor STOPPING] gobblin.runtime.TaskExecutor: Successfully shutdown ExecutorService: java.util.concurrent.ThreadPoolExecutor@25277b25[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1]
2016-08-19 14:13:25,025 INFO [TaskExecutor STOPPING] gobblin.runtime.TaskExecutor: Attempting to shutdown ExecutorService: java.util.concurrent.ScheduledThreadPoolExecutor@2be091f1[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
2016-08-19 14:13:25,025 INFO [TaskExecutor STOPPING] gobblin.runtime.TaskExecutor: Successfully shutdown ExecutorService: java.util.concurrent.ScheduledThreadPoolExecutor@2be091f1[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
2016-08-19 14:13:25,025 INFO [TaskExecutor STOPPING] gobblin.runtime.TaskExecutor: Attempting to shutdown ExecutorService: java.util.concurrent.ThreadPoolExecutor@2fdf1055[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 1]
2016-08-19 14:13:25,025 INFO [TaskExecutor STOPPING] gobblin.runtime.TaskExecutor: Successfully shutdown ExecutorService: java.util.concurrent.ThreadPoolExecutor@2fdf1055[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1]
2016-08-19 14:13:25,025 INFO [MRTaskStateTracker STOPPING] gobblin.runtime.mapreduce.MRTaskStateTracker: Stopping the task state tracker
2016-08-19 14:13:25,025 INFO [MRTaskStateTracker STOPPING] gobblin.runtime.mapreduce.MRTaskStateTracker: Attempting to shutdown ExecutorService: java.util.concurrent.ScheduledThreadPoolExecutor@759598aa[Shutting down, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
2016-08-19 14:13:25,026 INFO [MRTaskStateTracker STOPPING] gobblin.runtime.mapreduce.MRTaskStateTracker: Successfully shutdown ExecutorService: java.util.concurrent.ScheduledThreadPoolExecutor@759598aa[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
2016-08-19 14:13:25,029 INFO [main] org.apache.hadoop.mapred.MapTask: Ignoring exception during close for org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader@1cb3ec38
java.io.IOException: Filesystem closed
at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:689)
at org.apache.hadoop.hdfs.DFSInputStream.close(DFSInputStream.java:617)
at java.io.FilterInputStream.close(FilterInputStream.java:181)
at org.apache.hadoop.util.LineReader.close(LineReader.java:150)
at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.close(LineRecordReader.java:193)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.close(MapTask.java:500)
at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:1973)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:773)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
2016-08-19 14:13:25,031 WARN [main] org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:jnchang (auth:SIMPLE) cause:java.io.IOException: Filesystem closed
2016-08-19 14:13:25,031 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.io.IOException: Filesystem closed
at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:689)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1744)
at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1112)
at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1108)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1108)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1399)
at gobblin.metastore.FsStateStore.put(FsStateStore.java:135)
at gobblin.runtime.AbstractJobLauncher.runWorkUnits(AbstractJobLauncher.java:644)
at gobblin.runtime.mapreduce.MRJobLauncher$TaskRunner.run(MRJobLauncher.java:597)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)

@jinhyukchang Which test you saw these failures? It looks like the statestore fs gets closed before it should. I don't recall any change related to it.

@ydai1124 I just ran integration test case from HDFS(AVRO) --> Salesforce and HDFS(AVRO) -> MySQL which were working before merging.

@ydai1124 Found the root cause. There was a recent commit (8/14) that closes FileSystem.
https://github.com/linkedin/gobblin/commit/342d20b1dbe9f1f83fe53c6ee0299c04bb9d71b8#diff-f6b4e6837a023e738b2538aa22b84e8dR170
When I commented close part in HadoopFsHelper, it's completing task without exception. Since it's a separate problem, I will create an issue on this.
Please let me know if there's anything more to be done to be merged.

Created an issue.
https://github.com/linkedin/gobblin/issues/1219

@jinhyukchang Can you squash your commits before merging? Thanks!

@ydai1124 Since I've already merged with master branch and the changes are already pushed, I am not sure if there's an easy way to squash commits. Is there an easy way to do squash pushed commit when there's merge in it?
When I use git rebase -i, it brings almost hundred commits and brings conflicts.

@jinhyukchang That's fine. I am gonna merge this PR.


KAFKA-15581: Introduce ELR

Introduce ELR

new PartitionChangeRecord
new PartitionRecord
Basic logic of how ELR, lastKnownELR updated.

Note: No election-related changes.

No relevant UT failure.


You are attempting to use Flash Attention 2.0 with a model not initialized on GPU. Make sure to move the model to GPU after initializing it on CPU with model.to('cuda').

I get the following error message when I try to deploy the Docker Container. Do I need Flash Attention? Will there be a significant difference in speed?

I am getting the same issue.
I don't understand what's going on. If I execute nvida-smi inside the container it shows me the GPU
Did you manage to solve it ?

Hi there, this should be a warning and not a error. If deployed on fly this warning might actually be inaccurate as it does use the GPU but it's a timing issue as the where the GPU hasn't initialized on the start of the machine which throws the warning. on when the GPU is detected by the system.
You can read more here: https://fly.io/blog/transcribing-on-fly-gpu-machines
Attached the referece:


Add flag for persistent database connections

[public]

FixVersion Change: <IP_ADDRESS>

This issue will be / was solved in Shopware version <IP_ADDRESS>.
[created from NEXT-40008, comment 524608]

[public] Shopware automated response: This issue is linked to the internal issue: NEXT-40008
[created from NEXT-40008, comment 524611]


Increase Overall Unit Test Coverage - ECMPS UI - files w/0% to 20% coverage

Unit Test cases to address include:
src/components/MonitoringPlanTab/MonitoringPlanTab.js  - coverage at  17.6%
src/components/MonitoringPlanTabRender/MonitoringPlanTabRender.js - coverage at  3.2%


Is there a way to export all the post I liked?

I would like to automate downloading all the posts have liked. So far I am using the combination with #hashtag + --post-filter=viewer_has_liked, but it's not very convenient.
If there is no hashtag added, I get the message "No targets were specified, thus nothing has been downloaded."
Are there any options for this in instaloader I have missed?

Unfortunately not. As alternatives, there is --post-filter=viewer_has_liked :feed, which only loads the pictures of the feed, or saving all interesting pictures and using the :saved target.

@aandergr  thanks a lot, both options will go for me 👍

It is possible to see all the posts you've liked in the app, so it's not impossible to do similarly to how stories were handled before they were available on the web page...
import instaloader
import json
l = instaloader.Instaloader()
l.load_session_from_file(USERNAME)
del l.context._session.headers['Host']
del l.context._session.headers['Origin']
del l.context._session.headers['X-Instagram-AJAX']
del l.context._session.headers['X-Requested-With']
l.context._session.headers['User-Agent'] = 'Instagram 10.3.2 (iPhone7,2; iPhone OS 9_3_3; en_US; en-US; scale=2.00; 750x1334) AppleWebKit/420+'

params = {}
done = False
while not done:
	data = l.context.get_json(path='api/v1/feed/liked/', host='i.instagram.com', params=params)
	for item in data['items']:
		p = instaloader.Post.from_shortcode(l.context, item['code'])
		if not l.download_post(p, ':liked'):
			done = True
			break
	if not data['more_available']:
		break
	params['max_id'] = data['next_max_id']

Unfortunately, the responses to the "app" are incompatible with the json node format that Post expects,  so each post has to be infered from its shortcode. While I don't think the requests to instagram.com/p/code/ are counted towards the limit, it slows down the process. In either case it seems to be necessary to be able to tell if a post has a video or multiple images (might be solved by setting some magic HTTP headers).
A quick-and-dirty variant would be to rawly download just the base image itself:
for item in items:
	url = item['image_versions2']['candidates'][0]['url']
	l.context.get_and_write_raw(url, item['code'] + '.jpg')
	print(url)

(On a side note related to #80 , each item has a numerically encoded filter_type attribute.)

ADDENDUM:
Yes, by setting a newer user agent, such as: Instagram 10.26.0 (iPhone7,2; iOS 10_1_1; en_US; en-US; scale=2.00; gamut=normal; 750x1334) AppleWebKit/420+ all "album"/"sidecar" images gets exposed in the returned json.
for item in data['items']:
  def _get(x):
    url = x['image_versions2']['candidates'][0]['url']
    l.context.get_and_write_raw(url, item['user']['username'] + '-' + x['pk'] + '.jpg')
  if 'image_version2' in item:
    _get(item)
  elif 'carousel_media' in item:
    for i in item['carousel_media']:
      _get(i)

Ought to work (not tested) as a more complete fast way, without spamming the servers for each shortcode.


Mast: Watch condor-sw-firmware.blogspot.com

Mast requests the watch of the watch_keyword condor-sw-firmware\.blogspot\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

condor-sw-firmware\.blogspot\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by tripleee in Charcoal HQ

Approved by Daniil in Charcoal HQ

Approved by Daniil in Charcoal HQ


Campaign Name not populated on login

When a user has a campaign name stored in a cookie before login, the act of logging in will redirect to the current and correct campaign but does not populate the campaign in the header. This requires a hard refresh of the app for Ping to fire and populate it.
Login should do a campaign check or something.

Not sure that this actually a bug. Cookies are cleared on logout. You should never have a cookie and be logged out. Closing until this can be replicated.


Wrap g apps hook

Motivation for this change
uses the more correct hook instead of NIH-gnome-program-wrapping
Things done

[ ] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] OS X
[ ] Linux

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Fits CONTRIBUTING.md.

previous discussion #16183

Checking the travis output, gnome-bluetooth fails to build. Did you actually test your change?
installing
install flags: install    gsettingsschemadir=/nix/store/10s8hrw09y02sq0r9rmlskggn8nfqib2-gnome-bluetooth-3.18.2/share/gsettings-schemas/gnome-bluetooth-3.18.2/glib-2.0/schemas/
make: *** No rule to make target 'install'.  Stop.
builder for ‘/nix/store/brpvjc86wsqwg417a3y0jdrif7v67xr7-gnome-bluetooth-3.18.2.drv’ failed with exit code 2

Perhaps it's better to avoid wrapGAppsHook in gnome-bluetooth for now.

It works locally, im not sure when i can resume the experiment

@RonnyPfannschmidt indeed I've tried locally and it works.

i resumed the check, and the binary is broken it seems :/

@RonnyPfannschmidt you mean it compiles but the wrapper is broken?

correct - atm i can no longer use bluetooth

@RonnyPfannschmidt ok let's do it step by step, remove the bluetooth patch from this PR so I can merge it

@lethalman i force-pushed to remove the latest commits, please have a look

Built locally, thanks.

Backported to release-16.03 as requested by @RonnyPfannschmidt


check for docker availability

It would be good to have a way to check whether Docker is actually available or not, as a simple static boolean function, e.g. public static boolean DockerClient.isDockerAvailable(). That would allow us to use JUnit/TestNG-specific ways to skip tests which require docker containers.

The PR https://github.com/testcontainers/testcontainers-java/pull/2605 can be used as follows: DockerClientFactory.instance().isDockerAvailable().
Meanwhile you can add the following Kotlin code to your project as a workaround:
@Synchronized
fun DockerClientFactory.isDockerAvailable(): Boolean {
    // todo Remove when https://github.com/testcontainers/testcontainers-java/issues/2110 is fixed.
    val m = DockerClientFactory::class.java.getDeclaredMethod("getOrInitializeStrategy")
    m.isAccessible = true
    return try {
        m.invoke(this)
        true
    } catch (ex: Exception) {
        false
    }
}

In java one can use the following:
try {
	DockerClientFactory.instance().client();
} catch (Exception e) {
      	Assert.fail(e.getMessage());
}

For others searching for a similar solution, this works in the current version of TestContainers:
@Testcontainers(disabledWithoutDocker = true)
The unit tests are skipped without errors.


Vuejs template

You guys are rocking. Really a nice template. Is there any possibilities to fork this template with VueJS?

Hey @nasr18, thanks for your kind words. I believe you can not really 'fork' it, as you'd need to develop all of the components from scratch. Though, if your components will exactly repeat html structure, you can re-use the styles and theme system.


Training COCO fails with error - Not compiled with GPU support

Hi,
Thank you for sharing the code for YOSO. I followed the instructions from the Readme and installed PyTorch and other dependencies in that order. I am able to run the python setup develop but when I get to training COCO with R50 with the following command, I am getting a runtime error saying "Not compiled with GPU Support".
python projects/YOSO/train_net.py --num-gpus 1 --config-file /home/jmadinn/perception/YOSO/projects/YOSO/configs/coco/panoptic-segmentation/YOSO-R50.yaml
I have added the complete traceback at the bottom. I can confirm that the torch.cuda.is_available() returns True and CUDA version is 11.3 as in the instructions. Could you please confirm if this is an error you have faced before and how I can resolve it? I have already created a new environment twice but the issue persists.
Thanks in advance!
Traceback (most recent call last):
File "projects/YOSO/train_net.py", line 295, in
launch(
File "/home/jmadinn/perception/YOSO/detectron2/engine/launch.py", line 62, in launch
main_func(*args)
File "projects/YOSO/train_net.py", line 289, in main
return trainer.train()
File "/home/jmadinn/perception/YOSO/detectron2/engine/defaults.py", line 420, in train
super().train(self.start_iter, self.max_iter)
File "/home/jmadinn/perception/YOSO/detectron2/engine/train_loop.py", line 134, in train
self.run_step()
File "/home/jmadinn/perception/YOSO/detectron2/engine/defaults.py", line 430, in run_step
self._trainer.run_step()
File "/home/jmadinn/perception/YOSO/detectron2/engine/train_loop.py", line 228, in run_step
loss_dict = self.model(data)
File "/home/jmadinn/.conda/envs/YOSO/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/home/jmadinn/perception/YOSO/projects/YOSO/yoso/segmentator.py", line 84, in forward
neck_feats = self.yoso_neck(features)
File "/home/jmadinn/.conda/envs/YOSO/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/home/jmadinn/perception/YOSO/projects/YOSO/yoso/neck.py", line 186, in forward
features = self.deconv(features_list)
File "/home/jmadinn/.conda/envs/YOSO/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/home/jmadinn/perception/YOSO/projects/YOSO/yoso/neck.py", line 126, in forward
x = self.deform_conv1(x5)
File "/home/jmadinn/.conda/envs/YOSO/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/home/jmadinn/perception/YOSO/projects/YOSO/yoso/neck.py", line 60, in forward
out = self.dcn(out, offset, mask)
File "/home/jmadinn/.conda/envs/YOSO/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/home/jmadinn/perception/YOSO/detectron2/layers/deform_conv.py", line 473, in forward
x = modulated_deform_conv(
File "/home/jmadinn/perception/YOSO/detectron2/layers/deform_conv.py", line 220, in forward
_C.modulated_deform_conv_forward(
RuntimeError: Not compiled with GPU support

After much googling, found out this issue occurs due to incomplete cuda-toolkit installed along with pytorch. Resolved by installing cudatoolkit-dev version through Conda, deleting the already populated build folder within YOSO and re-running setup.py develop


Change the design of message box in dark theme

The message box looks the same on both light and dark theme.
We can change the look of message box under dark theme so it looks much better with the theme
Please assign me this issue under gssoc21

Any specific suggestions or ideas??

Any specific suggestions or ideas??

I will share it with you by tomorrow morning

Hey @abhijeet007rocks8 please let me know should I create a PR for this

please review this issue @DhairyaBahl  @abhijeet007rocks8


servant-0.7

servant-0.7 (Haskell Servant<EMAIL_ADDRESS>@jkarni) is out of bounds for:

[ ] dixi-<IP_ADDRESS> (>=0.5 && < 0.7). Liam O'Connor<EMAIL_ADDRESS>@liamoc
[x] lackey-0.3.1 (>=0.5 && < 0.7). Taylor Fausak<EMAIL_ADDRESS>@tfausak. @tfausak
[ ] servant-JuicyPixels-<IP_ADDRESS> (>=0.4 && < 0.7). Timo von Holtz<EMAIL_ADDRESS>@tvh. @tvh
[ ] servant-yaml-<IP_ADDRESS> (>=<IP_ADDRESS> && < 0.7). Oleg Grenrus<EMAIL_ADDRESS>@phadej. @phadej

servant-blaze-0.7 is out of bounds for:

[ ] dixi-<IP_ADDRESS> (>=0.5 && < 0.7). Liam O'Connor<EMAIL_ADDRESS>@liamoc

servant-docs-0.7 (Haskell Servant<EMAIL_ADDRESS>@jkarni) is out of bounds for:

[ ] dixi-<IP_ADDRESS> (>=0.5 && < 0.7). Liam O'Connor<EMAIL_ADDRESS>@liamoc

servant-foreign-0.7 is out of bounds for:

[x] lackey-0.3.1 (>=0.5 && < 0.7). Taylor Fausak<EMAIL_ADDRESS>@tfausak. @tfausak

servant-server-0.7 (Haskell Servant<EMAIL_ADDRESS>@jkarni) is out of bounds for:

[ ] dixi-<IP_ADDRESS> (>=0.5 && < 0.7). Liam O'Connor<EMAIL_ADDRESS>@liamoc
[ ] servant-yaml-<IP_ADDRESS> (>=<IP_ADDRESS> && < 0.7). Oleg Grenrus<EMAIL_ADDRESS>@phadej. @phadej

I'm not sure exactly what changed between 0.6.1 and 0.7. Here's the raw comparison: https://github.com/haskell-servant/servant/compare/v0.6.1...v0.7

Fixed in lackey-0.3.2.

Here's the changelogs between 0.6 and 0.7: https://gist.github.com/soenkehahn/b62011578690421d98ac5bb22e02878a

Seems that dixi is only one in the list, opened PR to bump the bounds.

@liamoc what are the chances of getting dixi updated to servant-0.7 soon?

I bumped the bounds on dixi-<IP_ADDRESS>

Thanks!!

Should be in Stackage Nightly 2016-05-17.


Introduce other point coordinate systems

We currently have only Affine and Projective coordinates logic for the curve arithmetic.
Being able to support other point coordinate systems may bring some performance benefits, for instance in the schnorr-sig crate.
For instance Jacobian coordinates are known for yielding relatively fast point doubling methods (about 2/3 of the running time of a point doubling in Projective coordinates) for a slightly slower addition algorithm. However, the use of Straus-Shamir's trick in signature verification reduces the impact of such change (as we have now N doublings and 2N additions).
Jacobian coordinates variants (like Chudnovsky) could also be investigated, though at the cost of a heavier point representation (240B instead of 144B for Jacobian/Projective).

There's actually one issue with the use of Jacobian coordinates is constant-time addition of points (either homogenous or mixed with an affine point). While doubling can be made easily constant-time and still efficient (necessitating a final check before returning, similarly to its projective counterpart), there is no efficient exception-free jacobian addition formula, which requires to perform several additional checks (whether p1 or p2 is the infinity point, or if p1 = +/- p2), which would make any jacobian scalar multiplication non-competitive with the existing projective coordinates based one.

It may make sense to have Jacobian coordinates (or variants) for variable-time operation purposes only, as in that case it would be  faster than equivalent variable time scalar multiplication methods in other coordinate systems.


Shard syncer tests are nearly impossible to debug and take a long time to run

So, I didn't notice these new tests until they were merged, and I only did because my latest PR (#684) introduced some changes to p2p.Server that caused those tests to break. Unfortunately, they are quite complex, high-level, integration tests that give no visibility into what's going on under the hood, so when they break it is a nightmare to debug. Also, they take a really long time to run:

19.36s call     tests/p2p/test_sharding.py::test_shard_syncer[3-connections1]
18.91s call     tests/p2p/test_sharding.py::test_shard_syncer[3-connections2]
10.41s call     tests/p2p/test_sharding.py::test_shard_syncer[2-connections0]

At this point, I understand writing unit tests for those parts will be tricky, but having a complex integration test like this as part of our CI is likely to slow down development of p2p-related code significantly -- I just spent more than 1h to get them to pass again. I think we'd be better off skipping those tests in CI, the same way we do for the tests in tests/p2p/test_lightchain_integration.py

I thought sharding was dropped from the repo in https://github.com/ethereum/py-evm/pull/518 - if that's correct, then the tests seem like cruft. But I haven't been following the sharding work closely enough to be sure.

Sorry for that! I'm also not happy with those tests, but just disabling them because they break is not a good long-term solution. The shard syncer and similar services (like the various chain syncers) are important components which should be tested in the CI (and are logic wise comparatively simple as well -- the shard syncer is at least).
I think the root issue is that there isn't an easy way to test the high-level service code only without running all of the low level code networking code as well. Do you think creating a MockPeerPool that mocks all the handshaking/connection/message passing stuff away, but has the right interface plus some entry points for test functions (like add_peer, remove_peer) would be a viable solution?

I thought sharding was dropped from the repo in #518 - if that's correct, then the tests seem like cruft.

No, the code that belonged to a now deprecated specification, the tests here are new.

Sorry for that! I'm also not happy with those tests, but just disabling them because they break is not a good long-term solution

It's not because they break that I suggest we skip them during the usual CI, it's because running a p2p.Server means running a PeerPool, DiscoveryProtocol, the coroutine that periodically refreshes the NAT portmap the asyncio server that listens for incoming peer connections, and now the syncer as well. We do need an integration test to make sure that all those things work together, but it should be a very basic integration test. I think tests for the shard syncer (or any other parts of py-evm) that run as part of our CI should not be running all of these services, because it makes the tests very fragile and hard to debug. And the shard syncer tests make things even worse because they do nasty things (like setting chaindb to None in p2p.Server) that should not be expected to work.
I think it might be worth try mocking those services, yeah


Reviewed content and updated date

#sign-off

Hi @charislove, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no contribution license agreement is required at this point. Real humans will now evaluate your PR.

TTYL, AZPRBOT;

@charislove : Thanks for your contribution to the Azure documentation! The author, @hning86, has been notified to review your proposed change.

good

@tysonn This is ready to merge. @charislove - You can submit changes to the private azure-content-pr repo rather than the public azure-content repo, and then you can sign them off without needing further approval. It's explained in the instructions in my weekly freshness reminder email, but let me know if you have any questions.

@garyericson - This PR has a merge conflict - can you resolve please to unblock this for merging?

@tysonn @charislove - Let's just close this pull request.
I couldn't figure out how to find and resolve a merge conflict in someone else's branch of the public repo. The only change here was updating ms.date, so I made that change in the azure-content-pr repo and submitted a separate pull request for it.


Figure 6 Code

Hello Everyone!
I am trying to find the code to generate figure 6 in the paper, but I am not able to find the piece of code which can generate figure 6 in the paper.
Can anyone help in this issue?

Hi @prakhar123pradhan, the visualization of the attention map? This related issue may help you generate the figure.

Hi @prakhar123pradhan, the visualization of the attention map? This related issue may help you generate the figure.

Thanks a lot !!

Thanks a lot !!


varLib.interpolatable --json with scipy installed: Object of type int64 is not JSON serializable

Hello.
It seems that, the moment I install scipy into my project, varLib.interpolatable starts using it, and then fails outputting JSON with:
TypeError: ('Object of type int64 is not JSON serializable')

To clarify, the above happens when running the following with scipy installed:
fonttools varLib.interpolatable font1.ttf font2.ttf --json

I assume this may be due to some conditional scipy imports? E.g.:
https://github.com/fonttools/fonttools/blob/90fed08964bba2b81f0e1ec9222c95b0bee05384/Lib/fontTools/varLib/interpolatableHelpers.py#L174-L188
https://github.com/fonttools/fonttools/blob/90fed08964bba2b81f0e1ec9222c95b0bee05384/Lib/fontTools/varLib/interpolatableHelpers.py#L306-L308
Happy to dig deeper, and maybe provide a minimal reproduction, with some guidance on how to best supply that?
Thank you :)

I assume that the solution here would be, when scipy is installed, to supply a custom JSON encoder that can handle numpy types?
So here:
https://github.com/fonttools/fonttools/blob/90fed08964bba2b81f0e1ec9222c95b0bee05384/Lib/fontTools/varLib/interpolatable.py#L915
You could do something like...
json_encoder = json.JSONEncoder
try:
    import scipy
    import numpy as np

    class NumpyEncoder(json.JSONEncoder):
        def default(self, obj):
            if isinstance(obj, np.integer):
                return int(obj)
            if isinstance(obj, np.floating):
                return float(obj)
            if isinstance(obj, np.ndarray):
                return obj.tolist()
            return super(NumpyEncoder, self).default(obj)

    json_encoder = NumpyEncoder
except ImportError:
    pass

print(json.dumps(problems, cls=json_encoder), file=f)

I can confirm that the above approach resolves the issue. Should I put together a PR?

I think we want to convert back to regular Python types or whatever it is that the non-scipy path is using. Thanks for the report!

Would be great if you can send a PR with the approach I suggested.

Ah, yes, that would keep all the scipy/numpy code contained to the conditional import blocks. Much tidier. Will set up a PR in the coming days, thank you.


No  CalibrateRobertson (HDR) wrapper

Summary of your issue
Though you wrapped NativeMethods.photo_Ptr_CalibrateRobertson, in NativeMethods
didn't find the corrosponding CalibrateRobertson
Environment
N/A
What did you do when you faced the problem?
Wrote my own wrapper
Example code:

using System;
using OpenCvSharp.Internal;

namespace OpenCvSharp
{
    /// <summary>
    /// The base class for camera response calibration algorithms.
    /// </summary>
    public class CalibrateRobertson : CalibrateCRF
    {
#nullable enable
        private Ptr? ptrObj;

        /// <summary>
        /// Creates instance by raw pointer cv::ml::Boost*
        /// </summary>
        protected CalibrateRobertson(IntPtr p)
        {
            ptrObj = new Ptr(p);
            ptr = ptrObj.Get();
        }

        /** @brief Creates CalibrateRobertson object
@param max_iter maximal number of Gauss-Seidel solver iterations.
@param threshold target difference between results of two successive steps of the minimization.
         */
        public static CalibrateRobertson Create(int maxIter = 30, float threshold = 0.01f, bool random = false)
        {
            NativeMethods.HandleException(
                NativeMethods.photo_createCalibrateRobertson(maxIter, threshold, out var ptr));
            return new CalibrateRobertson(ptr);
        }

        /// <summary>
        /// Releases managed resources
        /// </summary>
        protected override void DisposeManaged()
        {
            ptrObj?.Dispose();
            ptrObj = null;
            base.DisposeManaged();
        }

        internal class Ptr : OpenCvSharp.Ptr
        {
            public Ptr(IntPtr ptr) : base(ptr)
            {
            }

            public override IntPtr Get()
            {
                NativeMethods.HandleException(
                    NativeMethods.photo_Ptr_CalibrateRobertson_get(ptr, out var ret));
                GC.KeepAlive(this);
                return ret;
            }

            protected override void DisposeUnmanaged()
            {
                NativeMethods.HandleException(
                    NativeMethods.photo_Ptr_CalibrateRobertson_delete(ptr));
                base.DisposeUnmanaged();
            }
        }
    }
}

Output:

paste your output

What did you intend to be?

Thanks! :+1:


Switched to using new MongoDB service in Bluemix

Fixes #1029
The MongoLabs service in Bluemix is being removed so we need to use a different Mongo service.

LGTM

@lirantal LGTM.

LGTM


using Razzle plugins and extending Webpack at the same time

Hi, everybody.
I have developed a package and using it with some other plugins.
// razzle.config.js
module.exports = {
  plugins: [
      'typescript',
      'runtimeenv'
  ]
};

now I want to extend Webpack to add less loader.
something like this:
// razzle.config.js

module.exports = {
  modify: (config, { target, dev }, webpack) => {
    // do something to config

    return config;
  },
};

but I just can export one of them.
is there any way to use both of them together?

I have the same problem, how did you solve this?

module.exports = {
  plugins: [
      'typescript',
      'runtimeenv'
  ],
  modify: (config, { target, dev }, webpack) => {
    // do something to config

    return config;
  },
};


xosview2: 2.3.1 -> 2.3.2

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/xosview2/versions.
meta.description for xosview2 is: "Lightweight program that gathers information from your operating system and displays it in graphical form"
meta.homepage for xosview2 is: "http://xosview.sourceforge.net/index.html"
meta.changelog for xosview2 is: ""
Updates performed

Version update

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

/nix/store/p4ysv5b71xzgcs8r6wzk5nhlfqirbmay-xosview2-2.3.2/bin/xosview2 passed the binary check.

1 of 1 passed binary check by having a zero exit code.

1 of 1 passed binary check by having the new version present in output.

found 2.3.2 with grep in /nix/store/p4ysv5b71xzgcs8r6wzk5nhlfqirbmay-xosview2-2.3.2

found 2.3.2 in filename of file in /nix/store/p4ysv5b71xzgcs8r6wzk5nhlfqirbmay-xosview2-2.3.2

directory tree listing: https://gist.github.com/ff6208f6da03dcf74332989bf1f2ee91

du listing: https://gist.github.com/1f7129b06dd0d9303cd29a2eca424cfd

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)

1 package rebuild(s)

1 x86_64-linux rebuild(s)
1 i686-linux rebuild(s)
1 x86_64-darwin rebuild(s)
1 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
xosview2

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/p4ysv5b71xzgcs8r6wzk5nhlfqirbmay-xosview2-2.3.2 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A xosview2 https://github.com/r-ryantm/nixpkgs/archive/bd514f19b6856d6c8d91987ff818e99470f72cfe.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/p4ysv5b71xzgcs8r6wzk5nhlfqirbmay-xosview2-2.3.2
ls -la /nix/store/p4ysv5b71xzgcs8r6wzk5nhlfqirbmay-xosview2-2.3.2/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

xosview2

Maintainer pings
cc @SeanZicari for testing.

Result of nixpkgs-review pr 133981 at bd514f19 run on x86_64-linux 1

1 package built successfully:

xosview2

1 suggestion:

warning: unclear-gpl
gpl2 is a deprecated license, please check if project uses gpl2Plus or gpl2Only and change meta.license accordingly.
Near pkgs/tools/X11/xosview2/default.nix:37:5:
   |
37 |     license = with licenses; [ gpl2 bsdOriginal ];
   |     ^


Use native Object.defineProperty for POJSOs when needed

Plain old JS objects without defineGetter or defineSetter should use the native Object.defineProperty() function to prevent undefined function exception.
See: issue #1399 for more detail.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

This has been fixed via #1408


Set-PnPContext not switching the context!

Reporting an Issue or Missing Feature
Issue with using Set-PnPContext to switch between two connections
Expected behavior
as per the documentation example, Set-PnPContext must switch to the context defined as parameter
Connect-PnPOnline -Url $siteAurl -Credentials $credentials
$ctx = Get-PnPContext
Get-PnPList # returns the lists from site specified with $siteAurl
Connect-PnPOnline -Url $siteBurl -Credentials $credentials
Get-PnPList # returns the lists from the site specified with $siteBurl
Set-PnPContext -Context $ctx # switch back to site A
Get-PnPList # returns the lists from site A

Actual behavior
Regardless of the context mentioned in the command parameter, the context remains as per the last connection made. the command Set-PnPContext is doing nothing
Steps to reproduce behavior
Connect-PnPOnline -URL $SiteA_URL -Interactive
$ctxA=Get-PnPContext
$listA= Get-PnPList # show lists in site A

Connect-PnPOnline -URL $SiteB_URL -Interactive
$ctxB=Get-PnPContext
$listB= Get-PnPList # show lists in site B

$ctx=Get-PnPContext # return the context of the last connection made, for Site B
Set-PnPContext -Context $ctxA # Expected to set the context of the site A
$list= Get-PnPList # Still return the list from the site B, which is the last connection made
$ctx=Get-PnPContext # still return the context of Site B

What is the version of the Cmdlet module you are running?
1.9.0
Which operating system/environment are you running PnP PowerShell on?

[ ] Windows
[ ] Linux
[ ] MacOS
[ ] Azure Cloud Shell
[ ] Azure Functions
[ ] Other : please specify

This has been fixed in the latest nightly builds
https://github.com/pnp/powershell/issues/1089
Please try with that and let us know :)

Hey closing this issue.
Please update to the latest 1.10 release which contains the fix for this issue :)

for the information, problem still as is, even after upgrading to 1.10

Please do a clean install

@omarmallat ,can you please do a clean install and remove the old version of the module and they try ?

I removed all the old versions, I installed the latest 1.10.0 and still having the same issue.

The Set-PnPContext command seems to switch context correctly, but it gets overwritten when running e.g. Get-PnPList.
Other commands also seem to be using the url from the current connection, instead of the current context., for example Get-PnPField, which returns The object is used in the context different from the one associated with the object..
Steps to reproduce the context being overwritten by Get-PnPList
Connect-PnPOnline -URL $SiteA_URL -Interactive
$ctxA=Get-PnPContext
Connect-PnPOnline -URL $SiteB_URL -Interactive
$ctxB=Get-PnPContext

Set-PnPContext -Context $ctxA

Get-PnPContext # returns context for site A

Get-PnPSite # returns site A

Get-PnPList # returns lists from site B

Get-PnPSite # returns site B - the current context was switched by running get-pnplist

Running v1.10.24

As a workaround, it's possible to (for now) use the deprecated Web parameter, like so
Connect-PnPOnline -URL $SiteA_URL -Interactive
$ctxA=Get-PnPContext

Connect-PnPOnline -URL $SiteB_URL -Interactive
$ctxB=Get-PnPContext

Set-PnPContext -Context $ctxA
$webA = Get-PnPWeb

Get-PnPList -Web $webA # returns all lists from $webA, and doesn't change current PnPContext

The same workaround can also be used for e.g. Get-PnPField, Get-PnPUser and all commands doing CurrentWeb.Context.Load or ClientContext.Load(CurrentWeb.[whatever]) in ExecuteCmdlet

Having a look here to see if I can repro this. I've done some rework on the connection stuff. Perhaps this fixed this issue already as well. Let me try to reproduce this.

I so hope that this is the solution for https://github.com/pnp/powershell/issues/1949

This issue still exists in 1.11.0. See https://github.com/pnp/powershell/issues/2150


Add Tombstone format element

Summary
This PR adds a Tombstone format element. It's sole purpose is to allow for cheap "removal"
of FormatElements from an already created document by replacing them with a FormatElement that has a void semantics.

Test Plan

Current dependencies on/for this PR:

main

PR #6251   👈

PR #6252

This comment was auto-generated by Graphite.

is "Tombstone" standard nomenclature? (some other, not-great ideas: phantom, SkipRender, DeletedEntry)

Closing because I want to use a different approach to handle suppression comments (by being more restrictive where suppression comments are allowed)


[WebFramework] Version

Hello olaeld,
Do you have or are you working on a version for web based classes.
I've been trying for a bit but the compiler clearly doesn't like it that much?
Sincerely,
FrozenSource

Hello FrozenSource,
Really happy to see that you are considering using a test framework for DataFlex. DFUnit is a general purpose testing library for DataFlex, and you can use it for writing any type of test you want.
I'm not sure exactly what you are trying to do, and what kind of problems you have encountered. If you can provide a tiny sample workspace demonstrating what you are trying to do, that should make it easier to diagnoze.
You can post here on GitHub, but I actually recommend using the DataFlex forums. In any case, I will try to help you move ahead.

The problem is that the testrunner is running as a windows application. But both frameworks have functions and defines that are the same. Thus when using web components within the testrunner, it won't compile.

I see. Can you give an example of a test that fails to compile?
I guess this problem can arise if you want to test a method in a subclass of a web framework class. When you include the web framework class in the Windows project, you get a compile error because of duplicate definitions. It is possible to circumnavigate this problem by architecting the methods you want to test in such a way that they are defined in separate classes that are not inheriting from the web framework classes (and then use composition instead of inheritance in your application). This may or may not be desirable from your point of view, but it could be a possibility.
If you can provide a specific example of how this problem arises, I can hopefully provide more specific help.

Well that is exactly the problem, I'l try putting all functions etc. in another class.

Ok. I'm interested to hear how it worked out.


Short solution needed: "Convert Numpy bytes to int" (python-numpy)

Please help us write most modern and shortest code solution for this issue:
Convert Numpy bytes to int (technology: python-numpy)
Fast way
Just write the code solution in the comments.
Prefered way

Create pull request with a new code file inside inbox folder.
Don't forget to use comments to make solution explained.
Link to this issue in comments of pull request.

Convert Numpy bytes to int


[Discussion] Remove support for old compression algorithms

Expected behavior
Bzip2, FastLz, Lzma, Snappy are old and superseded by compression algorithms like Brotli, Zstd, Zlib, etc. We should consider removing them or moving them under netty-contrib.
Netty version
5.0.0.X

@idelpivnitskiy Will appreciate your opinion too.

While this is true for the web (HTTP/RPC), Bzip2 and Snappy are still actively used by Big Data related projects and data storages.
I'm agreed that FastLz and Lzma can be moved to netty-contrib, but is it worth a hassle for us to create and maintain a separate repo if it contains only 2-3 classes?

Another challenge for us will be to make public or replicate test scenarios that we reuse for all compression codecs: AbstractEncoderTest, AbstractDecoderTest, AbstractIntegrationTest.

A contrib project for legacy compression algorithms could import the abstract test classes from the main project, couldn't it?
Also what would you think about moving compression algorithms into an own module? I think this was mentioned in a contributor meetup before and also in #4024. Might be helpful for #8541 too.


If you remove bootstrap.css  reference all run ok

What is the expected behavior?
I'm asking if popper run also with bootstrap
What went wrong?
Is popper compatible with bootstrap.css ?
Any other comments?
no comment

Please consider opening an issue on the Bootstrap repository, this same codepen should be helpful to them.


[5.7-04182022][Static Mirror] Gather local type extension conformance infos correctly

Cherry-pick of https://github.com/apple/swift/pull/58982
Conformance Infos for nominal type declarations reference the conforming type by storing an offset to the address in the binary where the type's type descriptor is located. Conformance infos for conformances applied to an extension of a type use a different mechanism: they use an indirect reference to a dynamic symbol, which may be an external symbol or a resolved address to a local type descriptor. It is the latter case that the conformance-gather implementation was missing that is added in this PR.
Resolves rdar://93578419

@swift-ci please test

@swift-ci please test macOS platform

@swift-ci please test macOS platform


[BUG] Failed epicli upgrade may lead to broken repository machine (disabled system repos)

Describe the bug
Similar to #2324 but for upgrade mode.
Failed epicli upgrade may lead to broken repository machine.
How to reproduce
Steps to reproduce the behavior:

execute epicli upgrade command
let's assume that upgrade of any component fails
the current status on the repository machine is that epirepo is enabled and system repositories are disabled
re-run upgrade after about 2 hours (download_done_flag_expire_minutes is 120 by default) or manually remove download-requirements-done.flag file and re-run upgrade
it fails on task TASK [repository : Run download-requirements script, this can take a long time because all packages are removed from epirepo and no system repositories are available.

Downloading packages:
http://<IP_ADDRESS>/epirepo/packages/wget-1.14-18.el7_6.1.x86_64.rpm: [Errno 14] HTTP Error 404 - Not Found
Trying other mirror.
To address this issue please refer to the below knowledge base article

https://access.redhat.com/articles/1320623

If above article doesn't help to resolve this issue please open a ticket with Red Hat Support.

Error downloading packages:
  wget-1.14-18.el7_6.1.x86_64: [Errno 256] No more mirrors to try.

Full output:
2021-10-20T11:23:51.6752845Z11:23:51 INFO cli.engine.ansible.AnsibleCommand - TASK [repository : Check if flag file exists] **********************************
2021-10-20T11:23:53.3749257Z11:23:53 INFO cli.engine.ansible.AnsibleCommand - ok: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:23:53.9511762Z11:23:53 INFO cli.engine.ansible.AnsibleCommand -
2021-10-20T11:23:53.9512849Z11:23:53 INFO cli.engine.ansible.AnsibleCommand - TASK [repository : Remove download-requirements-done.flag file if expired] *****
2021-10-20T11:23:55.3052185Z11:23:55 INFO cli.engine.ansible.AnsibleCommand - changed: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:23:55.6872357Z11:23:55 INFO cli.engine.ansible.AnsibleCommand -
2021-10-20T11:23:55.6877932Z11:23:55 INFO cli.engine.ansible.AnsibleCommand - TASK [repository : Check whether to run download script] ***********************
2021-10-20T11:23:56.7045834Z11:23:56 INFO cli.engine.ansible.AnsibleCommand - ok: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:23:56.7992855Z11:23:56 INFO cli.engine.ansible.AnsibleCommand -
2021-10-20T11:23:56.7998737Z11:23:56 INFO cli.engine.ansible.AnsibleCommand - TASK [repository : include_tasks] **********************************************
2021-10-20T11:23:57.4211641Z11:23:57 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:23:57.6496915Z11:23:57 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:23:57.8248110Z11:23:57 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:23:58.0728224Z11:23:58 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:23:58.2406036Z11:23:58 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:23:58.2502871Z11:23:58 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:23:58.6031522Z11:23:58 INFO cli.engine.ansible.AnsibleCommand - included: /shared/build/10todevawrhcalico/ansible/roles/repository/tasks/clean-up-epirepo.yml for ec2-x-x-x-x.eu-west-3.compute.amazonaws.com
2021-10-20T11:23:59.1468367Z11:23:59 INFO cli.engine.ansible.AnsibleCommand -
2021-10-20T11:23:59.1475151Z11:23:59 INFO cli.engine.ansible.AnsibleCommand - TASK [repository : Load vars from old manifest] ********************************
2021-10-20T11:23:59.5609101Z11:23:59 INFO cli.engine.ansible.AnsibleCommand - ok: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:24:00.1428641Z11:24:00 INFO cli.engine.ansible.AnsibleCommand -
2021-10-20T11:24:00.1429711Z11:24:00 INFO cli.engine.ansible.AnsibleCommand - TASK [repository : Set fact on epirepo directory path] *************************
2021-10-20T11:24:00.5539548Z11:24:00 INFO cli.engine.ansible.AnsibleCommand - ok: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:24:00.9188694Z11:24:00 INFO cli.engine.ansible.AnsibleCommand -
2021-10-20T11:24:00.9189806Z11:24:00 INFO cli.engine.ansible.AnsibleCommand - TASK [repository : Get list of files in epirepo] *******************************
2021-10-20T11:24:03.5179008Z11:24:03 INFO cli.engine.ansible.AnsibleCommand - ok: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com] => (item=/var/www/html/epirepo/files)
2021-10-20T11:24:04.4136533Z11:24:04 INFO cli.engine.ansible.AnsibleCommand - ok: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com] => (item=/var/www/html/epirepo/images)
2021-10-20T11:24:04.9000904Z11:24:04 INFO cli.engine.ansible.AnsibleCommand -
2021-10-20T11:24:04.9001933Z11:24:04 INFO cli.engine.ansible.AnsibleCommand - TASK [repository : Remove old files from epirepo] ******************************
2021-10-20T11:24:05.9252449Z11:24:05 INFO cli.engine.ansible.AnsibleCommand - changed: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com] => (item=erlang-23.1.5-1.el7.x86_64.rpm.1)
2021-10-20T11:24:06.6530781Z11:24:06 INFO cli.engine.ansible.AnsibleCommand - changed: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com] => (item=haproxy_exporter-0.10.0.linux-amd64.tar.gz.1)
2021-10-20T11:24:07.4233549Z11:24:07 INFO cli.engine.ansible.AnsibleCommand - changed: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com] => (item=jmx_prometheus_javaagent-0.14.0.jar.1)
2021-10-20T11:24:08.1382537Z11:24:08 INFO cli.engine.ansible.AnsibleCommand - changed: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com] => (item=kafka_exporter-1.2.0.linux-amd64.tar.gz.1)
2021-10-20T11:24:08.9162020Z11:24:08 INFO cli.engine.ansible.AnsibleCommand - changed: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com] => (item=node_exporter-1.0.1.linux-amd64.tar.gz.1)
2021-10-20T11:24:09.6737502Z11:24:09 INFO cli.engine.ansible.AnsibleCommand - changed: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com] => (item=kafka_2.12-2.6.0.tgz.1)
2021-10-20T11:24:10.5943377Z11:24:10 INFO cli.engine.ansible.AnsibleCommand - changed: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com] => (item=prometheus-2.10.0.linux-amd64.tar.gz.1)
2021-10-20T11:24:11.3669825Z11:24:11 INFO cli.engine.ansible.AnsibleCommand - changed: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com] => (item=alertmanager-0.17.0.linux-amd64.tar.gz.1)
2021-10-20T11:24:12.1199090Z11:24:12 INFO cli.engine.ansible.AnsibleCommand - changed: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com] => (item=apache-zookeeper-3.5.8-bin.tar.gz.1)
2021-10-20T11:24:12.8965086Z11:24:12 INFO cli.engine.ansible.AnsibleCommand - changed: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com] => (item=apache-ignite-2.9.1-bin.zip.1)
2021-10-20T11:24:13.6180906Z11:24:13 INFO cli.engine.ansible.AnsibleCommand - changed: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com] => (item=helm-v3.2.0-linux-amd64.tar.gz.1)
2021-10-20T11:24:14.4009220Z11:24:14 INFO cli.engine.ansible.AnsibleCommand - changed: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com] => (item=filebeat-7.9.2.tgz.1)
2021-10-20T11:24:15.3439970Z11:24:15 INFO cli.engine.ansible.AnsibleCommand - changed: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com] => (item=node-exporter-1.1.2.tgz.1)
2021-10-20T11:24:15.6362399Z11:24:15 INFO cli.engine.ansible.AnsibleCommand -
2021-10-20T11:24:15.6374756Z11:24:15 INFO cli.engine.ansible.AnsibleCommand - TASK [repository : Load variables from "image_registry" role] ******************
2021-10-20T11:24:15.7118387Z11:24:15 INFO cli.engine.ansible.AnsibleCommand - ok: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:24:15.9927643Z11:24:15 INFO cli.engine.ansible.AnsibleCommand -
2021-10-20T11:24:15.9940625Z11:24:15 INFO cli.engine.ansible.AnsibleCommand - TASK [repository : Remove old images from epirepo] *****************************
2021-10-20T11:24:16.9429394Z11:24:16 INFO cli.engine.ansible.AnsibleCommand - changed: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com] => (item=keycloak-14.0.0.tar)
2021-10-20T11:24:17.6519485Z11:24:17 INFO cli.engine.ansible.AnsibleCommand - changed: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com] => (item=ignite-2.9.1.tar)
2021-10-20T11:24:17.8522107Z11:24:17 INFO cli.engine.ansible.AnsibleCommand -
2021-10-20T11:24:17.8527693Z11:24:17 INFO cli.engine.ansible.AnsibleCommand - TASK [repository : Remove all packages from epirepo] ***************************
2021-10-20T11:24:18.7296798Z11:24:18 INFO cli.engine.ansible.AnsibleCommand - changed: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:24:18.9887044Z11:24:18 INFO cli.engine.ansible.AnsibleCommand -
2021-10-20T11:24:18.9894150Z11:24:18 INFO cli.engine.ansible.AnsibleCommand - TASK [repository : Find leftovers in /tmp] *************************************
2021-10-20T11:24:19.7561743Z11:24:19 INFO cli.engine.ansible.AnsibleCommand - ok: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:24:20.1329898Z11:24:20 INFO cli.engine.ansible.AnsibleCommand -
2021-10-20T11:24:20.1339387Z11:24:20 INFO cli.engine.ansible.AnsibleCommand - TASK [repository : Remove leftovers from /tmp] *********************************
2021-10-20T11:24:20.4187329Z11:24:20 INFO cli.engine.ansible.AnsibleCommand -
2021-10-20T11:24:20.4283503Z11:24:20 INFO cli.engine.ansible.AnsibleCommand - TASK [repository : Copy requirements for offline installation to repository host, this can take a long time
2021-10-20T11:24:20.4287061Z11:24:20 INFO cli.engine.ansible.AnsibleCommand - Destination: /var/www/html/epirepo] ***
2021-10-20T11:24:20.5071704Z11:24:20 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:24:20.6525039Z11:24:20 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:24:20.8132181Z11:24:20 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:24:20.8487784Z11:24:20 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:24:20.9206402Z11:24:20 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:24:20.9252742Z11:24:20 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-3x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:24:20.9813525Z11:24:20 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:24:21.0218593Z11:24:21 INFO cli.engine.ansible.AnsibleCommand -
2021-10-20T11:24:21.0219757Z11:24:21 INFO cli.engine.ansible.AnsibleCommand - TASK [repository : Download Epiphany requirements] *****************************
2021-10-20T11:24:21.1548431Z11:24:21 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:24:21.2618454Z11:24:21 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:24:21.3154956Z11:24:21 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:24:21.4315862Z11:24:21 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:24:21.4330118Z11:24:21 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:24:21.4571771Z11:24:21 INFO cli.engine.ansible.AnsibleCommand - skipping: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]
2021-10-20T11:24:21.6093528Z11:24:21 INFO cli.engine.ansible.AnsibleCommand - included: /shared/build/10todevawrhcalico/ansible/roles/repository/tasks/download-requirements.yml for ec2-x-x-x-x.eu-west-3.compute.amazonaws.com
2021-10-20T11:24:21.8429730Z11:24:21 INFO cli.engine.ansible.AnsibleCommand -
2021-10-20T11:24:21.8430772Z11:24:21 INFO cli.engine.ansible.AnsibleCommand - TASK [repository : Run download-requirements script, this can take a long time
2021-10-20T11:24:21.8432245Z11:24:21 INFO cli.engine.ansible.AnsibleCommand - You can check progress on repository host with: journalctl -f -t download-requirements.sh] ***
2021-10-20T11:24:24.9334287Z11:24:24 ERROR cli.engine.ansible.AnsibleCommand - fatal: [ec2-x-x-x-x.eu-west-3.compute.amazonaws.com]: FAILED! => {"changed": true, "cmd": "set -o pipefail && /tmp/epi-download-requirements/download-requirements.sh /var/www/html/epirepo --no-logfile |& tee >(systemd-cat --identifier=download-requirements.sh)", "delta": "0:00:01.987291", "end": "2021-10-20 11:24:24.838161", "msg": "non-zero return code", "rc": 1, "start": "2021-10-20 11:24:22.850870", "stderr": "", "stderr_lines": [], "stdout": "Detected arch: x86_64
Docker platform: linux/amd64
PID is: 1782, creating file: /var/run/download-requirements.sh.pid
Testing yum connection:
- epirepo...
Connection successful.
Backuping /etc/yum.repos.d/ to /tmp/epi-download-requirements/download-requirements.sh-yum-repos-backup-tmp-do-not-remove.tar
tar: Removing leading `/' from member names
/etc/yum.repos.d/
/etc/yum.repos.d/redhat-rhui-beta.repo.disabled
/etc/yum.repos.d/redhat-rhui-client-config.repo
/etc/yum.repos.d/redhat-rhui.repo
/etc/yum.repos.d/epirepo.repo
tar: Archive contains file names with leading prefixes removed.
tar: Verification may fail to locate original files.
Verify etc/yum.repos.d/
Verify etc/yum.repos.d/redhat-rhui-beta.repo.disabled
Verify etc/yum.repos.d/redhat-rhui-client-config.repo
Verify etc/yum.repos.d/redhat-rhui.repo
Verify etc/yum.repos.d/epirepo.repo
Backup done
Package yum-utils already installed
Installing package: wget
Loaded plugins: amazon-id, search-disabled-repos, versionlock
Resolving Dependencies
--> Running transaction check
---> Package wget.x86_64 0:1.14-18.el7_6.1 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package       Arch            Version                   Repository        Size
================================================================================
Installing:
 wget          x86_64          1.14-18.el7_6.1           epirepo          547 k

Transaction Summary
================================================================================
Install  1 Package

Total download size: 547 k
Installed size: 2.0 M
Downloading packages:
http://<IP_ADDRESS>/epirepo/packages/wget-1.14-18.el7_6.1.x86_64.rpm: [Errno 14] HTTP Error 404 - Not Found
Trying other mirror.
To address this issue please refer to the below knowledge base article

https://access.redhat.com/articles/1320623

If above article doesn't help to resolve this issue please open a ticket with Red Hat Support.

Error downloading packages:
  wget-1.14-18.el7_6.1.x86_64: [Errno 256] No more mirrors to try.

Command failed: yum install -y \"wget\"", "stdout_lines": ["Detected arch: x86_64", "Docker platform: linux/amd64", "PID is: 1782, creating file: /var/run/download-requirements.sh.pid", "Testing yum connection:", "- epirepo...", "Connection successful.", "Backuping /etc/yum.repos.d/ to /tmp/epi-download-requirements/download-requirements.sh-yum-repos-backup-tmp-do-not-remove.tar", "tar: Removing leading `/' from member names", "/etc/yum.repos.d/", "/etc/yum.repos.d/redhat-rhui-beta.repo.disabled", "/etc/yum.repos.d/redhat-rhui-client-config.repo", "/etc/yum.repos.d/redhat-rhui.repo", "/etc/yum.repos.d/epirepo.repo", "tar: Archive contains file names with leading prefixes removed.", "tar: Verification may fail to locate original files.", "Verify etc/yum.repos.d/", "Verify etc/yum.repos.d/redhat-rhui-beta.repo.disabled", "Verify etc/yum.repos.d/redhat-rhui-client-config.repo", "Verify etc/yum.repos.d/redhat-rhui.repo", "Verify etc/yum.repos.d/epirepo.repo", "Backup done", "Package yum-utils already installed", "Installing package: wget", "Loaded plugins: amazon-id, search-disabled-repos, versionlock", "Resolving Dependencies", "--> Running transaction check", "---> Package wget.x86_64 0:1.14-18.el7_6.1 will be installed", "--> Finished Dependency Resolution", "", "Dependencies Resolved", "", "================================================================================", " Package       Arch            Version                   Repository        Size", "================================================================================", "Installing:", " wget          x86_64          1.14-18.el7_6.1           epirepo          547 k", "", "Transaction Summary", "================================================================================", "Install  1 Package", "", "Total download size: 547 k", "Installed size: 2.0 M", "Downloading packages:", "http://<IP_ADDRESS>/epirepo/packages/wget-1.14-18.el7_6.1.x86_64.rpm: [Errno 14] HTTP Error 404 - Not Found", "Trying other mirror.", "To address this issue please refer to the below knowledge base article ", "", "https://access.redhat.com/articles/1320623", "", "If above article doesn't help to resolve this issue please open a ticket with Red Hat Support.", "", "", "", "Error downloading packages:", "  wget-1.14-18.el7_6.1.x86_64: [Errno 256] No more mirrors to try.", "", "Command failed: yum install -y \"wget\""]}

To solve this I had to login to the repository vm and execute there:
sudo /tmp/epi-repository-setup-scripts/disable-epirepo-client.sh && sudo /tmp/epi-repository-setup-scripts/enable-system-repos.sh.
Expected behavior
Re-run should succeed at least when it comes to configuring the repository vm.
Config files
If applicable, add config files to help explain your problem.
Environment

Cloud provider: [all]
OS: [RHEL]

epicli version: [1.3.0dev]
Additional context
Add any other context about the problem here.

DoD checklist

[ ] Changelog updated (if affected version was released)
[ ] COMPONENTS.md updated / doesn't need to be updated
[ ] Automated tests passed (QA pipelines)

[ ] apply
[ ] upgrade

[ ] Case covered by automated test (if possible)
[ ] Idempotency tested
[ ] Documentation updated / doesn't need to be updated
[ ] All conversations in PR resolved
[ ] Backport tasks created / doesn't need to be backported

Since this issue doesn't occur on Ubuntu we may check and compare logic with the scripts for Ubuntu.

We may also change paths from /tmp/epi-repository-setup-scripts to /var/tmp/epi-repository-setup-scripts and from /tmp/epi-download-requirements to /var/tmp/epi-download-requirements in order to preserve files after OS restart.
They are removed in teardown playbook (task Clean up temporary files).

A similar problem also occurs on Ubuntu:
2021-11-25T12:45:18.9960903Z 12:45:18 INFO cli.engine.ansible.AnsibleCommand - TASK [repository : Run download-requirements script, this can take a long time
2021-11-25T12:45:18.9962164Z 12:45:18 INFO cli.engine.ansible.AnsibleCommand - You can check progress on repository host with: journalctl -f -t download-requirements.sh] ***
2021-11-25T12:45:25.8500190Z 12:45:25 ERROR cli.engine.ansible.AnsibleCommand - fatal: [ec2-1-1-1-1.eu-west-1.compute.amazonaws.com]: FAILED! => {"changed": true, "cmd": "set -o pipefail && /tmp/epi-download-requirements/download-requirements.sh /var/www/html/epirepo --no-logfile |& tee >(systemd-cat --identifier=download-requirements.sh)", "delta": "0:00:06.139469", "end": "2021-11-25 12:45:25.784458", "msg": "non-zero return code", "rc": 1, "start": "2021-11-25 12:45:19.644989", "stderr": "", "stderr_lines": [], "stdout": "Detected arch: x86_64\nDocker platform: linux/amd64\nOS repositories seems missing, restoring...\n/etc/apt/sources.list\n/etc/apt/sources.list.d/\nTesting apt connection:\n- /etc/apt/sources.list...\nConnection failure, reason: [ WARNING: apt does not have a stable CLI interface. Use with caution in scripts. E: Failed to fetch http://<IP_ADDRESS>/epirepo/packages/./Packages 404 Not Found [IP: <IP_ADDRESS> 80] E: Some index files failed to download. They have been ignored, or old ones used instead.]", "stdout_lines": ["Detected arch: x86_64", "Docker platform: linux/amd64", "OS repositories seems missing, restoring...", "/etc/apt/sources.list", "/etc/apt/sources.list.d/", "Testing apt connection:", "- /etc/apt/sources.list...", "Connection failure, reason: [ WARNING: apt does not have a stable CLI interface. Use with caution in scripts. E: Failed to fetch http://<IP_ADDRESS>/epirepo/packages/./Packages 404 Not Found [IP: <IP_ADDRESS> 80] E: Some index files failed to download. They have been ignored, or old ones used instead.]"]}

:heavy_check_mark: The apply and upgrade commands have been tested many times after the previous build was successfully completed and also on failure. No issues were noticed with the repository on re-run.