issue23: Fix getitem and setitem signatures

Modified the declarations to match real class use.
Added tests to verify that the stub allows for real use-cases.
Disabled one declaration from mypy check during internal CI verification due to an mypy bug.

@bzoracler A bit of description of how I test during development and in the CI.
I'm using tox to test multiple major Python versions at the same time. It does end-to-end testing. It's configured in the tox.ini file.
Tox performs two kind of tests while not having installed Pandas at all:

it runs mypy on the stubs themselves checking for consistency
it runs mypy on the snippets with stubs installed in the env checking if all the declarations are legal.

There's one type of testing that isn't included in tox. It's actually running the snippets. Mypy doesn't do that, it just analyzes the types. I test the snippet code by running it manually.
Seeing how I've made a mistake, it's probably a good idea to also automate this type of tests.
I'll add a description of this whole testing procedure, so it's easier to contribute.
I've fixed the PR. You're welcome to try testing it now by yourself.


Add support for BigAutoField

Hi @shimizukawa!
We've upgraded for Django==1.10 and there is a new interesting field that we want to use with Redshift: BigAutoField. You'll see the PR is pretty straightforward. I couldn't add tests because I don't want to break previous versions of Django that don't support the BigAutoField.
Cheers!

Thanks! I'll take a look.

BigAutoField is introduced by https://github.com/django/django/commit/2a7ce34600d0f879e93c9a5e02215948ed3bb6ac

LGTM!

that was pretty quick! thanks! When's the next pip release? 🍾

Released! 🍾
https://pypi.python.org/pypi/django-redshift-backend/0.5


BUG: add infeasibility checks to min_weight_full_bipartite_matching

The algorithm is already equipped with a handful of checks for problem infeasibility, but the tests added here show two different examples that were not caught by any of those checks. It is likely possible to dive in and determine infeasibility in _lapjvsp itself, as in both examples, the symptom is the algorithm hitting an infinite loop (but a different infinite loop for each case), but here we simply add the catch-all fix of checking that the problem is feasible before even trying to solve it. This does come with a performance hit, but at least the implementation of Hopcroft--Karp that we're using is fast.
Reference issue
closes gh-17269

CI test failures seem unrelated to the change.

Thanks @fuglede, and @rossbar for some detailed analysis in the matching issue. If either of you see issues after this merge feel free to ping me directly--better to fix things up sooner than at release time.

Thanks for diving in, @tylerjereddy!


Beaker Specification Draft

Contains more formally defined behaviour of the kernel. This is intended to address #112. Currently it is in Latex due to the large number of tables and regular use of mathematical symbols.

One big ticket item not yet added from #112 is the registration rules, that's the next task.

Looks awesome so far! 💯

This might be a natural point to merge if you're content with those changes.

This is awesome!!! Looks great! I have a couple of comments left, but other than that let's merge.


feat: determine platform details from local docker installation for jibDockerBuild

This code has been tested manually  with jib/examples/helloworld/.
Notes:

If manifest list found then the log picks the image that contains the same os and architecture as the local docker environment (derived through docker info). Otherwise, it throws an error when no matching os and architecture are found.
For example, if the platform configurations are as follows then the image with os of linux and architecture of arm64 will be pushed to the docker engine:

<platforms>
                <platform>
                  <architecture>arm64</architecture>
                  <os>linux</os>
                </platform>
                <platform>
                  <architecture>amd64</architecture>
                  <os>linux</os>
                </platform>
 </platforms>

I think this feature does not work with podman, did you test or consider this?
When using podman the docker info command differs :-/. It returns
{
  "host": {
    "arch": "amd64",
    "buildahVersion": "1.23.0",
    "cgroupManager": "systemd",
    "cgroupVersion": "v2",
    "cgroupControllers": [],

Please also see: https://docs.podman.io/en/latest/markdown/podman-info.1.html
So determining os and architecture fails and you get "The configured platforms don't match the Docker Engine's OS and architecture (/). It would be nice if we could support podman as well.

In addition this feature is also enabled when having just one single platform specified.
Even when specifying
<platform>
  <architecture>amd64</architecture>
  <os>linux</os>
</platform>

there is no need for this new feature. But it kicks in and fails the build. How can i disable this feature?
This leaves Jib is no longer usable with podman :-/

@profhenry #4265


code signature invalid for '/MacOS/plugins/libmspack.dylib'

Binary Ninja Version: 1.1.922, 4d40ebb7
Platform: OS X Yosemite (10.10)
error log:
Plugin module '/Applications/Binary Ninja.app/Contents/MacOS/plugins/libmspack.dylib' failed to load
dlerror() reports: dlopen(/Applications/Binary Ninja.app/Contents/MacOS/plugins/libmspack.dylib, 5): no suitable image found.  Did find:
/Applications/Binary Ninja.app/Contents/MacOS/plugins/libmspack.dylib: code signature invalid for '/Applications/Binary Ninja.app/Contents/MacOS/plugins/libmspack.dylib'

One of two things -- either a corrupted download (try download a fresh copy from https://binary.ninja/recover/  and reinstalling) or it's a problem with the old version of OS X (which we don't test on).
In the past, there was a problem running older OS X versions because of some linking issues, but that's not what this sounds like. Still, if you try running otool -L  on the libbinaryninjacore.dylib and look for any errors that might help.
That said,

No luck with fresh install.  Ill update OS X later.
Thanks

Updating to OS X 10.13 fixed this issue


Error at synthesis stage.

Description
Error at the synthesis stage. Error is shown in a code generated by sandpiper-saas tool which converts the TL-verilog into verilog.
Expected Behavior
Successful completion of synthesis stage.
Environment report
% python3 ./env.py issue-survey

WARNING: issue-survey appears to be running inside the OpenLane

container.

This makes it difficult to rule out issues with your

environment.

Unless instructed specifically to do so, please run this command

outside the OpenLane container.

---

Kernel: Linux v6.2.0-32-generic

Distribution: centos 7

Python: v3.6.8 (OK)

OpenLane Git Version: 00caae2b3186bb6b24a0b7db7ee1a6f056ad1702

pip: INSTALLED

python-venv: INSTALLED

---

PDK Version Verification Status: OK

---

Git Log (Last 3 Commits)

00caae2b 2023-08-22T19:35:09+03:00 Update PDK + Restore `RC_LAYER` variables (#1932) - Kareem Farid -  (HEAD -> master, tag: 2023.08.23, origin/master, origin/HEAD)

eaa6680c 2023-08-22T19:12:10+03:00 `run_designs.py`: Fix `mkdirp` not being called in some scenarios (#1945) - UMW -  ()

17559fa5 2023-08-22T18:54:25+03:00 CI Tweaks (#1951) - Mohamed Gaber -  ()

---

Git Remotes

origin	https://github.com/The-OpenROAD-Project/OpenLane (fetch)

origin	https://github.com/The-OpenROAD-Project/OpenLane (push)

%

Reproduction material
issue_synth.tar.gz
Relevant log output
/----------------------------------------------------------------------------\

 |                                                                            |

 |  yosys -- Yosys Open SYnthesis Suite                                       |

 |                                                                            |

 |  Copyright (C) 2012 - 2020  Claire Xenia Wolf<EMAIL_ADDRESS>       |

 |                                                                            |

 |  Permission to use, copy, modify, and/or distribute this software for any  |

 |  purpose with or without fee is hereby granted, provided that the above    |

 |  copyright notice and this permission notice appear in all copies.         |

 |                                                                            |

 |  THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES  |

 |  WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF          |

 |  MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR   |

 |  ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES    |

 |  WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN     |

 |  ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF   |

 |  OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.            |

 |                                                                            |

 \----------------------------------------------------------------------------/

 Yosys 0.30+48 (git sha1 14d50a176d5, gcc 8.3.1 -fPIC -Os)

[TCL: yosys -import] Command name collision: found pre-existing command `cd' -> skip.

[TCL: yosys -import] Command name collision: found pre-existing command `eval' -> skip.

[TCL: yosys -import] Command name collision: found pre-existing command `exec' -> skip.

[TCL: yosys -import] Command name collision: found pre-existing command `read' -> skip.

[TCL: yosys -import] Command name collision: found pre-existing command `trace' -> skip.

1. Executing Liberty frontend: /openlane/designs/vsdmemsoc/src/macros/sky130_sram_1kbyte_1rw1r_32x256_8_TT_1p8V_25C.lib

Imported 1 cell types from liberty file.

2. Executing Verilog-2005 frontend: /openlane/designs/vsdmemsoc/src/rvmyth.v

Parsing SystemVerilog input from `/openlane/designs/vsdmemsoc/src/rvmyth.v' to AST representation.

Generating RTLIL representation for module `\rvmyth'.

Warning: Replacing memory \CPU_Xreg_value_a5 with list of registers. See /openlane/designs/vsdmemsoc/src/rvmyth_gen.v:676

Warning: Replacing memory \CPU_Xreg_value_a4 with list of registers. See /openlane/designs/vsdmemsoc/src/rvmyth_gen.v:675

Warning: Replacing memory \CPU_Dmem_value_a5 with list of registers. See /openlane/designs/vsdmemsoc/src/rvmyth_gen.v:666

Successfully finished Verilog frontend.

3. Executing Verilog-2005 frontend: /openlane/designs/vsdmemsoc/src/rvmyth_gen.v

/openlane/designs/vsdmemsoc/src/rvmyth_gen.v:12: ERROR: syntax error, unexpected TOK_GENVAR

Found the bug. Resolved.

how was the error resolved?

If i remember correctly, this error was due to sandpiper-saas tool; there should be an issue now.

faced similar issue still, can you please tell me what you did?


wine{Unstable,Staging}: 6.14 -> 6.15

Motivation for this change
Upstream update.
Things done
Verified that the Windows software I use still works.
$ NIXPKGS_ALLOW_UNFREE=1 nixpkgs-review rev wine-6.15
[...]
14 packages updated:
grapejuice lutris lutris lutris-original pipelight wineUnstable (6.14 → 6.15) winePackages.unstable (6.14 → 6.15) wine-staging (6.14-staging → 6.15-staging) winePackages.staging (6.14-staging → 6.15-staging) wineStaging (6.14-staging → 6.15-staging) wineWowPackages.unstable (6.14 → 6.15) wineWowPackages.staging (6.14-staging → 6.15-staging) yabridge yabridgectl
[...]
12 packages built:
grapejuice lutris lutris-free lutris-unwrapped pipelight wineStaging winePackages.staging wineUnstable wineWowPackages.staging wineWowPackages.unstable yabridge yabridgectl

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
Result of nixpkgs-review pr 134137 run on x86_64-linux 1

12 packages built:

grapejuice
lutris
lutris-free
lutris-unwrapped
pipelight
wine-staging
winePackages.staging
winePackages.unstable
wineWowPackages.staging
wineWowPackages.unstable
yabridge
yabridgectl


feat(dataframe): copy the items before putting them into "ArraySeries"

Task linked: CU-2pkhncg ArraySeries 对更新的数据进行复制


What permissions are needed for Athena queries?

In the documentation for making read_sql_query requests, it says that if you choose to use ctas_approach=True it requires create/delete table permissions on Glue. If I also set ctas_database_name, why do I still need permissions on the database?
For example:
df = wr.athena.read_sql_query(
            SELECT * FROM my_table LIMIT 10;",
            database="temp1",
            ctas_database_name="temp2",
            boto3_session=session,
        )

is erroring unless I give my role permission to CreateTable and DeleteTable on 'temp1'. Can you provide a list of iam and LakeFormation permissions or more verbose examples of how to use "ctas_database_name"?
I want wrangler to have no access to create/delete tables in my main db but full access to do whatever it wants in the ctas_database_name database for any temporary tables

Thank you for raising this @andy-han, I believe it's a bug on our end. Specifically this line where the temporary table is deleted once the submitted query has completed. At the moment it always attempts to delete a table in the original database, regardless of whether a ctas_database_name parameter was provided or not. Modifying that line to:
catalog.delete_table_if_exists(database=ctas_database_name or database, table=name, boto3_session=boto3_session)

should solve the issue. I will raise a PR to mitigate this.

This has now been merged to main and can be tested:
pip uninstall awswrangler -y
pip install git+https://github.com/awslabs/aws-data-wrangler.git@main

until it's available in the next release.

Thanks!


Maitnence objects suddenly appearing in hand breaks immersion

I want to change it so the object appears in your inventory instead of your hand

Code is very weirdly set up, and hard to read. If i were to change this i would have to rewrite the code entirely.


⚠ Bofrost.at has degraded performance

In 329bc00, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 863 ms

Resolved: Bofrost.at performance has improved in f6579bd after 21 minutes.


Update prerequisites and setup scripts

Update macOS prerequisites installation script.
Update setup scripts.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: hklai
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [hklai]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Bug 2090048: Boot mode should be editable while VM is running

📝 Description
Enabling boot mode edit when VM is in Running status
🎥 Demo
before:

after:
https://user-images.githubusercontent.com/67270715/171174907-500d63c0-9d3e-44f4-bdf0-00547b27f662.mp4
Signed-off-by: Aviv Turgeman<EMAIL_ADDRESS>

/retest

/hold
you also need to show the "restart needed" thing that we show when we edit something while running :-(

/hold cancel
/lgtm

/hold cancel

/lgtm


Explicitly control when reallocation happens in bitset

cc @robskillington @kobolog @martin-mao @cw9
This PR addresses an inefficiency in how we use bitset in commit logs. The current bitset implementation keeps track of the maximum value it has seen so far and triggers a reallocation if it needs to set a value larger the current max. However it only reallocates space enough to represent the new value, which means in case each reallocation only grows the internal slice the bitset implementation by 1 slot and copies the existing over, causing significant CPU overhead and GC time. As a result, we explicitly control when the reallocation happens and grow the slice explicitly to amortize the allocation cost.

Coverage increased (+0.03%) to 78.305% when pulling c0f9fb65ecb6637bd0273f62ccbfa3626d8a467f on xichen-optimize-bitset into b98352433bb1b3bb1b6c9b57b5f5e99ebf876ebb on dev.


sync code from dev-jar into dev

Codecov Report

Merging #126 into dev will increase coverage by 7.74%.
The diff coverage is 50.00%.

@@             Coverage Diff              @@
##                dev     #126      +/-   ##
============================================
+ Coverage     16.03%   23.77%   +7.74%
+ Complexity      793      166     -627
============================================
  Files           452       76     -376
  Lines         15296     2805   -12491
  Branches       1629      288    -1341
============================================
- Hits           2453      667    -1786
+ Misses        12601     2040   -10561
+ Partials        242       98     -144

Impacted Files
Coverage Δ
Complexity Δ

...op/src/main/java/org/fisco/bcos/sdk/amop/Amop.java
66.66% <ø> (ø)
1.00 <0.00> (?)

...ain/java/org/fisco/bcos/sdk/amop/AmopCallback.java
100.00% <ø> (ø)
1.00 <0.00> (?)

...src/main/java/org/fisco/bcos/sdk/amop/AmopImp.java
0.00% <0.00%> (ø)
0.00 <0.00> (?)

.../main/java/org/fisco/bcos/sdk/amop/AmopMsgOut.java
0.00% <ø> (ø)
0.00 <0.00> (?)

...g/fisco/bcos/sdk/amop/exception/AmopException.java
0.00% <ø> (ø)
0.00 <0.00> (?)

.../java/org/fisco/bcos/sdk/amop/topic/AmopMsgIn.java
80.95% <ø> (ø)
10.00 <0.00> (?)

...a/org/fisco/bcos/sdk/amop/topic/AmopRespError.java
0.00% <ø> (ø)
0.00 <0.00> (?)

...g/fisco/bcos/sdk/amop/topic/RequestVerifyData.java
100.00% <ø> (ø)
7.00 <0.00> (?)

.../java/org/fisco/bcos/sdk/amop/topic/TopicType.java
0.00% <ø> (ø)
0.00 <0.00> (?)

...g/fisco/bcos/sdk/amop/topic/UpdateTopicStatus.java
100.00% <ø> (ø)
7.00 <0.00> (?)

... and 256 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d18c2a8...129e3b5. Read the comment docs.


Not installing, python 3.6

Do you want issuse or pull requests?  The Readme should be updated to inform users pyinstaller doesn't support python 3.6 and latest version they can install is 3.5.3 This is what version I had to installed to get it running and it can't be built with the latest python. If like me they don't have python installed will just grab the latest version and will be stumped what it doesn't build.
Prerequisites ( Update to include python is needed)
Python installed on your windows PC, Tekken Bot is developed on Python 3.4.3, this is the recommend version to install. Please not Pyinstaller doesn't support Python 3.6 currently the highest release version you can install is Python 3.5.3
Installation
You will need to have a version of Python installed on your Windows pc, download the python installer for Python 3.4.3 from www.python.org You will have to navigate to the full windows download list and scroll down to find the corect version. Follow the steps in the installer and ensure you check add to Path.  once Python is installed run the cmd prompt (Windows KEY + R, then type in "cmd"). Once cmd is up and running installed pyinstaller by typing in pip install pyinstaller.  Once that is done close the cmd window navigate to the Tekken Bot folder and run the windows bat file "build_project.bat"

If you have python installed, you should be able to just run 'python GUI_FrameDataOverlay.py' to launch the overlay.
I'll add a note in the readme that the distributable is built with 3.4.3.


Skip the TokenStream overhead when indexing simple keywords.

Indexing simple keywords through a TokenStream abstraction introduces a bit of overhead due to attribute management. Not much, but indexing keywords boils down to adding to a hash map and appending to a postings list, which is quite cheap too so even some low overhead can significantly impact indexing speed.
The way that this change works is by making IndexingChain check binaryValue() when a field is indexed but tokenStream() returns null. Then KeywordField only has to return null in tokenStream() to take advantage of this optimization. I hesitated doing the same with StringField but wondered if this could be breaking to some users who might pull a TokenStream themselves.

I ran this made-up benchmark to try to assess the benefits of the change. It's not representative of a real-world scenario since it disables merging (to reduce noise), but it still indexes a combination of terms plus doc values and includes flush times so it includes more than just keyword indexing.
	public static void main(String[] args) throws IOException {
	  Directory dir = FSDirectory.open(Paths.get("/tmp/a"));
	  for (int iter = 0; iter < 100; ++iter) {
	    IndexWriterConfig cfg = new IndexWriterConfig(null)
	        .setOpenMode(OpenMode.CREATE)
	        .setMergePolicy(NoMergePolicy.INSTANCE)
	        .setMaxBufferedDocs(200_000)
	        .setRAMBufferSizeMB(IndexWriterConfig.DISABLE_AUTO_FLUSH);
	    long start = System.nanoTime();
	    try (IndexWriter w = new IndexWriter(dir, cfg)) {
	      Document doc = new Document();
	      KeywordField field1 = new KeywordField("field1", new BytesRef(1), Field.Store.NO);
	      doc.add(field1);
	      KeywordField field2 = new KeywordField("field2", new BytesRef(1), Field.Store.NO);
	      doc.add(field2);
	      KeywordField field3 = new KeywordField("field3", new BytesRef(1), Field.Store.NO);
	      doc.add(field3);
	      for (int i = 0; i < 10_000_000; ++i) {
	        field1.binaryValue().bytes[0] = (byte) i;
	        field2.binaryValue().bytes[0] = (byte) (3 * i);
	        field3.binaryValue().bytes[0] = (byte) (5 * i);
	        w.addDocument(doc);
	      }
	    }
	    long end = System.nanoTime();
	    System.out.println((end - start) / 1_000_000 + " ns per doc");
	  }
	}

Before the change, indexing takes 5.3us per document. After the change it takes 4.3us.

I hesitated doing the same with StringField but wondered if this could be breaking to some users who might pull a TokenStream themselves.

Maybe we can somehow deprecate using a tokenstream there in 9.x, pulling a tokenstream is very expert and doesn't seem like StringField needs to support that?

high level, i dont think its a big problem, but we are adding some type-guessing, with a lot of runtime checks, versus the user somehow having some type safety via the .document package. Similar to what got fixed recently in stored fields (#12116).
Just worth thinking about, is there anyway this can be more type-safe to the user in the API.

You make a good point about adding more type guessing. Ideally IndexableField would have its APIs designed around how IndexingChain consumes it.
I imagine that we could have an IndexableField#invertableValue() method or something like that that would return a similar object as StoredValue that could hold either a BytesRef or a TokenStream.

I imagine that we could have an IndexableField#invertableValue() method or something like that that would return a similar object as StoredValue that could hold either a BytesRef or a TokenStream.

This sounds better to me than type-guessing and null values. Then my complaint goes away.

also maybe the Reader/String -> Analyzer -> TokenStream path could be fixed to use this? That's also confusing and this PR makes it even more so.

I'm lost, i see type guessing and an InvertableType class that does nothing. Maybe you forgot to 'git add' or something?

Yes! Sorry about that.

its better, i'm only sad about a naming issue:

InvertableType: OK
InvertableType.TERM: Terrible, it isn't a Term at all, its a BytesRef.
InvertableType.TOKEN_STREAM: OK

yes, better thanks! The only thing good about the "Term" was that it did capture the singleton nature. I'd just suggest a small improvement to the javadocs for BINARY to mention that its "a single value" or similar?
We don't want someone to pass a large UTF-8 encoded document in this way :)


Rework of servlet binding (3.0)

I've added a java configuration of the context to the integration test and updated the README.
In addition I've added the coresponding annotations of the servlet API 3.0 but I'm not sure how it works in conjunction with the web-fragment.xml file.
WDYT?

I'm not sure if we're really compatible with servlet 2.5 because of the annotations and some new generic stuff. But we support the configuration by web.xml. Is that your question?
I've removed (hopefully) all 2.5 version numbers from the documentation. -- Even when we're compatible with 2.5, we should promote the servlet module of TracEE 2.0 for servlet 3.0 and above to keep our focus small(er).

LGTM!


Chapter 7 : Terratests

Hi,
I am unable to run terra tests mentioned in chapter 7 because of the below errors. can you please help
/alb_example_test.go:36:30: not enough arguments in call to http_helper.HttpGetWithRetry
have (*"testing".T, string, int, string, int, time.Duration)
want ("prod-grade-infra/vendor/github.com/gruntwork-io/terratest/modules/testing".TestingT, string, *tls.Config, int, string, int, time.Duration)

Ah, you must've ended up with a newer version of Terratest. The newer one now takes in a *tls.Config argument after the URL. You can either fix it by setting that argument to nil (which should be fine since we're making HTTP requests in this test, not HTTPS) or downgrading to an older Terratest version.

Hi I have updated the tls.Config to nil, but i got this error instead.
TestAlbExample 2020-04-07T12:18:22+08:00 retry.go:72: HTTP GET to URL https://terraform-up-and-running-669391245.us-east-2.elb.amazonaws.com
TestAlbExample 2020-04-07T12:18:22+08:00 http_helper.go:27: Making an HTTP GET call to URL https://terraform-up-and-running-669391245.us-east-2.elb.amazonaws.com
TestAlbExample 2020-04-07T12:18:32+08:00 retry.go:84: HTTP GET to URL https://terraform-up-and-running-669391245.us-east-2.elb.amazonaws.com returned an error: Get "https://terraform-up-and-running-669391245.us-east-2.elb.amazonaws.com": dial tcp <IP_ADDRESS>:443: i/o timeout (Client.Timeout exceeded while awaiting headers). Sleeping for 20s and will try again.

looks like something to do with the tls?

What code are you running? The http_helper will use whatever URL you pass in... And I'm noticing the URL is https, so are you setting that to https intentionally? The examples in the book all use http.

my mistake.. typo there. thanks for pointing that out.

different but somehow related, but in the next test case for dependency injection, the test kept failing with :-
`TestHelloWorldAppExample 2020-04-07T18:52:43+08:00 retry.go:72: HTTP GET to URL http://hello-world-test-MZnpP5-1676154927.us-east-2.elb.amazonaws.com
TestHelloWorldAppExample 2020-04-07T18:52:43+08:00 http_helper.go:32: Making an HTTP GET call to URL http://hello-world-test-MZnpP5-1676154927.us-east-2.elb.amazonaws.com
TestHelloWorldAppExample 2020-04-07T18:52:44+08:00 retry.go:84: HTTP GET to URL http://hello-world-test-MZnpP5-1676154927.us-east-2.elb.amazonaws.com returned an error: Validation failed for URL http://hello-world-test-MZnpP5-1676154927.us-east-2.elb.amazonaws.com. Response status: 200. Response body:
Hello, World
DB address: mock-value-for-test
DB port: 3306. Sleeping for 10s and will try again.
`
Am I supposed to change anything, i.e. the mock DB address?

Which test is this? What chapter/section?

Chapter 7, Dependency Injection Test Codes, Pg 243 onwards

Oh, you're right, that is a bug! The test is looking for the response body to be exactly "Hello, World" (see here), but instead, it should be looking for the body to contain "Hello, World". Not sure how that slipped by. I'll fix it this weekend.

Fixed in https://github.com/brikis98/terraform-up-and-running-code/pull/58.

thanks for the fix


[SIL] Verify that the method referenced in {Class,Super}MethodInsts appears in the class vtable.

@swift-ci Please test

Build failed
Swift Test OS X Platform
Git Sha - 9bbadb1dbea3d9a9ce479a028bf357eaedc792fc

Build failed
Swift Test Linux Platform
Git Sha - 9bbadb1dbea3d9a9ce479a028bf357eaedc792fc

@swift-ci Please test

Build failed
Swift Test OS X Platform
Git Sha - 0d7bcc06ef3dbc26a210ebb7f534f3b5933359be

Build failed
Swift Test Linux Platform
Git Sha - 0d7bcc06ef3dbc26a210ebb7f534f3b5933359be

Compatibility suite failures look like the known issues revealed by compiling in Debug mode.


[Network] az network firewall policy rule-collection-group collection rule add: Add parameter 'web-categories'

https://github.com/Azure/azure-cli/issues/16282
https://github.com/Azure/azure-cli/issues/15377
Re: Firewall Premium changes - Azure CLI
(1)az network firewall policy rule-collection-group collection add-filter-collection: Add parameter 'web-categories'
(2)az network firewall policy rule-collection-group collection rule add: Add parameter 'web-categories'
(3)
intrusion-dectection->idps
Intrusion   Signature Override Arguments->IDPS Signature Override Arguments
Traffic Bypass   Rule Arguments->IDPS Bypass Rule Arguments
Transport   Security Arguments->TLS Inpection Arguments
intrusion   detection general state->IDPS mode
detect-mode->idps-mode
enable-terminate-tls->enable-tls-inspection
(4)az network firewall policy: Add parameter "identity"
(5)az network firewall policy: Add custome exception_handler
(6)network firewall policy rule-collection-group collection rule update: New command

This checklist is used to make sure that common guidelines for a pull request are followed.

For new extensions:

[ ] My extension description/summary conforms to the Extension Summary Guidelines.

About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your PR is merged into master branch, a new PR will be created to update src/index.json automatically.
The precondition is to put your code inside this repo and upgrade the version in the PR but do not modify src/index.json.

Network

Network

/azp run

/azp run


Using make as a new hook

Some projects need to run custom tasks when merging with upstream, like
vendoring or code generation. Let's try to run make merge-bot and if
the target doesn't exist we simply don't return an error.
Example on where it'll be used: https://github.com/openshift/cluster-api-provider-openstack/pull/332

/assign pierreprinetti
per our conversation

/test cluster-api-provider-openstack-main


[SPARK-37023][CORE] Avoid fetching merge status when shuffleMergeEnabled is false for a shuffleDependency during retry

What changes were proposed in this pull request?

At high level, created a helper method getMapSizesByExecutorIdImpl on which getMapSizesByExecutorId and getPushBasedShuffleMapSizesByExecutorId can rely. It takes a parameter useMergeResult, which helps to check if fetching merge result is needed or not, and pass it as canFetchMergeResult into getStatuses.
Why are the changes needed?

During some stage retry cases, the shuffleDependency.shuffleMergeEnabled can be set to false, but there will be mergeStatus since the Driver has already collected the merged status for its shuffle dependency. If this is the case, the current implementation would set the enableBatchFetch to false, since there are mergeStatus, to cause the assertion in MapOutoutputTracker.getMapSizesByExecutorId failed:
assert(mapSizesByExecutorId.enableBatchFetch == true)

The proposed fix helps resolve the issue.
Does this PR introduce any user-facing change?

No.
How was this patch tested?

Passed the existing UTs.

Can one of the admins verify this patch?

cc @mridulm @Ngone51 @venkata91 @zhouyejoe Please take a look. Thanks.

Ok to test

Test build #144884 has started for PR 34461 at commit b1cca0c.

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49354/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49354/

Test build #144884 has finished for PR 34461 at commit b1cca0c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144884/

Added unit test. cc @mridulm @dongjoon-hyun @zhouyejoe

Test build #144940 has started for PR 34461 at commit 9c399b4.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49412/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49412/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49412/

Test build #144940 has finished for PR 34461 at commit 9c399b4.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/144940/

I'm thinking that this condition may be wrong:
https://github.com/apache/spark/blob/31b6f614d3173c8a5852243bf7d0b6200788432d/core/src/main/scala/org/apache/spark/shuffle/sort/SortShuffleManager.scala#L134-L136
For a retried map stage, we don't push&merge shuffle blocks for tasks due to dep.shuffleMergeEnabled=false. However, dep.shuffleMergeEnabled=false should be the reason for the reduce stage to not fetch from the existing merged shuffle data in the previous attempt of the map stage. Right?

IIUC, for a retried map stage, it would go with below code path due to dep.shuffleMergeEnabled=false:
https://github.com/apache/spark/blob/31b6f614d3173c8a5852243bf7d0b6200788432d/core/src/main/scala/org/apache/spark/shuffle/sort/SortShuffleManager.scala#L138-L140
Prior to this PR, getMapSizesByExecutorId would call getPushBasedShuffleMapSizesByExecutorId, which would fetch mergeOutputStatuses in getStatuses even for a retried map stage where push-based shuffle is disabled. This further causes enableBatchFetch=false in the convertMapStatuses, thus results in the assertion failure.
The proposed change tries to avoid this behavior - when dep.shuffleMergeEnabled=false, getMapSizesByExecutorId would pass canFetchMergeResult=false to getStatuses to not fetch mergeOutputStatuses for this case. @Ngone51

IIUC, for a retried map stage, it would go with below code path due to dep.shuffleMergeEnabled=false:
...

That's what we have today. But I'm thinking the current behavior may be wrong. (Your fix based on the current behavior is correct). For example, assuming we have stage 1 and stage 2, and stage 2 hits fetch failure and leads to stage 1 rerun.
So, with the current behavior, when stage 2 reruns, it'd have stage1.dep.shuffleMergeEnabled=false, and stage 2 can't fetch merge data of stage 1. However, I think we do can fetch the merge data, right?

@Ngone51 For the example you gave, namely:
stage1/attempt1 (parent) -> stage2/attempt2 (child)  -- fetch failed --> stage1/attempt2 -> stage2/attempt2
Here, we have two cases:

If stage1 was non determinate stage - we throw away all merged output from stage1/attempt1 when we run stage1/attempt2.

This happens via shuffleDep.newShuffleMergeState() which is invoked from beginning of submitMissingTasks

If stage1 was deterministic stage - we disable shuffle push for stage1/attempt2, and run stage1/attempt2.

This happens in submitMissingTasks if it is already merged finalized, see here.

In all of these combinations, whether stage2/attemptN uses merged output or not must only be based on whether merged output exists for it to use - based on parent stage.
Currently, we are using a single variable to control behavior at both mapper side (push side) and reducer side (using merged output) - this was fine initially - but given for deterministic parent stages and retried child stages, we should make them separate.
IMO we should add a new variable, say useMergedShuffleInput - which will be based on whether there exists merged output from parent or not for that shuffle dep.
Thoughts @Victsm, @zhouyejoe, @otterc, @rmcyang ?

Ok this makes sense to me - current behavior indeed seems not a very efficient way to handle the scenario of stage retry when the parent stage is determinate and the parent stage has merge data, which should not be ignored by that condition.
Given SPARK-37023 is designated to fix the enableBatchFetch assertion failure bug for current behavior, shall we file another spark jira and implement the improvement there? @Ngone51 @mridulm @Victsm @zhouyejoe @otterc @venkata91

That sounds fine to me, we can fix the immediate assertion issue (which will fix the problem of stage retry causing application failure when push based shuffle is enabled) and follow up with the fix to ensure we allow determinate stages to read on stage retry.
How does that sound @Ngone51 ?

sgtm.

@rmcyang Can you please file a jira for the issue that @Ngone51 detailed ? Thanks !
Will merge this to master and branch-3.2

Merged to master and branch-3.2
Thanks for working on this @rmcyang !
Thanks for the reviews @dongjoon-hyun, @Ngone51 :-)

Filed SPARK-37313 for the issue that @Ngone51 raised. Thanks for the reviews! @mridulm @dongjoon-hyun @Ngone51


Initial Finnish Translation

100% translated, 68 strings (0 fuzzy, 0 bad tokens, 0 not translated)

Awesome, thanks!


[Feature]:  Add unit testing framework

Context
The project needs to have unit tests added ASAP as it was deferred to post launch (considering the simplicity of the site)
Problem and/or Opportunity
Adding a robust unit testing suite to help ensure the proper functionality of the site going forward
Proposed Solution
Add unit tests with jest and react-testing libraries
Testing the changes
The tests will be expected to pass 100% upon completion and cover the main functionality of the code base:

component rendering
api call to the arweave grpahql endpoint
user interactions with components

Additional Information
No response

Gonna know this one out so there's a good base to build our testing upon.  i will set the unit test coverage to 70% as a base and we can adjust it as we move forward


REP: KubeRay RayService Incremental Rollout

Add REP for RayService incremental upgrade strategy using max_surge_percent and Gateway API.

cc: @kevin85421 @andrewsykim


[Development] Build Configuration Object (similar to scout)

See: https://github.com/cardinal-dev/scout/pull/8

CardinalEnv() has been created. Cardinal v3.0 will heavily utilize this environment object.


feat: add tuple

add tuple validator

Codecov Report

Merging #39 (4640cd0) into main (b30ed19) will increase coverage by 0.52%.
The diff coverage is 92.85%.

:exclamation: Current head 4640cd0 differs from pull request most recent head 04b012f. Consider uploading reports for the commit 04b012f to get more accurate results

@@            Coverage Diff             @@
##             main      #39      +/-   ##
==========================================
+ Coverage   75.62%   76.14%   +0.52%
==========================================
  Files          37       38       +1
  Lines         886      914      +28
  Branches      182      190       +8
==========================================
+ Hits          670      696      +26
- Misses        202      204       +2
  Partials       14       14

Impacted Files
Coverage Δ

src/validators/TupleValidator.ts
92.00% <92.00%> (ø)

src/lib/Shapes.ts
35.55% <100.00%> (+2.99%)
:arrow_up:

src/validators/imports.ts
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b30ed19...04b012f. Read the comment docs.


App cleaner doesn't find Info.plist file for iPhone/iPad apps on new Mac with Apple Silicon chip

Guide
How to clean uninstall macOS apps using AppCleaner open source alternative
Summary
I wanted to uninstall Chess from chess.com (iPad version) but got the error:
... % app-cleaner.sh /Applications/Chess.app
Cannot find app plist

I was able to reproduce the error for another iPhone app.
The error message is printed here. So for new Macs the if condition should be adapted.

Ha sorry😅
I wanted to add something:
It seems that the Info.plist file is saved in the wrapped app ($1/Wrapper/AppName.app/Info.plist).
In this file the condition gives plausible output.
... % /usr/libexec/PlistBuddy -c "Print CFBundleIdentifier" "/Applications/Chess.app/Wrapper/Chess.app/Info.plist"
com.chess.iphone

The WrappedBundle is apperantly simply an alias for the true app which is in the Wrapper folder.
Since the app name and the wrapped app name don't doesn't have to be the same, it is easier to check for the file with "$1/WrappedBundle/Info.plist".
So my suggestion would be:
if [ -e "$1/Contents/Info.plist" ]; then
  bundle_identifier=$(/usr/libexec/PlistBuddy -c "Print CFBundleIdentifier" "$1/Contents/Info.plist" 2> /dev/null)
elif [ -e "$1/WrappedBundle/Info.plist" ]; then
  bundle_identifier=$(/usr/libexec/PlistBuddy -c "Print CFBundleIdentifier" "$1/WrappedBundle/Info.plist" 2> /dev/null)
else
  printf "%s\n" "Cannot find app plist"
  exit 1
fi

Convincing output:
... % app-cleaner.sh /Applications/Chess.app
Checking for running processes…
Finding app data…
/Applications/Chess.app
/Users/username/Library/Application Scripts/com.apple.Chess
/Users/username/Library/Containers/com.apple.Chess
/var/folders/c2/flxghxsd6x1c3fjbngkzlpx00000gn/C/com.apple.Chess
/var/folders/c2/flxghxsd6x1c3fjbngkzlpx00000gn/C/com.chess.iphone
Move app data to trash (y or n)? y
Moving app data to trash…
Done

I agree. App-cleaner does not find iPhone/iPad apps installed into silicon Macs. I ended up manually following https://www.macrumors.com/how-to/uninstall-iphone-ipad-apps-m1-mac/
To note, app-cleaner should look through ~/Library/Containers. I don't think I know enough scripting to enable that, so I end up doing it manually right now.

I ended up modifying the original script to work with the iOS apps on Macs. But additionally, we need to delete the app containers (and any error log files or whatnot else still persists). I shared the script and wrote it up: https://verityj.github.io/2023/06/21/clean.html. In case it helps anyone.
I specifically kept it as a two step process to have control of exactly what stays and what goes.


Update houdahspot to versioned url

Thank you for the contribution. It was merged directly as 22804494ef6892863d567656b913c90374e898ae to keep commit history cleaner. Your contribution is still credited to you.


Add RecordReader trait and proc macro to implement it for a struct

Which issue does this PR close?
Closes #4772.
What changes are included in this PR?

Added a parquet::record::RecordReader trait similar to the existing parquet::record::RecordWriter

Added a parquet_derive::ParquetRecordReader derive macro to implement the new RecordReader trait for slices of structs, similar to the existing parquet_derive::ParquetRecordWriter macro

Added tests to parquet_field.rs for reader_snipped similar to the existing tests for writer_snipped

Added a test to parqued_derive_test to test writing a struct slice to a parquet file, reading it again, and then checking the output is the same as the original struct.

Not all types supported by ParquetRecordWriter are supported by ParquetRecordReader yet. Specifically there is no support for options or references. References is difficult because using Rc would mean we have to manually annotate types for the compiler which requires quite a lot of matching on the Field's ty field. See: https://github.com/Joseph-Rance/arrow-rs/blob/f49472eb93c4bf27aa20d8040c57539fc73d1059/parquet_derive/src/parquet_field.rs#L452-L454

Updated the readme to reflect the above point

Are there any user-facing changes?

Users will now have access to the parquet::record::RecordReader trait and the parquet_derive::ParquetRecordReader proc macro

@tustvold would it be possible to get a review on this? Apologies if I am pinging the wrong person. Please do let me know if there is anything further I need to do to get this ready for review

Apologies, it is on my list to review, but I'm not very familiar with the non-arrow APIs so need longer to sit down and get the necessary context. Development has largely focused over the last couple of years on the columnar, arrow-based APIs as they have better performance characteristics, so I lack any major context 😅

No problem. Just wanted to make sure this PR hadn't been missed

@tustvold Apologies for pinging you again. I was wondering if you could estimate a possible timeline for review of this PR?

@tustvold I've resolved most of your comments. Left two open:

For the references, I think there isn't anything more to do, but I left it open in case I have misunderstood the situation
For the other comment, I wonder if you have any opinions on what I should write?

Any idea why clippy is suddenly failing on code that (I think) has nothing to do with this PR? Can I ignore these issues or is it due to something I have changed?

You probably need to merge the latest master to pick up fixes for the lints added in the latest Rust version

@tustvold could you approve the workflows please?

Workflows should pass now


parent body missing

Challenge http://beta.freecodecamp.com/challenges/waypoint-target-the-parent-of-an-element-using-jquery has an issue.
The explanation states:
For example, your "jQuery Playground" h3 element has the parent element of , which itself has the parent body.
HOWEVER, there is no body element in the code.

Closing as stale


Command missing for production

Hello!
After some troubleshooting, it appears that some extra command are necessary to deploy the production part.
Instead of:
cd docker/prod
docker compose -f docker-compose.yml -f proxy.yml up -d

This, should work (at least for me it worked):
git describe --tags >CURRENT_VERSION
cd docker/prod
docker compose -f docker-compose.yml -f proxy.yml build
docker compose -f docker-compose.yml -f proxy.yml up -d

Moreover, if I am not mistaking, an user (with uid 1000) needs to be created and permissions for files adjusted accordingly

This should be improved with completion of #28: you can use make config (choose production), make build and make start


Add "Why HTTP bots" page.

I believe this should add the page requested in #102

Wheres the content 💀

it just says add 💀 i didnt know what to put there lmao

I'd like to hold off on this until there's actual content


Latest version violates SemVer!

Your latest release 1.3.0 has several breaking changes from 1.2.8. This release does not follow semver and causes build failures in apps that depend on your package. See SemVer documentation summary: https://semver.org/#summary
1.3.0 is a minor upgrade which should be: "MINOR version when you add functionality in a backward compatible manner"
Your change was not backward compatible and therefore should have been a major version change: 2.0.0. Please be more careful in future releases.

hi @dustin-graham thanks for suggestion. I will notice it in next work.


WIP: Adding ICA_AROMA functionality

This is my attempt at adding ICA_AROMA functionality to fmriprep. Almost 100% certain it doesn't work as is.
I have ICA_AROMA completing it's own transforms since it appears the template space could be something something different than ICA AROMA is expecting, in addition, ICA_AROMA requires a affine fsl matrix and fnirt fsl warp to pass in so ANTs is off the table (afaik).
I may continue editing/working on this, but first I wanted feedback if this is a worthwhile endeavour or if it's better to hi-jack the original code from ICA-AROMA to make it flexibly fit in the fmriprep workflows (as was done with compcor).
Thanks!
James

Hey. This is a great contribution. In fact we planned to do this - see https://github.com/poldracklab/fmriprep/issues/334#issuecomment-283877729 so I am really happy to see your contribution! I prefer reusing rather than reinventing so using the Nipype interfaces as you are doing is preferable to hijacking the code.
A few things I would suggest:

Do we need to perform the normalization again using FNIRT or maybe we can just reused the normalized times generated using ANTs (outputs of epi_mni_trans_wf)? It would save some time.

We should make sure the smoothing is performed with the same algorithm/parameters as in the original paper to match the training set.

Things worth considering, but potentially a separate PR:

It would be great to add visualization of the good/bad components to the report. I wrote a generic melodic reportlet generator that could be reused here: https://github.com/poldracklab/niworkflows/pull/134. This would require running melodic outside of aroma, but is should support this use case.

Not sure we need to make users chose between aggressive and non-agressive aggressors. We could output both sets in the confounds.tsv file as a default. There should be a way to generate both sets without much computational overhead. It would avoid users having to rerun all of fmriprep if they change their minds.

Once again - thanks for this PR!

Thanks you for your comments! Would the template: MNI152NLin2009cAsym always have to be specified to run ICA_AROMA, since it looks like it's currently not mandatory.

I can see about reporting the good/bad components, but I will at least run melodic separately to generate the report.

aggressive v. non-aggressive. This is completed via fsl_regfilt, where the non-aggressive denoising is completed through partial regression (e.g. variance shared with the other components is kept in the data) versus full regression (e.g. the shared variance is removed from the data). So I would want to return the coefficients from the partial regression, I think. fsl_regfilt does not return the coefficients sadly, so I would need to calculate that myself, perhaps with this?

It would be easier just to pass in the epi_mni and forgo the transforms, is this less recommended? I guess I'm less sure about whether the ouputs are organized so we can pass one affine and one warp if there are fieldmaps in the processing stream.

I will look into the published smoothing method

On Sat, May 27, 2017 at 5:33 PM, James Kent<EMAIL_ADDRESS>wrote:

Would the template: MNI152NLin2009cAsym always have to be specified to run
ICA_AROMA, since it looks like it's currently not mandatory.

Currently ANTs normalization is only performed if "template" is specified
as one of output spaces (which is the default). I would change this to "or
ICA AROMA is required".

I can see about reporting the good/bad components, but I will at least
run melodic separately to generate the report.
Great!

aggressive v. non-aggressive. This is completed via fsl_regfilt, where
the non-aggressive denoising is completed through partial regression (e.g.
variance shared with the other components is kept in the data) versus full
regression (e.g. the shared variance is removed from the data). So I would
want to return the coefficients from the partial regression, I think.
fsl_regfilt does not return the coefficients sadly, so I would need to
calculate that myself, perhaps with this?
http://scikit-learn.org/stable/modules/generated/sklearn.cross_decomposition.PLSRegression.html
I don't think we need to run fsl_regfilt for this. The aggressive
regressors would be just the timecourses of the "bad" ICs. The
non-agressive regressors would be the timecourses of "bad" ICs after
regressing out timecourses of "good" ICs. The main difference is that the
non-agressive regressors do not share any variance with timecourses of
"good" ICs. This should be pretty straightforward to do, but we can leave
it to a separate PR.

It would be easier just to pass in the epi_mni and forgo the
transforms, is this less recommended? I guess I'm less sure about whether
the ouputs are organized so we can pass one affine and one warp if there
are fieldmaps in the processing stream.
I think ICA AROMA has a mode where you do not pass any transformations and
the inputs are just assumed to be in the MNI space. We should use that.

I will look into the published smoothing method
Thanks!

I'm still not done with this. Things I still need to work on:

ica aroma partial regression: I've given it some effort, but I can't replicate the results from fsl_regfilt for non-aggressive denoising. My attempt followed this line of reasoning.
linear regression: y_hat = mx +b, so the residual (or partial) after regression should be y - y_hat, right? I can send a link to some example data if what I did wasn't a simple mathematical error.

I still need to work on the melodicRPT, I'll continue reading up on that to add that functionality.

clean up the code and get formatting consistent. I still don't have a good sense of what should be a method, a node, and a workflow, and I saw in a tutorial that some simple mathematical operations (like number output from ImageStats) can be modified before being passed into another node.

Just making an update to keep consistent with working on the code/learn the nipype framework.

On Wed, May 31, 2017 at 8:35 PM, James Kent<EMAIL_ADDRESS>wrote:

I'm still not done with this. Things I still need to work on:

ica aroma partial regression: I've given it some effort, but I can't
replicate the results from fsl_regfilt for non-aggressive denoising. My
attempt followed this line of reasoning.
linear regression: y_hat = mx +b, so the residual (or partial) after
regression should be y - y_hat, right? I can send a link to some example
data if what I did wasn't a simple mathematical error.
That sounds correct.  Are you including all "good" IC in the right hand
side of the equation? We can also leave it to another PR.

I still need to work on the melodicRPT, I'll continue reading up on
that to add that functionality.
Let me know if you have any questions. Here's an example how this can be
used:
https://github.com/poldracklab/mriqc/blob/91368d14167267009bea81ab833c4a76dd919371/mriqc/workflows/functional.py#L167

clean up the code and get formatting consistent. I still don't have a
good sense of what should be a method, a node, and a workflow, and I saw in
a tutorial that some simple mathematical operations (like number output
from ImageStats) can be modified before being passed into another node.
Just making an update to keep consistent with working on the code/learn
the nipype framework.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/poldracklab/fmriprep/pull/539#issuecomment-305380157,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAOkp_fKsvtldaYlP8yzhlF24MzCUJ-Iks5r_jGQgaJpZM4NoPBt
.

RE: regression
here is the relevant bit of code:
def calc_residuals(x,y):
        X = np.column_stack(x+[[1]*len(x[0])])
         beta_hat = np.linalg.lstsq(X,y)[0]
         y_hat = np.dot(X,beta_hat)
         residuals = y - y_hat

         return residuals

Where y is the "bad" motion IC, and x is a matrix of the good_ICs. Maybe will leave it to a separate PR if I can't figure it out soon. I asked on the FSL listserve to possibly get an answer

Great move!
On Thu, Jun 1, 2017 at 1:35 PM, James Kent<EMAIL_ADDRESS>wrote:

RE: regression
here is the relevant bit of code:
def calc_residuals(x,y):
X = np.column_stack(x+[[1]*len(x[0])])
beta_hat = np.linalg.lstsq(X,y)[0]
y_hat = np.dot(X,beta_hat)
residuals = y - y_hat
     return residuals

Where y is the "bad" motion IC, and x is a matrix of the good_ICs. Maybe
will leave it to a separate PR if I can't figure it out soon. I asked on
the FSL listserve
https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=FSL;719d1657.1706 to
possibly get an answer
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/poldracklab/fmriprep/pull/539#issuecomment-305612540,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAOkp60uj1IVUf55hnVDAMiq-BL7knxIks5r_yCKgaJpZM4NoPBt
.

Okay, I'm getting closer to making this a reality, there are still several TODOs before it's ready for a formal review:

I can't get the basic melodic reportlet to work, I get the following error:

170612-01:58:13,360 workflow INFO:
	 Executing node ds_ica_aroma_report in dir: /out/work2/fmriprep_wf/single_subject_controlGE159_wf/func_preproc_task_flanker_wf/func_reports_wf/ds_ica_aroma_report
170612-01:58:13,463 workflow ERROR:
	 ['Node ds_ica_aroma_report failed to run on host 6c638518fad3.']
170612-01:58:13,466 workflow INFO:
	 Saving crash info to /out/fmriprep/sub-controlGE159/log/20170612-015349_3ed5f63d-a4cb-4a3c-9244-2955be966222/crash-20170612-015813-root-ds_ica_aroma_report-8fc746fc-5653-40cb-99ba-ced522bd80e3.pklz
170612-01:58:13,467 workflow INFO:
	 Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.6/site-packages/nipype/pipeline/plugins/multiproc.py", line 322, in _send_procs_to_workers
    self.procs[jobid].run()
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 372, in run
    self._run_interface()
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 482, in _run_interface
    self._result = self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 613, in _run_command
    result = self._interface.run()
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base.py", line 1066, in run
    self._check_mandatory_inputs()
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base.py", line 971, in _check_mandatory_inputs
    raise ValueError(msg)
ValueError: DerivativesDataSink requires a value for input 'in_file'. For a list of required inputs, see DerivativesDataSink.help()

^it looks like a simple error but I can't find the solution right now.
Here is the link to the workflow graph made, in case I connected the nodes incorrectly.

Have ica_aroma output in the correct directory: right now all the output is going into a /tmp/ directory, but I think with nipy/nipype#2056, this issue should be fixed as soon as niworkflows uses the updated version of nipype.

change the testing suite to reflect ica aroma functionality.

I've changed the dockerfile to include fsl atlases, but I don't know if this is needed, or if I could make a symbolic link between the template image fmriprep already uses to the standard place ica aroma expects the MNI images.

Good news: within a docker container, I have it working (albiet the melodic report error above), but the confounds tsv file is generated with the expected output (the partial regression implemented here is not identical to fsl_regfilt's partial regression, but it's close)

Great progress! I had a look at the code, but unfortunately I cannot see what could be wrong with the way you use MELODICRPT. Things worth checking:

Is the SVG file present in the working directory of the MELODICRPT node? You might need to set remove_unnecessary_outputs to false in nipype.cfg

the only difference between how you are using MELODICRPT and how it it used in mriqc is setting the report_mask. This should not be necessary, but it is a good idea anyway since you already have a mask available and there is no need to calculate it again the the MELODICRPT.

Thanks @chrisfilo and @effigies, you guys are rockstars! That fixed the report error I was getting in the workflow. Sorry that re-running takes forever, when nipy/nipype#2056 is implemented, debugging will be way faster (I think).

I think the code is pretty much ready for a review, only a couple items remain that I know of:

The melodic report may need some extra space on the report to load correctly (I think i'll leave the labeling of the components as good/bad to a separate PR; right now people can painstakingly reference the components in the confounds.tsv with the melodic report.)

Still need to wait on the updated nipype to automatically set the output directory of ICA_AROMA

misc: formatting code to be in line with PEP8 standards

Add more documentation to places I may have forgotten.

For the test case, I'd suggest enabling on ds005. ds054 is already taking a bit longer because of the fieldmaps.

I think (I'm positive) I added the test to circleci incorrectly

I've deleted the dependency of having to set the ICA_AROMA outdir, but fmriprep throwing an error, because niworkflows hasn't been updated to include nipy/nipype#2056. Is there a timeline for the next release of niworkflows, since this should not be merged until the next release of niworkflows (which contains the relevant version of nipype).

I think I (or someone with more knowledge) should change how the melodic report is printed. On my tests with noisy subjects that have many components, the melodic report is very tiny, and doesn't load correctly in the html file.

To update niworkflows, you can somewhat follow its circle.yml to get working with submodules.
git clone git@github.com:jdkent/niworkflows.git
cd niworkflows
git submodule init
git submodule update
git -C nipype fetch origin
git -C nipype checkout origin/master
git checkout -b update_pin
git add nipype
git commit -m 'PIN: Update nipype to include ICA_AROMA fixes'
git push -u origin update_pin

I demonstrated checking out origin/master for simplicity. You can reference another commit if it seems like a good idea.
Once this is merged into niworkflows, then we can pin fmriprep to that commit. When it's time to release fmriprep (or mriqc), we will release niworkflows and set the dependency to the new version.

@effigies, Thank you for walking me through the commands to do that. That did help me understand the process better. When I look at niworkflows though (before I submit a pull request), I see the pinned version of nipype already contains the fixes to ICA_AROMA, but the requirements.txt  for fmriprep only selects the released versions of niworkflows (as far as I understand it). version 0.1.2 does not include the selected fix for ICA_AROMA. So I followed your instruction to pin fmriprep to the most recent niworkflows commit. Thanks again for walking me through this!

Looks like the AROMA images aren't getting included in the report.
https://888-53175327-gh.circle-artifacts.com/1/home/ubuntu/ds005/out/fmriprep/sub-01.html

Yeah, that's what happened in my tests too, I can spend some time learning how to make those reports, it's what I meant by my number 3 point in my last post. The melodic reports are large, so they probably deserve their own page. So do you think the images aren't being included at all in the report or are the images too large to fit in that space?

As long as we leave AROMA as optional and experimental feature I am happy to leave the reports for a separate PR. The visualization of the components without an indication how they were classified is of limited use anyway.
I had a look at the confounds.tsv created on Circle and there are only two extra regressors - one agressive and one non aggressive. The non aggressive is just a row of zeros. Is it because of the downsampled nature of the test data?
There is also a more general question what to do if no signal or no noise components were detected. You opted for a warning and dummy regressors. I feel this is too silent - I would prefer at least to not include dummy regressors in such cases. We can also consider raising an error - after all this is a sign that something is wrong. What do others think?

If we're not going to fix the reports in this PR, the blank images should be removed from the report file.
I agree a row of zeros isn't great to include in the confounds file. Raising an error seems like a reasonable way to handle the issue.

RE: what to do if there are no signal/noise components. I agree to raise a runtime error if there are no signal components in the data as that signals something is wrong with the data, but I don't know what are the implications if there are no noise components in the data, ICA-AROMA doesn't treat no noise components as an error, but doesn't really leave a clue on how to represent the data in a tsv.

Hello, great to hear about adding ICA_AROMA to fmriprep! Seems like you've come a long way already! I don't think I have enough knowledge of the fmriprep code to comment on its correctness, but I did want to add a couple of remarks that might be useful.

Getting the residuals should indeed be a straightforward regression. The one thing to keep in mind is how you treat the mean. If you remove this by demeaning the data or by adding a mean regressor, it is a good idea to add the mean back to the residuals afterwards. Demeaned data might screw up some calculations afterwards.

In AROMA, as you've probably noticed, noise component classification happens in standard space, the denoising regression in native space. This fits with our (and fsl's) philosophy to keep data in native space as long as possible, only bringing results to standard space. That said, I see no reason to not use an ANTs calculated transformation to bring the data into standard space. The spatial features used (i.e., edge mask and csf mask) are not so fine-grained that they would be hugely affected by using a different registration. In fact, we do have the option built in to only use linear registration files.

When applying ICA_AROMA we always investigate the number of ICA components derived, and the number of components flagged as noise, as low/high values in either of these could signal some weird stuff in the data. Of course this nicely fits with: 'ALWAYS LOOK AT YOUR DATA' ;)

Thanks for your comments @maartenmennes. Since we are not denoising the data (only producing the regressors) I think we should opt for ones that are demeaned (which is what I think is happening now).
@jdkent I cannot think of a situation when no components classified as noise is the correct input, but I might be lacking experience with AROMO. For the time being I would go for an error in such case and see what different use cases people will come back to us with.

But this will silently pass for any other errors now right?
What about additional flag something along the lines of --ignore-missing-noise-error?

Only other errors in ICA, but yes. I like the CLI flag option better than mine (sorry @jdkent for prompting you to change that before discussion completed). I'd say it should have "(for testing only)" or similar in the description. And I'd group it with --debug.

@chrisfilo, @effigies Would it be best to treat the flag option as I did with --use_aroma? As in, pass the variable on through the workflow until it reaches its destination in confounds?

@jdkent Yes.

I added a flag called --ignore-aroma-denoising-errors, but I found the melodic code needed to be changed as well. Passing a mask into melodic makes a big difference in ICA AROMA classification, and I thought I added it in before when @chrisfilo suggested, but I only added the report mask, not the actual mask to use in melodic. This fix results in the classifier giving fewer noise components. However, my result with using Non-aggressive denoising is still not identical to ICA-AROMA's fsl_regfilt calculation. The code I use to check is:
tail -n +2 'nonaggr_tsv' > 'no_header_nonaggr_tsv'
Text2Vest 'no_header_nonaggr_tsv'  'nonaggr_des'
fsl_glm -i 'smoothed_image' -o 'denoised_glm' -d  'nonaggr_des' --out_res='fmriprep_denoised_data'

The difference between calculations amounts to more than what I would expect from a rounding error.
P.S. Thank you all for helping me with this, I have learned a lot by going through this process.

Thank you for doing this!
Could you share correlation plots between the regressors provided by
fsl_glm and those calculated by you? This could help us see what is going
on.
On Tue, Jun 20, 2017 at 1:39 PM, James Kent<EMAIL_ADDRESS>wrote:

I added a flag called --ignore-aroma-denoising-errors, but I found the
melodic code needed to be changed as well. Passing a mask into melodic
makes a big difference in ICA AROMA classification, and I thought I added
it in before when @chrisfilo https://github.com/chrisfilo suggested,
but I only added the report mask, not the actual mask to use in melodic.
This fix results in the classifier giving fewer noise components. However,
my result with using Non-aggressive denoising is still not identical to
ICA-AROMA's fsl_regfilt calculation. The code I use to check is:
tail -n +2 'nonaggr_tsv' > 'no_header_nonaggr_tsv'
Text2Vest 'no_header_nonaggr_tsv'  'nonaggr_des'
fsl_glm -i 'smoothed_image' -o 'denoised_glm' -d  'nonaggr_des' --out_res='fmriprep_denoised_data'
The difference between calculations amounts to more than what I would
expect from a rounding error.
P.S. Thank you all for helping me with this, I have learned a lot by going
through this process.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/poldracklab/fmriprep/pull/539#issuecomment-309884378,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAOkpw-At9LzvXklb22T14lG65CVWIvFks5sGC3vgaJpZM4NoPBt
.

Sorry, I wasn't clear. I'm trying to compare the denoised result from fsl_regfilt as implemented by ICA_AROMA to the regressors derived from ICA_AROMA in the script passed into fsl_glm. I'm trying to get an identical between the two methods. I can make the results identical with aggressive denoising, but I cannot do the same with the non-aggressive method.
So the ICA_AROMA method would be something like:
fsl_regfilt -i smooth_file -o denoised_file -f classified_motion_ICs.txt -d melodix_mix -a
Then using the non-aggressive derived in fmriprep, I would do this:
fsl_glm -i smooth_file -o glm_file -d AROMANonAggr.mat -out_res=denoised_file
I've tried adding the demean option in fsl_glm and add the mean back in after, but I can't get the residual image from fsl_glm to match the output from fsl_regfilt.
Here is a link to ica_aroma_wf for a subject.
Unless I misunderstood what you are asking, I don't think fsl_regfilt outputs regressors as ICA_AROMA uses it.

Hmm - I'm not quite sure what is going on then. I tried looking through the  source of fsl_regfilt https://surfer.nmr.mgh.harvard.edu/pub/dist/freesurfer/tutorial_packages/OSX/fsl_501/src/melodic/fsl_regfilt.cc but the regression is don in a funky way.
Maybe @maartenmennes could give us a hint. Are we doing the orthogonalization in a wrong way?

To help facilitate my understanding and for those that can help, I've made an ipython notebook to explain the steps I'm taking to perform multiple linear regression with the output from ICA-AROMA. I'll start here to assume there is some mistake in the code I wrote, before I move on to thinking there is something wrong in the way I called fsl_glm.

additional insight: I may need to load the entire image into a numpy array and then do the regression described in the link except return the residuals (This description should match what's in the source code @chrisfilo posted).  I'll keep you posted as I try to emulate this method.

Sounds good. Let us know if there's anything we can do. I've just pushed out a new release today that includes the PR that's been taking up a lot of my energy, so I can direct some more attention to this PR.

I submitted another question  to the fsl listserve (the first one didn't get a reply). Since Dr. Christian F. Beckmann is the author of the fsl_regfilt code, perhaps I should directly contact him?
I'm having trouble converting his denoising partial regression to a regression equation that would output  residual noise components.
Conceptually, I want to remove any shared variance in the noise components. Previously I was only giving the shared variance to the "signal" melodic components, but it looks like there is additional shared variance between the noise components and the image data. To include the data, I first regress out all the melodic components and use the residual image data as a regressor to soak up shared variance between itself and the noise components.

Yay! Dr. Christian F. Beckmann replied to my question. Now I need to figure out how to implement the equation he gave me in numpy/scipy:

If X=(X_1|X_2) is the melodic mixing matrix where we want to remove effects in X_2 only then the denoised data is given by
Y_clean = ( Id - X_2 * (0 | Id_2) pinv(X)) * Y
Where Y is the original data and Id is the Identity matrix (t x t) and Id_2 is the identity matrix of size n2, where n2 is the number of nice components.

This may take some work, but will be a good learning opportunity
Given his answer I believe  to accomplish our goal we need to solve the equation
pinv(( Id - X_2 * (0 | Id_2) pinv(X)) * Y)
I understand most the operations, but I am unfamiliar with the (0 | id_2) notation (specifically the vertical bar)
That's my first question.

I think in this context this means concatenation.
On Jun 22, 2017 11:23 AM, "James Kent"<EMAIL_ADDRESS>wrote:

Yay! Dr. Christian F. Beckmann replied
https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ind1706&L=fsl&F=&S=&X=0E95A4350CBB09A848&Y=james-kent%40uiowa.edu&P=204617
to my question. Now I need to figure out how to implement the equation he
gave me in numpy/scipy:
If X=(X_1|X_2) is the melodic mixing matrix where we want to remove
effects in X_2 only then the denoised data is given by
Y_clean = ( Id - X_2 * (0 | Id_2) pinv(X)) * Y
Where Y is the original data and Id is the Identity matrix (t x t) and
Id_2 is the identity matrix of size n2, where n2 is the number of nice
components.
This may take some work, but will be a good learning opportunity
Given his answer I believe to accomplish our goal we need to solve the
equation
pinv(( Id - X_2 * (0 | Id_2) pinv(X)) * Y)
I understand most the operations, but I am unfamiliar with the (0 | id_2)
notation (specifically the vertical bar)
That's my first question.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/poldracklab/fmriprep/pull/539#issuecomment-310413670,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAOkp7DrbCBLGYIIQF9Mxzy_0sD4aI52ks5sGocDgaJpZM4NoPBt
.

Alright, I've given my best faith effort, and I wasn't able to emulate the equation. Moreover, even if I was, based on Dr. Beckmann's response, the matrix would have as many columns as there are voxels of interest in the dataset (if I'm interpreting the equation correctly). A tsv of that size would be unruly and potentially confusing to users.
Here's my suggestion: pass the ICA_AROMA classification file and the melodic mix to the subject derivative directory and let the user run fsl_regfilt on their data if they want non-aggressive denoising applied to their data. The drawback to this being more clutter in the directory. Less-so of a drawback is coming up with another naming scheme for this unique type of confound that is not a voxelwise regressor, but doesn't quite fit in with the confound tsv.
My Naming Proposal:
<BIDS_subject_info>_aroma_noise_classification.csv (for the file that contains a comma separated list of components suggested to be related to noise)
<BIDS_subject_info>_aroma_melodic_mix.tsv (the full melodic mixing matrix)
Then document furiously to let users know how to implement non-aggressive de-noising.
Thoughts?

Okay. Let me try to summarize the situation. As I think I've said, I've never used MELODIC or AROMA; hopefully I understand roughly what's going on, but I recognize what follows is likely to be laughably simplistic. Please correct me if it's also wrong.

MELODIC finds IC time series, which I assume are global signals, independent of space. Some of these get labeled noise, some get labelled signal.
Aggressive de-noising simply regresses out all noise ICs from every voxel.
Non-aggressive does some kind of per-voxel modification of the ICs, so we can't actually generate these regressors in such a way as to apply in a GLM anyway.

If that's basically right, then I think your strategy seems reasonable.
I believe the BIDS preference is always for TSV, not CSV, so I'd use TSV for both of your files. We should also avoid _ in the descriptive parts of the filenames, since that's used as a field delimiter in BIDS. How about:

aroma_noise_classification.csv -> AROMAnoiseICs.tsv
aroma_melodic_mix.txv -> MELODICmix.tsv

Or something like that? Since the MELODIC output is actually not specific to AROMA post-processing, I don't think we need to label it with both AROMA and MELODIC. I'm open to alternative capitalization schemes.

Looks like you got it right for points 1 and 2, and I'm not sure if I have it right for point 3, but I'll try explaining what I think I know more.
RE: 3, The response I got from the fsl listserv suggests it's possible to make a non-aggressive denoising glm matrix, but I'm not mathematically inclined enough to recreate the process. Additionally, the pseudo-inverse of Y_clean (where Y_clean is the image data) is what we need to pass into the glm. It may be possible to simplify the matrix, but the direct solution appears to be each voxel would get a column. (if my understanding is correct, haha).

I believe the BIDS preference is always for TSV, not CSV,

I also believe the preference is for tsv, but this is another special case. I include the noise IC classification as a csv because fsl_regfilt does not accept tab separated values with it's "-f" flag. However, fsl_regfilt does not tell you it can't read the .tsv, instead it only filters the first index/number in the list. Making the AROMAnoiseICs.tsv could lead to misuse on the user end if they try to pass the tsv directly to fsl_regfilt without knowing that quirk. ugh, that is annoying.

Alright. I think in any case, it's reasonable to go ahead in this PR and not produce the non-aggressive regressors, and just add documentation. If future investigation shows we can do it for users without producing a column per voxel, that can be another PR. (Incidentally, if it would end up being a column per-voxel, it probably makes more sense to output it as a .nii.gz time series so that the spatial information is directly encoded.)
Thanks for the context on the CSV issue. That makes sense and let's definitely go ahead with your plan, there.

I'm not entirely sure what the errors for the ci/circle mean for me, I've tested it on my data (however, there were not edge cases where no signal components were detected), so I am unsure where I should look to fix the error.

Now I'm out of ideas on how to pass the tests, it looks like running ICA_AROMA.py from the container failed, perhaps due to some property of the data?

@effigies @chrisfilo Question: what software do you use to preview the *rst files in docs? I tried retest for ubuntu, but my first impression is disappointing.
Again thank you for taking the time to help me through this journey, I believe the code is ready barring documentation formatting and any further suggestions you have.

python setup.py build_sphinx

Generally run it in a docker container, because it's pretty touchy.

And I'll have a thorough look through (probably Monday). Thanks so much for all the effort you've put into this.

@jdkent Thanks so much for spending a month on this. This was a huge contribution.

Thank you! It's great to see this merged!
On Jun 28, 2017 1:08 PM, "Chris Markiewicz"<EMAIL_ADDRESS>wrote:

@jdkent https://github.com/jdkent Thanks so much for spending a month
on this. This was a huge contribution.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/poldracklab/fmriprep/pull/539#issuecomment-311773551,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAOkp1XK3Oi2t8l3NfWEQZSXcpHQRVR1ks5sIrKxgaJpZM4NoPBt
.

@effigies @chrisfilo This was an amazing learning experience for me and I can't emphasize enough how thankful I am for you two taking the time to guide me through this contribution. I plan to keep contributing to fmriprep and projects like it (hopefully I'll need less help and be able to help others as I learn more about nipype and python).


Add Discord JS Client event typings

Information
Typings for the Discord JS Client were added for the levelUp event.
Issue Related
No related issue, it's a feature.
Checks

[x] Does the package work as intended?

Thonk u akora


feat(icons-angular): add innerClass input

adds innerClass input to support directly styling icons
updates package.json to reflect our Angular 7 compatibility
updates README and adds an API documentation section

Looks like CI is failing just because of some prettier stuff 👍 Should be able to run yarn format and then will go green ✅

Yep, just fixed that 🎉 slightly non-intuitive error, but yarn format seems to have sorted it out 👍


Wire example

I have prepared example of use Wire library for state management and separation of concerns, another way of Flux implementation, it's almost complete clone (except on line in model) and reuse of the business logic from example of Wire use in original repository - web version with template from TodoMVC. Also this example use Flutter Widget for Wire - WireDataBuilder.
The library is still in developing, but I think major API and concepts won't change, I have plan to add "middleware" and "traces" to inspect current state and registered wires/signals and data.
@brianegan could you please check my PR from wire branch?

@brianegan any resolution on my PR?


What about AR STI?

I have set up a User model and a Manager model with Single Table inheritance like this :
class User < ActiveRecord::Base
  include Tenacity

  t_has_many :posts, :foreign_key => "user_id"
end

class Manager < User
end

And a Mongoid model :
class Post
  include Mongoid::Document
  include Tenacity

  field :name

  t_belongs_to :user
end

However the association only works when the user is a User and not a Manager.
If i try to add posts to a manager I just get an empty array when trying to fetch them trough user.posts.
Any thoughts on this?
Thanks

Tenacity currently does not support Single Table Inheritance.  I'm not sure what it would take to implement, but I can take a look later on this week after I return from vacation.  However, implementing many-to-many will be the next feature I plan on adding.  So, it may be a while until I get to this.
Feel free to give it a shot if you'd like.

Ok, after some testing.
It does seem to work if I add the user_id manually. However if I change the type of the user, user_id on associated documents gets nil'ed.
I'll see if I can fix it.


Bump libffi version to 3.4.4

In the local test env, the old upper bound is giving conda solver some headache to solve to the latest snowpark version
Please answer these questions before submitting your pull requests. Thanks!

What GitHub issue is this PR addressing? Make sure that there is an accompanying issue to your PR.
Fixes #NNNN

Fill out the following pre-review checklist:

[ ] I am adding a new automated test(s) to verify correctness of my new code
[ ] I am adding new logging messages
[ ] I am adding a new telemetry message
[ ] I am adding new credentials
[ ] I am adding a new dependency

Please describe how your code solves the related issue.
Please write a short description of how your code change solves the related issue.

Test job in https://ci-dev-142.int.snowflakecomputing.com/job/PythonStoredProcBuildPrecommitTest/393/
BUWA: https://buwa.ci1.us-west-2.aws-dev.app.snowflake.com/builds/precommit/ci-dev-133.int.snowflakecomputing.com/12223


refactorings on #61,#65 and #92

Adds the missing criteria on unit tests for the features duplicate detection, missing value imputation and ARIMA prediction.
Closes #61
Closes #65
Closes #92

LGTM


Fix clean build from VS

Resolves #383
The Packages project needs to build after its dependencies

WebJobs.Script
WebJobs.Script.Extensibility
WebJobs.Script.WebHost

This fix explicitly adds these dependencies so VS will build the projects in the right order.

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot


Feature Request: Consolidate Nuget packages

Has there been any thought to consolidating the nuget packages?
If you look at
https://i.imgur.com/HSG3Qus.png
There are 6 packages and it is really difficult to determine which one to include when.
Maybe combine FFImageLoading.Windows with Xamarin.FFImageLoading?

@ravensorb In fact there are only 4. I have no idea why somebody created 2 alternative packages (they end with f3).

ok - it was/is still a little confusing :)
Thanks

I contacted the package owner and he seem to have removed f3 nuget packages.
So now we have only 4 packages visible via search.
Having windows in the same package won't be complicated.

Any thoughts on combing the Windows version into a the main package?

It is a very good idea, it is planned for 2.2.

2.1 released which has only 3 nugets now.
Old windows package has been unlisted, it will take a few hours to be visible.

And thanks for your help !! 👍


feat(overview): update spinnaker architecture diagram

I haven't updated the copy on the page. I reference this diagram fairly often though, and it makes me sad that we don't even have Operator in the current one.
This is the source for the image:
https://app.lucidchart.com/invitations/accept/db579abc-3da3-4434-9004-c56da72c2498
It is used on this page (old image shown in prod, obvs):
https://docs.armory.io/docs/overview/architecture/
If we want to update the copy as a blocker to merging this in... idk, I could probably get to it later this week?

@dogonthehorizon thanks for updating the diagram! Approved if we want to merge without updating the page copy ... which we really should do...

Deploy preview:  https://deploy-preview-667--armory-docs.netlify.app/docs/overview/architecture/


[WIP] Cast optimizer: Bail out on a conditional downcast to a subclass of a bridged ObjC class.

rdar://problem/32316838

@swiftix is picking this up in #9821.


Fix metadata description in transaction-model.rst

Metadata description in docs was wrong after remove-metadata-uuid, this pr fixes, and also reflects that metadata may be NULL even in the case of CREATE transactions.

I rendered it locally and it looks good!
:+1:

I forgot to include my usual screenshot this time, will try to remember next time :stuck_out_tongue:


[macOS] Fix IME focus in popup windows.

Same as https://github.com/godotengine/godot/pull/72497 and https://github.com/godotengine/godot/pull/77977
Fixes https://github.com/godotengine/godot/issues/73591

Thanks!


Thorium Win-Updater does not work with 126.0.6478.231

System Details

OS: [Windows 10]
Thorium Version [126.0.6478.231]

Problem
Thorium Win-updater by Itguillaume doesn't work with thorium 126.0.6478.231
Additional Notes
When I changed the thor_ver file from Win, AVX2 to Win,Th24 the win-updater worked again.

While the Th24 version is marked as the "latest" release in github, the release name itself begins with "BETA".
So, it's probably by-design to not have it automatically update to that version. It also probably shouldn't be marked as the latest regular release, since github has options for marking something as a beta/pre-release.


Container for service "lotus-miner" is unhealthy and fails to start

Description:
I recently experienced an issue with the Docker container running the "lotus-miner" service. Upon tearing down the compose setup using docker-compose down, I've been unable to bring it back up due to an error stating that the container for the "lotus-miner" service is unhealthy.
Steps to reproduce:
Ran docker-compose down to tear down the running setup.
Attempted to bring the setup back up using docker-compose up.
Expected result:
The "lotus-miner" service container should have started normally without any issues.

Thanks! I'm not sure what has caused this, it may be a timing issue. I've updated to newest version of Lotus and Boost and added extra startup checks, so please try with latest version and re-open issue if still a problem.


dont disable xdebug manually anymore in travis builds.

travis updates composer within the build to 1.3.x which contains a native workaround for xdebug problems.

we might even drop 5.5 support

The build is failing currently, but only PHP 5.6 and I don't know why...

Build is fixed on master. Merging this knowing it will be fine.


Call to unknow data element(s - maybe due to redefine?)

Not sure what the error is about - maybe thefiller redefinesis not recognized?
Best refer to the attached sample where in particular the declaration of AUSGABE has been shortened. The original contains approx 15 redefines - because of "periodic groups in Adabas".
ISSUE13.CBL.txt
Preprocessing file ISSUE13.CBL.
Parsing file ISSUE13.CBL.
Collecting units in file ISSUE13.CBL.
Analyzing program units of compilation unit ISSUE13.
Analyzing identification divisions of compilation unit ISSUE13.
Analyzing environment divisions of compilation unit ISSUE13.
Analyzing data divisions of compilation unit ISSUE13.
Analyzing procedure divisions of compilation unit ISSUE13.
 The problem was, that qualified data names such as T11-DFU IN V0DATA12 were not resolved, correctly.
I made a fix in 67c146f467cc9955ae10f25e4808effc756774c5 and added a unit test based on the snippet provided by you. Thanks for issuing!


docs: about this guide page

This PR adds the new About This Guide page to the style guide folder.
Resolves #1243

ping @alequetzalli

oops! I also forgot to remind you earlier that for the netlify preview link to generate, it needs:

_sections.md
index.md

➡➡ can you add those to your PR and lemme know if that fixes the lack of preview link? :) I see that @BhaswatiRoy seems to have successfully done it for her PR https://github.com/asyncapi/website/pull/1327/files


How to clear a figure?

I'm making a UI for data exploration, and want to have a button that I can click to switch between different views of the data, kind of like tabs. For example, in one view, I have one big axis that takes up most of the figure, and in another view, I have two smaller axes that take up half of the figure each. In each view, I don't want to see the axes from the other view.
Crucially, I don't want to have to switch between two figures, as that will launch another window. I need to use the same window, with the ability to change a figure's entire layout table on the fly.
Is there a way to clear, and then reconstruct, a Figure's layout table?

One can do a lot with Observables. Take a look at this page with the video:
https://makie.juliaplots.org/stable/documentation/nodes/index.html
But I would also appreciate an easy method to clear a figure. I did not find any.

I got it, I need to delete entries in Figure.content

I got it, I need to delete entries in Figure.content

And how do you delete the entries exactly😅


Stable

push to our fork and create a pull request

@shravanthium I don't think this belongs here :)

It's also breaking the one rule (there is just one) of the MIT license - that you leave our copyright in place.


i3lock-color: fix manpage-name and manpage

Motivation for this change
The manpage of i3lock-color should also be named i3lock-color (and not i3lock), and talk about i3lock-color (and not i3lock), so:

rename manpage from i3lock.1 to i3lock-color.1
change "i3lock" to "i3lock-color" in manpage-synopsis

Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

cc @garbas @malyn

Now, I've completely renamed everything:

/etc/pam.d/i3lock to i3lock-color
every occurence of "i3lock" to "i3lock-color" in the manpage.

@GrahamcOfBorg  build i3lock-color

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: i3lock-color
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

Success on x86_64-linux (full log)
Attempted: i3lock-color
Partial log (click to expand)
glibPreFixupPhase
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/bcwc9z9c9pva3h5y7ldxlm2icid82xy2-i3lock-color-2.11-c
shrinking /nix/store/bcwc9z9c9pva3h5y7ldxlm2icid82xy2-i3lock-color-2.11-c/bin/i3lock-color
gzipping man pages under /nix/store/bcwc9z9c9pva3h5y7ldxlm2icid82xy2-i3lock-color-2.11-c/share/man/
strip is /nix/store/j75dgadrff2d1fyc4fczmcgqkid2imdx-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/bcwc9z9c9pva3h5y7ldxlm2icid82xy2-i3lock-color-2.11-c/bin
patching script interpreter paths in /nix/store/bcwc9z9c9pva3h5y7ldxlm2icid82xy2-i3lock-color-2.11-c
checking for references to /build in /nix/store/bcwc9z9c9pva3h5y7ldxlm2icid82xy2-i3lock-color-2.11-c...
/nix/store/bcwc9z9c9pva3h5y7ldxlm2icid82xy2-i3lock-color-2.11-c


Wi-Fi DPP code causing build error

Describe the bug
DPP build error when CONFIG_WIFI_NM_WPA_SUPPLICANT_DPP=n and CONFIG_WIFI_NM_HOSTAPD_AP=y
To Reproduce
CONFIG_WIFI_NM_WPA_SUPPLICANT_DPP=n and CONFIG_WIFI_NM_HOSTAPD_AP=y
build wifi example
Expected behavior
build pass
Impact
wifi examples build error
Logs and console output
zephyr_downstrem_build/zephyr/modules/hostap/src/supp_api.h:388:56: warning: 'struct wifi_dpp_params' declared inside parameter list will not be visible outside of this definition or declaration
388 | int hapd_dpp_dispatch(const struct device *dev, struct wifi_dpp_params *params);
Environment (please complete the following information):

OS: Linux
Toolchain: Zephyr SDK
Commit SHA or Version used: 1da74ef705bcf1723a3271266622948251253c0a

Additional context

@krish2718 https://github.com/zephyrproject-rtos/zephyr/pull/79224
is causing build error because hapd_dpp code is not wrapped.
FYI DPP macros are removed as per review requests in https://github.com/zephyrproject-rtos/zephyr/pull/73707.
So why do we need to hide these again? Could you pls share build errors you were trying to fix?

@krish2718 #79224 is causing build error because hapd_dpp code is not wrapped. FYI DPP macros are removed as per review requests in #73707. So why do we need to hide these again? Could you pls share build errors you were trying to fix? In my view zephyr L2 DPP implement is independent from hostap.

I guess by wrapping you are referring to using #ifdef DPP? The build errors are due to below compilation warning (pasted one, but there are many). and in general if we have a feature flag (optional feature) we need to use that to save memory.
include/zephyr/net/wifi_mgmt.h:946:24: error: 'struct wifi_dpp_params::<unnamed union>::wifi_dpp_configurator_add_params' invalid; an anonymous union may only have public non-static data members [-fpermissive]
  946 |                 struct wifi_dpp_configurator_add_params {

I guess by wrapping you are referring to using #ifdef DPP? The build errors are due to below compilation warning (pasted one, but there are many). and in general if we have a feature flag (optional feature) we need to use that to save memory.
include/zephyr/net/wifi_mgmt.h:946:24: error: 'struct wifi_dpp_params::<unnamed union>::wifi_dpp_configurator_add_params' invalid; an anonymous union may only have public non-static data members [-fpermissive]
  946 |                 struct wifi_dpp_configurator_add_params {

Yes by wrapping I mean "#ifdef DPP".
And these build errors are because union structure defined in structure, which is C++ (or C99?) tools error.
We have a fix toup. I'm raising it.
I agree that we want to save memory, I'll also add macro guard in hostapd DPP.

And these build errors are because union structure defined in structure, which is C++ (or C99?) tools erro

Yes, we have a few samples that use C++ (Matter) they throwed this error.

And these build errors are because union structure defined in structure, which is C++ (or C99?) tools erro

Yes, we have a few samples that use C++ (Matter) they throwed this error.

Fix is already merged in this https://github.com/zephyrproject-rtos/zephyr/pull/79542


OSSM-755: Fix TLS certificates used in security tests

This is a manual cherry-pick of #661.

/test istio-unit-2-1

/test istio-unit-2-1


How to install pyogg 0.7 in termux?

I once installed pyogg 0.7 with .as_array avalaibe in VorbisFile, but now forgot the step. In termux, there's no 0.7 in the database. Anyone could help?

pip install git+https://github.com/TeamPyOgg/PyOgg