[QUESTION] What does it mean the source is not consistent?

Hello,
Thank you for an extremely useful project and documentation. I followed the instructions today without any problems
I do have one question. When I run,
> chameleon show_status --source mysql

  Source id  Source name    Type    Status    Consistent    Read lag    Last read    Replay lag    Last replay
-----------  -------------  ------  --------  ------------  ----------  -----------  ------------  -------------
          1  mysql          mysql   stopped   No            N/A                      N/A

I see the source database mysql is labelled as not consistent.
What does the mean? What would you recommend I investigate to see if I can fix this issue?
To investigate further I looked at the logging,
> chameleon start_replica --config default --source mysql --debug

2020-09-24 20:35:00 MainProcess DEBUG global_lib.py (563): Replica process for source mysql is running
2020-09-24 20:35:00 MainProcess DEBUG pg_lib.py (659): Changing the autocommit flag to True
2020-09-24 20:35:00 MainProcess DEBUG pg_lib.py (1276): Cleaning replayed batches for source mysql older than 1 day
2020-09-24 20:35:01 read_replica DEBUG pg_lib.py (679): There is already a database connection active.
2020-09-24 20:35:01 read_replica DEBUG pg_lib.py (679): There is already a database connection active.
2020-09-24 20:35:01 read_replica DEBUG pg_lib.py (3110): Collecting schema mappings for source mysql
2020-09-24 20:35:01 read_replica DEBUG mysql_lib.py (1452): Batch data [(1, 'mysql-bin.000001', 1786, 't_log_replica_mysql_2', '')]
2020-09-24 20:35:01 read_replica DEBUG mysql_lib.py (1000): collecting table type map
2020-09-24 20:35:01 read_replica DEBUG mysql_lib.py (1217): GTID DISABLED - log_file mysql-bin.000001, log_position 1786. id_batch: 1
2020-09-24 20:35:01 read_replica DEBUG mysql_lib.py (1231): ROTATE EVENT - binlogfile mysql-bin.000001, position 1786.
2020-09-24 20:35:01 read_replica DEBUG pg_lib.py (3321): Checking consistent status for source: mysql
2020-09-24 20:35:01 read_replica DEBUG pg_lib.py (3352): The source: mysql is not consistent

But I don't really understand.

Hi, the init_replica process locks the tables one by one collecting the binlog position for that specific table.
The copy is therefore inconsistent as data may change on the other tables meanwhile the copied table is locked.
At the end of the init_replica process the last binglog position is collected to determine whether all the tables are in replica status and therefore the source is consistent.
When the replica starts the data is received and, using the binlog coordinates for each table is applied or discarded.
When the last binlog position is reached then the source is marked consistent, meaning that all the tables are replicated and the data received is no longer discarded.
If your mysql never got updates since the init_replica it may display the inconsistent status.
The status will be cleared as soon as one event is replicated from mysql to postgres.
I hope I've answered your question.

Thank for the detailed explanation!
I am performing the MySQL to Postgres transfer on a isolated test server. It contains a small amount of data but it nothing is inserting data. Would this trigger the inconsistent message?

If your mysql never got updates since the init_replica it may display the inconsistent status.

definitely.
you can just create a new table on mysql and the inconsistent message should go away


As a user, I want to see basic details for individual items in the archive so that I can see the record details and get to the HathiTrust version.

Notes for testing

accessible from search results or from admin interface (use the 'view on site' button)
display is provisional and unstyled

It's great to see the record number so I can click to the Hathi version. Now that I am looking at it, I realize that this link could be a button that says "view item in Hathi" instead of that long record details, since that's really more for us. I think it's messy to have it on the front-end. But the link needs to be live to the site, so that is what I'll test.
Clicking on the live record number link, I notice that it needs to open in a new window automatically.
Searching inside an item for a word doesn't yet take us to the page where that word appears in Hathi; that isn't what we're testing. Because that isn't working yet.
Since the Hathi URL contains the record id inside it, I'm not sure that we need to list both on the single item page, maybe just keep this record for us as admins and have some button that says "access Hathi" and then after a search (from the search results page with the pages from Hathi) "access page in Hathi" or something like that?


Reduce rows lru cache in /rows

someone is querying an image dataset ioclab/animesfw a lot at different offsets which fills the lru cache of image data and leads to OOM

Codecov Report
Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (49a55cc) 90.58% compared to head (fcd533f) 90.67%.
Report is 4 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1864      +/-   ##
==========================================
+ Coverage   90.58%   90.67%   +0.09%
==========================================
  Files         234      225       -9
  Lines       14559    13988     -571
==========================================
- Hits        13188    12684     -504
+ Misses       1371     1304      -67

Flag
Coverage Δ

jobs_cache_maintenance
96.67% <ø> (ø)

jobs_mongodb_migration
85.95% <ø> (ø)

libs_libapi
?

libs_libcommon
91.31% <ø> (ø)

services_admin
89.25% <ø> (ø)

services_api
86.79% <ø> (ø)

services_rows
83.45% <ø> (ø)

services_search
79.16% <ø> (ø)

services_sse-api
94.16% <ø> (ø)

services_worker
92.45% <84.61%> (-0.02%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files
Coverage Δ

services/worker/src/worker/executor.py
94.16% <84.61%> (-1.25%)
:arrow_down:

... and 9 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


in quickstart add a check for dependency

could be more elegant
FILES_VOLUME_PATH=/files ./quickstart.sh
./quickstart.sh: line 2: docker-machine: command not found
./quickstart.sh: line 3: docker-machine: command not found
./quickstart.sh: line 4: docker-machine: command not found
./quickstart.sh: line 5: docker-machine: command not found
./quickstart.sh: line 6: docker-compose: command not found
./quickstart.sh: line 7: docker-compose: command not found
./quickstart.sh: line 11: docker-compose: command not found
./quickstart.sh: line 11: docker-compose: command not found
./quickstart.sh: line 11: docker-compose: command not found
./quickstart.sh: line 11: docker-compose: command not found

Updated readme.


Reference variable

I set the configuration file with app.config.update() and how to use these variables in other files (not referencing this method again via import file)

Is there a question here?

Once app.config is set, you can access them anywhere you can access app.


[BUG] Output flooding on some specific case

Describe the bug
Hey!
Running scan against scanfactory.io results in the terminal being flooded with enormous amount of data.

I am not sure of what is happenning here. Looks like it starts recursive bruteforcing and with that outputs every single request it has.
This problem is seen not only on scanfactory website but this is the first website that has the described problem and conscent to scan them.
To Reproduce
Steps to reproduce the behavior:

Run feroxbuster -w ~/wordlists/no-extensions.txt -k --url  https://scanfactory.io

Expected behavior
Expected to not have huge output of every request being sent
Environment (please complete the following information):
Arch linux
Feroxbuster: v2.3.3
Alacritty (but same in urxvt)

Can you check if this is the problem/solution?
https://epi052.github.io/feroxbuster-docs/docs/faq/progress-bars/

@epi052  thanks for a quick answer!
Not really. Maximum width doesn't help. Also putting -q doesn't fix the problem

@c0rv4x not a problem.  It looks like you may own the domain, if so, am i free to scan and see if i can reproduce what you're seeing?

@epi052 yes. My email is anatoly at scanfactory dot io in case you want a verification from me

good deal, i can take a look this evening and let you know.
It feels like a terminal emulator / rendering problem, but I'll check it out.

Thought so too. But the same issue in urxvt and alacritty. Thanks anyway, will wait for your tests

Scanning from my machine looks normal

installed alacritty and ran it from that terminal as well, still looks good

installed rxvt-unicode and ran a third scan, still looks good.
Do you have a different VM / OS to try from?

Super weird! I will give it a try and let you know. Thanks for chechkin it out
I am thinking this might be zsh for some reason

I have just tested on MacBook and it works fine. I am trying to make it work on my arch linux as it is my main device.
I have tried using bash and no Xorg environment and the issue is still there. Wow. Do you have any advice on what to move further with the dissecting this issue?

the terminal emulator, the font, and the width of your terminal vs. the width of your output are the three primary suspects in my mind.
Do you use a PS1? Is it colored? If so, try exporting an ascii PS1
export PS1='> '

This shouldn't affect output that isn't on the same line as your prompt, but we can test it for funsies (i've seen terminal emulators behave poorly with colored PS1's when the line becomes too long, even if after rendering the ansi escape codes, it doesn't look long at all).
You could post up the output of env.
Have you tried something like xterm (i.e. a more traditional terminal emulator)?
Can you try changing the font in one of your emulators to NotoColorEmoji?

OMG I just figured out.
In my wordlist I had the following entries:
.htaccess/
.htpasswd/

When a website had 403 on ht.* looks like it just went recursive scan on those dirs. I just deleted those and now it is fine. Pretty weird tho.
Sorry for the hassle and thanks for your help

no worries, glad you got it resolved.  closing this issue


Remove .npmignore support

does anyone actually use this? I didn't even know it was a thing

nope, I don't use it.  not sure why you'd even have an .npmignore in your up app, considering npm is for packages (not applications) :P

ahh yea I wasn't sure, seems a little weird, going to remove it and see if anyone has an issue there


Link checker false positives

The link checker GitHub Action is hitting quite a few false positives.  This seems to be due to two problems:

Checks are too aggressive: if subsequent links refer to the same host, that host gets a rapid series of requests that can lead it to temporarily blocking the checker.
Checks are too brittle: if a check fails with a potentially transient error (eg. 403, 500,...) the checker gives up. It would make sense to wait a little while and retry, just to rule out very short term issues. Ideally, we'd allow links to fail for a few days before flagging them, but this is much more difficult.

Workaround is to not check some problematic links at treat 429s as 200s. I'll leave this open as it's not really a proper solution.  There have been a couple of issues lodged with the underlying tool, which seems to think delaying only on 429 is sufficient. Perhaps I'll try that again.

Okay, with a suitably large default timeout, that seems to work okay.


Harmonize video file naming to avoid mismatch

This PR moves all video file naming definitions to one file.
There is still a mismatch where recorder uses different folder than UI video playback. Recorder is using config file which can be used to set video output into different folder than default /mnt/extsd/movies/.

@cruwaller Please slove the conflicts.

Thanks! I will fix.

rebased to main.


MAJOR ASSET REPOSITORY OUTAGE

Due to issues on our GitHub pipelines, it is currently no longer possible to download the assets for TSH from the Asset Downloader or the Asset Portal. We are currently working on a fix for said issue.
In the meantime, we have set up the following workaround:

Grab the game’s asset packs from this link. Each game has its own ID which you can find on the Asset Portal

Extract the archive into the user_data/games directory

We will keep you informed when we have updates about the situation. Thank you for your understanding.

END OF THE ASSET REPOSITORY OUTAGE
We are now fairly confident that the asset repository outage is now over, and asset packs should now once again be available from both the Asset Downloader and the Asset Portal. Thank you for your patience.


[FEATURE REQUEST] Add new tests for Infrastructure generators

Is your feature request related to a problem? Please describe.
Right now we have limited number of tests for Infrastructure generators.
Describe the solution you'd like
We want to add new tests to cover more cases for Infrastructure generators.
Describe alternatives you've considered
None.
Additional context
None.

DoD checklist

Changelog

[ ] updated
[ ] not needed

COMPONENTS.md

[ ] updated
[ ] not needed

Schema

[ ] updated
[ ] not needed

Backport tasks

[ ] created
[ ] not needed

Documentation

[ ] added
[ ] updated
[ ] not needed

[ ] Feature has automated tests
[ ] Automated tests passed (QA pipelines)

[ ] apply
[ ] upgrade
[ ] backup/restore

[ ] Idempotency tested
[ ] All conversations in PR resolved
[ ] Solution meets requirements and is done according to design doc
[ ] Usage compliant with license

Not going todo anymore feature requests for Epiphany.


Kotlin duplicated at debugging section

At Debugging doc's section there are duplicated Kotlin tabs.
Probably the first occurrence of the Kotlin tab should be the Javascript tab.
Bellow is the screenshot from the page of the web site.

Thanks for pointing that out. Should be fixed now (& live soon)


New battery health has some issue

I tried to set some custom value with no benefit.
I have a model 3 LR 2020 (so battery capacity shall be 75kWh or a little bit more).

Hi, give me some context about your data logged.
was this dashboard working in a previous version? Is this a clean Teslamate installation?
This dashboard works getting estimations from the charges session, and you must have at least one session charge that added  more than (let's say) 15kWh.
As I can see in the graph you shared, only the ranges have been calculated, read from drivings/positions table and the current SOC, but it couldn't get any data from charges, that's why battery capacity cannot be calculated.
Could you share a screenshot of your charges dashboard?

Hi @djpalmis
I see. Everything seems to be fine. There's no problem about driving, since all estimations are based on chargers. It's really strange.
Could you try this? Go to the "dashboard settings", then variables, then click on "aux" variable and scroll down. You should see preview of values like the following picture. Let me know if you got something or absolutely nothing.

Something

Great, that's something. We have to dig a little bit more to see which query is not working.
Could you run this query from your Database to see what you get?
WITH PositionsCharges AS ( 	SELECT  date         ,battery_level         ,usable_battery_level         ,rated_battery_range_km         ,ideal_battery_range_km 	FROM positions 	WHERE car_id = 1 	AND ideal_battery_range_km is not null  	UNION ALL 	SELECT  date         ,battery_level         ,coalesce(usable_battery_level,battery_level) AS usable_battery_level         ,rated_battery_range_km         ,ideal_battery_range_km 	FROM charges c 	JOIN charging_processes p 	ON p.id = c.charging_process_id 	WHERE p.car_id = 1 ), AllRanges AS ( 	SELECT floor(extract(epoch from date)/86400)*86400 AS time 	       ,convert_km((SUM(rated_battery_range_km) / nullif(SUM(coalesce(usable_battery_level,battery_level)),0) * 100)::numeric,'km') AS "rated_range_km" 	FROM PositionsCharges 	GROUP BY  1 ), MaxRange AS ( SELECT MAX("rated_range_km") AS "max_rated_range_km" 	FROM AllRanges ), CurrentRange AS (     SELECT convert_km(rated_battery_range_km,'km') /  nullif(coalesce(usable_battery_level,battery_level),0) * 100 AS "current_rated_range_km",     usable_battery_level, convert_km(rated_battery_range_km,'km') AS "rated_battery_range_km"     FROM positions WHERE  car_id = 1 AND (rated_battery_range_km IS NOT NULL AND usable_battery_level IS NOT NULL AND battery_level IS NOT NULL)     ORDER BY "date" DESC LIMIT 1 ), Aux AS (     SELECT charge_energy_added / NULLIF(convert_km(end_rated_range_km - start_rated_range_km,'km'), 0) * 100.0 AS DiscardValue FROM charging_processes     WHERE car_id = 1 AND end_date IS NOT NULL AND end_rated_range_km IS NOT NULL AND start_rated_range_km IS NOT NULL     ORDER BY end_date DESC LIMIT 1 ), ChargesRAW AS ( 	SELECT  cp.id         ,cp.end_date         ,cp.charge_energy_added         ,c.usable_battery_level 	FROM charging_processes cp 	INNER JOIN charges c 	ON cp.id = c.charging_process_id 	WHERE cp.car_id = 1 AND cp.charge_energy_added > (SELECT  DiscardValue FROM Aux) ), Charges AS ( 	SELECT  (100.0 * charge_energy_added) / (GREATEST(1,MAX(usable_battery_level) - MIN(usable_battery_level))) AS bc 	FROM ChargesRAW 	GROUP BY  id, charge_energy_added, end_date 	ORDER BY end_date DESC ), MaxCapacity AS (     SELECT MAX(bc) AS MaxCapacity FROM Charges ),  CurrentCapacity AS (     SELECT CurrentRange."rated_battery_range_km" * Aux.DiscardValue /  CurrentRange.usable_battery_level  AS CurrentCapacity     FROM CurrentRange, Aux ), Battery AS ( 	SELECT  (100.0 - (CurrentCapacity.CurrentCapacity * 100.0 / MaxCapacity.MaxCapacity)) AS Degradation FROM CurrentCapacity , MaxCapacity )  SELECT * FROM MaxRange, CurrentRange, Aux, Battery, MaxCapacity, CurrentCapacity;

Ehm.. sorry to ask you this: How can I run a query?
In the meantime I'll go on searching on Google 😄

You should use any GUI postgresql client like pgAdmin or DBeaver, even Visual Studio Code has some free plug-ins for it.

As we discovered it has something to do with the database. It seems that there is some charging sessions not closed. It needs some data tweeks.


api: remove buggy SegmentFileConfiguration

This creates a weird type, but it's an openapi-generator bug. We don't need this type right now.
// SegmentFileOpts Optional parameters for the method 'SegmentFile'
type SegmentFileOpts struct {
    XRequestID optional.String
    XIdempotencyKey optional.String
    Body optional.Map[string]interface{} // invalid
}

Codecov Report

Merging #710 into master will increase coverage by 0.56%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #710      +/-   ##
==========================================
+ Coverage   92.51%   93.07%   +0.56%
==========================================
  Files          65       65
  Lines        7438     5775    -1663
==========================================
- Hits         6881     5375    -1506
+ Misses        397      240     -157
  Partials      160      160


Kai UX

Priorities

[ ] Remove frame reload flickering
[ ] Remove custom slots from kai (use defaults)
[ ] Minimal component variants for editable Table frame
[ ] Profile menu to replace actions
[ ] Dense mode variant for all controls
[ ] Standardize margins/spacing for all controls/lists, etc.
[ ] Prefer controlled components (e.g., Dialog)
[ ] Standardize scroll container (see Radix)
[ ] Support light/dark mode
[ ] Settings panel/dialog

Next

[ ] Shell panels
[ ] Swipeable sidebar
[ ] Mosaic

[ ] Horizontal/Vertical/Grid patterns
[ ] Complete kanban

Settings Panel/Dialog

Controlled vs uncontrolled pattern:
const Dialog: FC<{ open, onChange? = (open: boolean) => void }> = ({ open: controlledOpen }) => {
  const [open, setOpen] = useState(controlledOpen);
  useEffect(() => setOpen(controlledOpen), [controlledOpen]);

  ...
}

This won’t be necessary, Radix’s Dialog’s open state is already controllable. We should either forward all those props or proxy them with useControllableState.

EditableObjectList: hanging "Plus" button doesn't naturally pick up same density as list

EditableList is working as expected, please provide repro steps and screenshots.


Automatically promote releases on Maven Central

Also, document this insanity.
I still don't see the release on Maven Central yet, though hopefully it will appear soon.
cc @hzariv

Ah! This seems to have worked.
<dependency>
  <groupId>io.opentracing.contrib</groupId>
  <artifactId>grpc-opentracing</artifactId>
  <version>0.2.0</version>
</dependency>

@hzariv if you actually understand gradle (I do not) and want to make any suggestions, go for it. Otherwise I will merge in 24-48h.

@bhs Thanks. It is available now in maven central. Haven't used gradle much yet but this artifact's gradle script looks way more complicated than the one's I have seen. What is the deal with jar signing? this is not a JCE jar :-)

@hzariv

What is the deal with jar signing? this is not a JCE jar :-)

I have no idea. Such a hassle!


Karibu tests for UI components with Vaadin 14.6+

Hello, the project I'm working on was previously on Vaadin 14.5.5 & Karibu v10 1.1.27
I was trying to update Vaadin from 14.5.5 to 14.6.4 / 14.6.5, and then the existing Karibu tests started to break, so I thought to update Karibu to v1.3.0, however it didn't solve the problem.
This is not the full stacktrace, but most of the test failures are due to the same issue:
(For both server-side created components & polymer template components)
For _get()
Message:
N/A
Stack trace:
java.lang.NoClassDefFoundError: kotlin/KotlinNothingValueException
	at com.github.mvysny.kaributesting.v10.LocatorKt.find(Locator.kt:193)
	at com.github.mvysny.kaributesting.v10.LocatorKt._find(Locator.kt:139)
	at com.github.mvysny.kaributesting.v10.LocatorKt._get(Locator.kt:107)
	at com.github.mvysny.kaributesting.v10.LocatorJ._get(LocatorJ.java:65)

For _click()
Message:
N/A
Stack trace:
java.lang.NoClassDefFoundError: kotlin/KotlinNothingValueException
	at com.github.mvysny.kaributesting.v10.LocatorJ._click(LocatorJ.java:94)
	at io.skymind.pathmind.webapp.ui.layouts.components.NotesFieldTest.notesFieldSaveButtonTest(NotesFieldTest.java:44)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

In our project, we set up Karibu tests to test the behaviour of the UI components without its attachment to a specific view. It seems like _get(component) stopped working with the latest Vaadin & Karibu version.
In our tests, this is the basic setup:
(The following example is a PolymerTemplate component)
The test worked fine before updating Vaadin & Karibu.
@Before
public void setUp() {
    MockVaadin.setup(new Routes(), () -> new MockedUI());
    notesField = new NotesField("title", "abc", text -> {}, false, true, false);
}

@After
public void tearDown() {
    MockVaadin.tearDown();
}

@Test
public void notesFieldSaveButtonTest() {
    _click(notesField.save);  // this would result in the error java.lang.NoClassDefFoundError: kotlin/KotlinNothingValueException
}

Please let me know if there is anything I can change to fix. If unfortunately Karibu requires a specific view to test the components, we'll have to remove Karibu tests since we already have a comprehensive set of end-to-end tests.

Hi, im on a vacation right now, but the main problem is probably java.lang.NoClassDefFoundError: kotlin/KotlinNothingValueException . Sounds like the kotlin-stdlib jar is missing from the classpath. Please make sure that kotlin 1.5 stdlib is on the classpath

This did the trick for me:

org.jetbrains.kotlin
kotlin-stdlib
1.3.72

Thank you @mvysny that was helpful.

No problem, glad that the issue was solved 👍


change value of app.kubernetes.io/version label

Changes
change value of app.kubernetes.io/version label in publish, #2898
Submitter Checklist
These are the criteria that every PR should meet, please check them off as you
review them:

[ ] Includes tests (if functionality changed/added)
[ ] Includes docs (if user facing)
[x] Commit messages follow commit message best practices
[ ] Release notes block has been filled in or deleted (only if no user facing changes)

See the contribution guide for more details.
Double check this list of stuff that's easy to miss:

If you are adding a new binary/image to the cmd dir, please update
the release Task to build and release this image.

/release-note-none  /kind bug

/release-note-none

/kind bug

/ok-to-test

/lgtm


Automate running the entire process

Create a method for a user to run the entire process.

Fixed by #28 the run-workflow.sh script runs the entire process.


Allows passing extra headers to the Client Credentials strategy

Hiya,
This fixes (what I think) is a wee bug with the client_credentials grant where it won't allow passing through extra headers to the token request.
It includes a passing spec, although I'll be the first to admit it's a little hacky... any guidance there appreciated.
I also didn't add any documentation since it's a bugfix, and there's language to that effect under Authorization Grants but happy to do that if you'd like.
Cheers!
Nik

Coverage decreased (-0.5%) to 96.078% when pulling a66534fa6461db67d9549314bd9b86c267e6f915 on nikz:allow-passing-extra-headers-to-client-credentials into 7691a2c851a14f08c6f33356a0041d015963de01 on intridea:master.

I ran into the same issue. It is a bug!

Seems like this ought to be merged. The failures appear to be JRuby and/or Ruby 1.9.x or earlier only.

@pboling sent up #381 for you, so I'll close this one up now. Cheers!


Travis Failing

@jarrodmillman @rossbar
Hi Jarrod and Ross,
I'm not sure why this Travis is failing.  Could you help me?
https://travis-ci.org/berkeley-stat159/project-lambda/jobs/95106536
Thanks!

I think I fixed this issue. I had to update the .travis yml file @jarrodmillman or @rossbar can you confirm what I did makes sense: https://github.com/AlonDaks/project-lambda/commit/89aa9a5f1813e5ba0a55842ae4e4d7f08c0f37e9

@AlonDaks Can I close this issue?

Closed it


fix hint handle bug in subquery

question
 	_, _, err = parser.Parse("SELECT 1 FROM DUAL WHERE 1 IN (SELECT /*+ DEBUG_HINT3 */ 1)", "", "")
 	c.Assert(err, IsNil)

will be fail in current master code
why
parser are try its best to reuse yySymType (https://github.com/pingcap/parser/blob/947d4ab3052f1ca608820e8dd5dd1d8c0c838fdf/yy_parser.go#L68)
what happen in here is  /*+ DEBUG_HINT3 */ reuse 1 of 1 IN  and doesn't reset.
we should never forgot set $ = nil if we want it be nil :rofl:
fix
reset $=nil

This change is

@zz-jason @tiancaiamao  PTAL


Add xcrun extension

Description

Screencast

Checklist

[x] I read the extension guidelines
[x] I read the documentation about publishing
[x] I ran npm run build and tested this distribution build in Raycast
[x] I checked that files in the assets folder are used by the extension itself
[x] I checked that assets used by the README are placed outside of the metadata folder

Hi 👋
How is tihs compared to @SvenTiigi's xcode extension? Seems like there are some overlaps, but I'm not the target audience 🙂

Hi @pernielsentikaer ,
My extension was built to execute commands for the IOS Simulator itself using 'xcrun' which gives you full control over the simulator like renaming a simulator, getting app container, pushing notification, launching app, uninstalling app, etc...
I can see that the other extension provides functionalities that are Xcode-related like showing recent projects, creating new projects, etc..

Hi @3llomi,
The functions you are describing are available via the "Search Recent Builds" command of the Xcode Raycast extension where you can launch an app, uninstall it and also get all relevant directories as well as resetting permissions for example push notifications, location services, etc...
The "Manage Simulators" command contains additional functionality which internally are making use of xcrun.
Renmaing a simulator is currently not implemented so this would be a good PR to the Xcode extension 👍

Hi @SvenTiigi ,
Great job with the 'Search Recent Builds' command, I didn't notice it, even though I had your extension installed. probably because of the naming 😅
I can see that your extension has the most features except for the following:

Restart Simulator
I use this A LOT especially when the Simulator gets stuck for some reason.

Push Notifications using a Payload

Trigger iCloud Sync

Get app container path
I can see that your extension already have that, but it doesn't have the 'groups' directory which is a shared directory between app targets. Not sure if i'm missing something.

Let me know if you're interested in these so I can open a PR and close this.

@3llomi yes of course! Please feel free to open up a PR ✌


Correctly process tilde sign in composes url

Fixes https://github.com/webpack/css-loader/issues/254

Looks like a problem with github-codecov integration.

I don't know what I'm fixing here because I found a real reason of #254

This PR is not needed. composes use always module style imports:
composes: a from "b/x" => require("a/x").a


Add amadeus

Insert URLs to the project here: https://github.com/NIEHS/amadeus

[x] The projects is active, documented, open source licensed, shows usage from external parties and is directly targeting environmental sustainability. Find more details in the Contribution Guide.

All issues labeled as Good First Issue of the project listed on OpenSustain.tech will be visible on ClimateTriage.com. This is a great way to welcome new community members to your project.
All projects listed will be posted on our Mastodon channel

@mitchellmanware @sigmafelix


WIP Typing Notification Backend Fix #2110

WIP to add end to end tests and fix #2110 users passed to send_event() in do_send_typing_notification()

Closing this and opened https://github.com/zulip/zulip/pull/2134 instead


Path API

I came across this library when I referenced your tree_at  issue with Equinox as one of the motives for PyTreeClass existence. Since PyTreeClass now supports equinox or any Pytree, and adds functional/composable lenses-like setters/getters
Why don't you give it another shot and let me know if it solves your problem?
Looking at this library implementation, this looks very similar to the first version of PyTreeClass, which had some limitations; however, as jax now supports path API, I think it's better to migrate to their API.
For example:
import equinox as eqx
import pytreeclass as pytc
import jax

class Tree(eqx.Module):
    weight: jax.Array = jax.numpy.array([-1, 2, 3])
    bias: jax.Array = jax.numpy.array([1])
    counter: int = 1

    @property
    def at(self):
        return pytc.AtIndexer(self, ())

tree = Tree()

tree = (
    tree.at["counter"]
    .set(1)  # set counter to 1
    .at[jax.tree_map(lambda x: x < 0, tree)]
    .set(0)  # set negative values to 0
    .at["bias"].set(100)  # set bias to 100
)

print(tree.weight)
# [0 2 3]
print(tree.bias)
# 100
print(tree.counter)
# 1

Hey, I really appreciate you reaching out! Yeah I remember chatting over email a while ago about this stuff when both PyTreeClass and Zodiax were in early dev. Apologies for taking so long to get to this I have had too many things on my plate lately and have wanted to engage with this properly.
This is really cool and a bit 🪄 magic 🪄 to me right now, it seems like you've solved the eqx.tree_at syntax problem like we were describing in the original Equinox issue! I've also toyed with the idea of using the Jax path API but haven't got anywhere. I really like how you can index via a tree_map, I can see some good use cases for that!
I'm curious as to how this would extend to nested classes, where pytree leaves are other pytrees, dictionaries etc. My guess looking at this example is that you would need to nest the .at[] calls? The nesting functionality is crucial to Zodiax since the focus is scientific class structures that are typically hierarchical and modular. A common problem we want is to set parameters deep in the pytree, so we use a syntax like this:
model = model.set('telescope.optics.aberrations.coefficients')

or to operate on multiple specific leaves at the same time:
model = model.multiply(['leaf1', 'leaf2'])

I'd love your thoughts on operations like this within the PyTreeClass framework!

I also just had a look at your benchmarks against Equinox, I'm very curious as to why PyTreeClass outperforms Equinox and others! I would have thought the underlying computations would be virtually the same once compiled to XLA.

I'm curious as to how this would extend to nested classes, where pytree leaves are other pytrees, dictionaries etc. My guess looking at this example is that you would need to nest the .at[] calls?

You can use where in AtIndexer directly, something like this:

tree = {"a": {"b": 1}, "c": 2}
where = ("a", "b")
print(pytc.AtIndexer(tree, where).get())

Here each tuple item represents a matching key for each level, so ("a", "b") means match tree["a"]["b"]

or to operate on multiple specific leaves at the same time:

You can do this by using a tuple of tuples,
tree = {"a": {"b": 1}, "c": 2, "d": 3}
where = (("a", "c"),)
print(pytc.AtIndexer(tree, where).get())
# {'a': {'b': 1}, 'c': 2, 'd': None}

Here the first item in the where tuple represents the first level, and the two-item tuple means matches any of them
For set you can use .set(..., is_leaf=...) and for multiply you can use .apply(lambda x:x*number) .
For more check here and here to extend it for custom matching strategies .

I also just had a look at your benchmarks against Equinox, I'm very curious as to why PyTreeClass outperforms Equinox and others! I would have thought the underlying computations would be virtually the same once compiled to XLA.

pytreeclass is faster because it adopts a different pattern that achieves the same functionality while using less expensive python looping that is used for each filtered transformation (inside tree_map) and inside the flattening step in the equinox module.
For example, consider the example for a pytree of m leaves with  training loop ofn epochs, and let's assume we use filter_{jit,grad,vmap}, then this should result in O(2(nm)) for each filtered transformation -O(nm) for each tree_map in eqx.partition and eqx.combine- , and O(nm) for the flattening step defined in the equinox module, this will lead to a total ~ O(7(nm))
For the pytreeclass pattern, traversing is only done to undo the freezing/masking of tree leaves, so this will lead to a ~ O(nm).  Note that by filtering on pytrees not functions, I don't need to wrap jax transformations. And by using tree_mask/tree_unmask I don't need to change the function signature or partition the tree into dynamic/static parts.
The following is the complete benchmark code I used and the results on colab cpu.
Feel free to let me know if you have any questions.

import jax
import jax.numpy as jnp
import pytreeclass as pytc
import equinox as eqx
import optax

class PyTCLinear(pytc.TreeClass):
    def __init__(self, in_dim: int, out_dim: int, key: jax.random.KeyArray, name: str):
        self.name = name
        self.weight = jax.random.normal(key, (in_dim, out_dim))
        self.bias = jax.numpy.array(0.0)

    def __call__(self, x: jax.Array):
        return x @ self.weight + self.bias

class EqxLinear(eqx.Module):
    name: str
    weight: jax.Array
    bias: jax.Array

    def __init__(self, in_dim: int, out_dim: int, key: jax.random.KeyArray, name:str):
        self.name = name
        self.weight = jax.random.normal(key, (in_dim, out_dim))
        self.bias = jax.numpy.array(0.0)

    def __call__(self, x: jax.Array):
        return x @ self.weight + self.bias

def sequential_linears(layers, x):
    *layers, last = layers
    for layer in layers:
        x = layer(x)
        x = jax.nn.relu(x)
    return last(x)

x = jnp.linspace(100, 1)[:, None]
y = x**2
key = jax.random.PRNGKey(0)
optim = optax.adam(1e-3)

@jax.value_and_grad
def pytc_loss_func(layers,x,y):
    layers = pytc.tree_unmask(layers)
    y = sequential_linears(layers, x)
    return jnp.mean((x-y)**2)

@jax.jit
def pytc_train_step(layers, optim_state, x,y):
    loss, grads = pytc_loss_func(layers,x,y)
    updates, optim_state= optim.update(grads, optim_state)
    layers = optax.apply_updates(layers, updates)
    return layers, optim_state, loss

@eqx.filter_value_and_grad
def eqx_loss_func(layers,x,y):
    y = sequential_linears(layers, x)
    return jnp.mean((x-y)**2)

@eqx.filter_jit
def eqx_train_step(layers, optim_state, x,y):
    loss, grads = eqx_loss_func(layers,x,y)
    updates, optim_state= optim.update(grads, optim_state)
    layers = eqx.apply_updates(layers, updates)
    return layers, optim_state, loss

def pytc_train(layers, optim_state, x,y, epochs=100):
    for _ in range(epochs):
        layers, optim_state, loss = pytc_train_step(layers,optim_state, x,y)
    return layers, loss

def eqx_train(layers,optim_state, x,y, epochs=100):
    for _ in range(epochs):
        layers, optim_state, loss = eqx_train_step(layers,optim_state, x,y)
    return layers, loss

for linear_count in [10,100]:
    pytc_linears = [PyTCLinear(1,1, key=jax.random.PRNGKey(i), name=f"linear_{i}") for i in range(linear_count)]
    # mask non-differentiable parameters
    pytc_linears = pytc.tree_mask(pytc_linears)
    pytc_optim_state = optim.init(pytc_linears)

    eqx_linears = [EqxLinear(1,1, key=jax.random.PRNGKey(i), name=f"linear_{i}") for i in range(linear_count)]
    eqx_optim_state = optim.init(eqx.filter(eqx_linears, eqx.is_array))

    pytc_linears , pytc_loss = pytc_train(pytc_linears, pytc_optim_state, x,y, epochs=1000)
    eqx_linears, eqx_loss = eqx_train(eqx_linears, eqx_optim_state, x,y, epochs=1000)

    assert pytc_loss == eqx_loss

    time_pytc = %timeit -o pytc_train(pytc_linears, pytc_optim_state, x,y, epochs=100)
    time_eqx = %timeit -o eqx_train(eqx_linears, eqx_optim_state, x,y, epochs=100)
    print(f"Eqx/PyTc: {time_eqx.average/time_pytc.average} for {linear_count} layers")

WARNING:jax._src.xla_bridge:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
35.3 ms ± 9.48 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
186 ms ± 52.7 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
Eqx/PyTc: 5.290240573795192 for 10 layers
698 ms ± 127 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
1.74 s ± 231 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
Eqx/PyTc: 2.495322682506591 for 100 layers

Thanks for the quick response! Yeah it looks like the pytc.AtIndexer has a very similar syntax to Zodiax 'paths'. Does your AtIndexer call __getattr__ under the hood? Due to the highly nested nature of the kind of classes I work with I commonly raise attributes via dictionary keys etc through my classes so I can access attributes like optics.layers.aberrations -> optics.aberrations. I assume this would similarly work through the AtIndexer?
Interesting, thanks for this example! So if I've understood correctly you've made accessing attributes requires less tree operations, resulting in the speed gains not from the underlying computation but more efficient operation on the pytree itself? Would this result in a generic speed increase for updating pytrees generally? This has been a point in the back of my mind for a while, since in my work its often necessary to update singular values large pytrees, resulting in slow-downs from loading the pytree into a new instance. I've always thought my looping/nesting implantation of accessing leaves could be improved.
Another thing that do that breaks from the more general ML use cases is take gradients with respect to either a single, or small subset of parameters (since we are modelling physical objects, we know that things like the diameter of our telescope isn't going to change 😝). This used to be a default(ish) behaviour of Equinox's filter_grad etc, but it was removed and I had to implement a eqx.partition and eqx.combine, see here. From what you're saying it sounds like this might be able to be improved? I assume to achieve the same thing in PyTreeClass you would use a similar pattern with tree_mask and tree_unmask, can you speak to efficiency of PyTreeClass vs Equinox here?
I also see that you've implemented a equivalent of eqx.field(static=True) using the tree_mask and tree_unmask inside the functions themselves. I'm wondering if there would be a way to gain a similar syntax (specified in class definitions) without needing to modify the functions operating on the pytrees? Furthermore, is it possible using with a syntax like this to update these 'static' (typically non-jax type) parameters during calculations? I've been toying with the idea of implementing a 'unit' string parameter to my classes so that users can work in units that keep them away from float precision error (optical sciences work work with wavelength ranges from 1e-9 : 1e9), so being able to control units with a static non-jax type without requiring users to apply extra transformations in their functions has been a long-term goal.
One of the primary goals of Zodiax is to make working with the objects require as little knowledge as possible. As my target is a science audience most of the users have little to no programming background and are mostly self-taught (no wonder why most scientific code has quirks, mine included 🙃). That was a large motivator for Zodiax, to hide users from needing to learn about lambda functions, tree_maping, filtering etc. This is why all operations in Zodiax have a pure 'path based' interface, granted there is likely a loss in computational efficiency, but the gain in being able to get users up to speed with as little knowledge as possible is crucial. I'm definitely interested in some of the gains by using PyTreeClass under the hood as it seems like you've addressed quite a few things I've been thinking about!
Thanks again for these responses, and sorry for hammering you with so many questions, it seems like we're thinking about the same stuff!

Yeah it looks like the pytc.AtIndexer has a very similar syntax to Zodiax 'paths'. Does your AtIndexer call getattr under the hood?

No, AtIndexer deals with path entries defined using jax.tree_map.tree_flatten_with_path; this enables integer/string / user-defined matching strategies in addition to boolean pytrees. the file defines them is here. The path entry is converted by dispatching to a class with __eq__ defined for use in jax.tree_util.tree_map_with_path matching. This yields a mask that is used later to get/set/apply on leaves.

Interesting, thanks for this example! So if I've understood correctly you've made accessing attributes requires less tree operations, resulting in the speed gains not from the underlying computation but more efficient operation on the pytree itself?

If you mean the speed up with equinox, then this is purely due to the design of TreeClass and traversing only when needed. I think it's ok to trade speed with convenience in non-expensive paths in the program - e.g. jitted train step-

Another thing that does that breaks from the more general ML use cases is take gradients with respect to either a single, or small subset of parameters (since we are modelling physical objects, we know that things like the diameter of our telescope isn't going to change 😝).

see here

I also see that you've implemented a equivalent of eqx.field(static=True) using the tree_mask and tree_unmask inside the functions themselves.

There is a meaningful difference between the two for your use case:

First eqx.field(static=True) is defined as part of the class definition, tree_mask wraps a leaf with a wrapper that yields no leaves when flattened. It is not possible to set a field static after class creation, but you can mask a leaf with a frozen wrapper at any time [here(https://pytreeclass.readthedocs.io/en/latest/notebooks/dealing_with_non_jaxtype.html#). Similarly, you can not undo a static field after class creation, but you can undo masking a leaf with tree_umask.
static field makes sense if you don't assume a masking/filtering mechanism like in flax's PytreeNode class.

Second, eqx.field handles hashable items. Using static=True on a jax array is considered a bug, but masking an array with tree_mask will wrap the array with a frozen wrapper that can handle arrays

Third, From the design POV, the static field contradicts the mental model of equinox itself and pytreeclass, namely static field moves the burden of handling non-jaxtype (e.g. strings) to the user, instead of being handled by tree_mask or filtered transformation.

Fourth, performance, for a tree of m>>1 leaf with a single static field, the equinox module flatten step will involve a loop over m leaves with logic to eliminate the static field. For pytreeclass TreeClass, this is not happening.

I'm wondering if there would be a way to gain a similar syntax (specified in class definitions) without needing to modify the functions operating on the pytrees? Furthermore, is it possible using with a syntax like this to update these 'static' (typically non-jax type) parameters during calculations? I've been toying with the idea of implementing a 'unit' string parameter to my classes so that users can work in units that keep them away from float precision error (optical sciences work work with wavelength ranges from 1e-9 : 1e9), so being able to control units with a static non-jax type without requiring users to apply extra transformations in their functions has been a long-term goal.

I think this is XY problem. Can you give me an MWE of your desired functionality?

Ah yes I suspected that using the Jax KeyPaths might end up breaking the parameter raising, which is a bit of an issue for our use case. I wonder if its possible to build the correct path under the hood by building it from the accessors directly. This will incur some overhead though, so not an ideal solution. What are your thoughts on this?
Yeah it seems like overall PyTreeClass operates similarly to Equinox in regards to freezing parameters. The behavior in both is somewhat 'inverse' to our desired functionality. Rather than freezing specified leaves, it makes more sense for us to only take gradients with respect to specific leaves. I was able to make a small wrapper that achieves this functionality using Equinox, but it seems like it would be more complex for nested classes in PyTreeClass.

Using static=True on a jax array is considered a bug, but masking an array with tree_mask will wrap the array with a frozen wrapper that can handle arrays

Does this mean that Jax will understand these array leaves as 'static' under jit transformations? They obviously can't be treated as actually static since they aren't hashable, but does this lead to a meaningful difference under the hood? We have always though that speeds might be improved by preventing Jax tracing these arrays that can be considered 'static'.
What are your thoughts on a filtered-transformation equivalent for jit and grad with a where mask that automatically masks out every leaf not in the where function? I've built an equivalent using Equinox here. Do you suspect there would be any meaningful difference under the hood once compiled to XLA, as opposed to the partition -> combine pattern used in Equinox?

I think this is XY problem. Can you give me an MWE of your desired functionality?

Yes it more or less is. This is not necessarily needed functionality, but more an idea I've been toying with as a way to define and update some units parameter of leaves or pytrees. Here's a MWE
from jax import Array
import pytreeclass as pytc

class Foo(pytc.TreeClass):
    leaf : Array
    unit : str = pytc.static_leaf() # Something similar to this

    def __call__(self, new_unit: str):
        new_value = convert(self.leaf, self.unit, new_unit) # some simple unit conversion function
        return self.at[('leaf', 'unit')].set((new_value, new_unit))

The idea here is that since the leaves are already treated as 'static' you avoid the need to mask/unmask in functions.
The other option would be a filtered-transform type operation:
from jax import Array
import pytreeclass as pytc

class Foo(pytc.TreeClass):
    leaf : Array
    unit : str

    def __call__(self, new_unit: str):
        new_value = convert(self.leaf, self.unit, new_unit) # some simple unit conversion function
        return self.at[('leaf', 'unit')].set((new_value, new_unit))

param = 'leaf'
@filter_jit(param)
@filter_grad(param)
def f(pytree):
    return pytree('nm').leaf**2

The idea here is that the filtered transformation automatically masks all leaves except for the leaf parameter, by building the mask programmatically from the param input. This pattern is actually more ideal than the first example since it alleviates the need for users to think about it in the first place and relaxes the need to be more considered about input array dtypes.
Generally, the goal is to have 'mask-by-path' behavior as this is more in-line with the goals of scientific models.

Yeah it seems like overall PyTreeClass operates similarly to Equinox in regards to freezing parameters. The behavior in both is somewhat 'inverse' to our desired functionality. Rather than freezing specified leaves, it makes more sense for us to only take gradients with respect to specific leaves.

Its easier with pytreeclass,
import pytreeclass as pytc
import jax.numpy as jnp
import jax

class Tree(pytc.TreeClass):
    def __init__(self):
        self.weight = jnp.array([1.0])
        self.bias = jnp.array([1.0])
        self.name = "tree"  # non-jax type

tree = Tree()

# mask all leaves except weight
mask = tree.at[...].set(True).at["weight"].set(False)
masked_tree = pytc.tree_mask(tree, mask)

@jax.jit
@jax.grad
def loss_func(tree, x: jax.Array):
    tree = pytc.tree_unmask(tree)
    return jnp.mean(tree.weight * x**2)

grads = loss_func(masked_tree, jnp.array([2.0]))
# Tree(weight=[4.], bias=#[1.], name=#tree)

new_tree = jax.tree_map(lambda x, g: x + g, masked_tree, grads)
new_tree = pytc.tree_unmask(new_tree)
# Tree(weight=[5.], bias=[1.], name=tree)

Does this mean that Jax will understand these array leaves as 'static' under jit transformations?

Yes, freeze or tree_mask wrap arrays with a wrapper that defines a hash rule, something similar to this

What are your thoughts on a filtered-transformation equivalent for jit and grad with a where mask that automatically masks out every leaf not in the where function? I've built an equivalent using Equinox here.

Its harder using equinox style because you have to 1) filter pytrees to work with libraries like optax and then you have to 2) filter over functions. This is a common usability issue in equinox.
From your code, this means that you have to supply the boolean filter explictly to eqx.filter before you supply it to optax, i.e. you are repeating the process twice, once explicitly and other implicitly.
In contrast, using the above code sample, the masked_tree can be supplied to optax with no further operations.

The idea here is that since the leaves are already treated as 'static' you avoid the need to mask/unmask in functions.

If you want to take the burden of defining static nodes inside the class definition, then the following pattern works :

import jax
import pytreeclass as pytc

# changing a "frozen" leaf value without tree_mask/tree_unmask on user side

class Foo(pytc.TreeClass):
    def __init__(self, leaf: jax.Array, unit: str):
        self.leaf = leaf
        self.unit = pytc.freeze(unit)  # non-jax type

    def change_unit(self, new_unit: str) -> "Foo":
        return self.at["unit"].set(pytc.freeze(new_unit), is_leaf=pytc.is_frozen)

foo = Foo(jax.numpy.array([1.0]), "m")
# Foo(leaf=f32[1](μ=1.00, σ=0.00, ∈[1.00,1.00]), unit=#m)
new_foo = foo.change_unit("cm")
# Foo(leaf=f32[1](μ=1.00, σ=0.00, ∈[1.00,1.00]), unit=#cm)

Your MWE is a strong case for using pytreeclass over equinox, because in equinox if you define unit as static field, its not possible to change it afterward immutably.
Please let me know what you think.

Okay so I've had some time to go through the PyTreeClass docs and... wow you have implemented some seriously awesome functionality! Automatic data type and shape checking, bcmap, default frozen leaves, these all could be very useful in my downstream software. I'm very seriously considering migrating Zodiax to a PyTreeClass backend, so I hope my questions aren't too annoying 😝.
I can envision a better version of Zodiax that circumvents the need for wrapping specific Equinox function, using a single default filter function wrapper that automatically masks and unmasks pytrees before being passed to the original function. Ideally this could be used to make all pytree inputs 'safe' for Jax functions, with syntax like this:
params = ['param1', 'param2']

@jax.jit
@jax.grad
@zdx.filter(params)
def loss(pytree):
    return pytree()**2

Ideally this would result in gradients only with respect to the params list, with all others parameters treated as static under jit. I had a quick stab at implementing this but couldn't quite get it to work:
import jax
import jax.numpy as np
from functools import wraps
import pytreeclass as pytc
import equinox as eqx
import zodiax as zdx

# Functional wrapper to build the mask using __getattr__, not super important right now
def set(pytree, parameters, values):
    if values is None:
        values = [None]
        if isinstance(parameters, str):
            parameters = [parameters]
    new_parameters, new_values = zdx.base._format(parameters, values)
    leaves_fn = lambda pytree: zdx.base._get_leaves(pytree, new_parameters)
    return eqx.tree_at(leaves_fn, pytree, new_values, is_leaf = lambda leaf: leaf is None)

# Simple class
class Foo(pytc.TreeClass):
    x : None
    y : None

    def __init__(self, x, y):
        self.x = np.asarray(x, float)
        self.y = np.asarray(y, float)

    def __call__(self):
        return self.x * self.y

foo = Foo(1, 2)
foo

>>> Foo(x=f32[](μ=1.00, σ=0.00, ∈[1.00,1.00]), y=f32[](μ=2.00, σ=0.00, ∈[2.00,2.00]))

And now some code to automate the masking/unmasking
def filter(params):
    def decorator(func):
        def wrapper(pytree, *args, **kwargs):

            # Generate Mask
            false_mask = pytree.at[...].set(True)
            bool_mask = set(false_mask, params, False)
            masked_tree = pytc.tree_mask(pytree, bool_mask)

            # Generate unmasked func
            def unmask(masked):
                unmasked = pytc.tree_unmask(masked, bool_mask)
                return func(unmasked, *args, **kwargs)

            # Mask and call wrapped function
            return unmask(masked_tree)

        return wrapper
    return decorator

params = 'x'

@jax.jit
@jax.value_and_grad
@filter(params)
def loss_fn(pytree):
    return np.abs(pytree())**2

loss, grads = loss_fn(foo)

print(loss)
print(grads)

>>> 4.0
>>> Foo(x=8.0, y=4.0)

Which is still taking gradients with respect to all parameters. I assume this is simply to do with my construction of the wrapper function? Although it seems like this construction might not work either way, since the parameter passed into the jax transformations will always be the unmasked pytree, with the masking and unmasking happening downstream. I would prefer not to need to write specific wrappers to individual transformations like jit, grad etc. Although it may be possible to write a single wrapper that applies to all jax functions as a good compromise.
What are your thoughts on a generalized filter function that can wrap any pytree input based on a mask generated by a list of parameters?

Another question I have is when the callbacks of pytc.field are invoked? From the provided examples only show it being called during the constructor. I did some testing and this seems to be the case. It would be super handy if these callbacks could be invoked during calls to at[where].set(value), since its always been a bit of a 'loose end' that leaves can be set to anything of any dtype. It would be great to build classes with strict conditions on leaves, what are your thoughts on this?

Okay so I've had some time to go through the PyTreeClass docs and... wow you have implemented some seriously awesome functionality! Automatic data type and shape checking, bcmap, default frozen leaves, these all could be very useful in my downstream software. I'm very seriously considering migrating Zodiax to a PyTreeClass backend, so I hope my questions aren't too annoying 😝.

I am glad you liked it :)

Another question I have is when the callbacks of pytc.field are invoked? From the provided examples only show it being called during the constructor.

Yes, field is just a descriptor, whose __set__ method invokes the callbacks. this design choice makes field independent of autoinit, i.e. can work with normal class ( i added a note with example in the docs for this point

I did some testing and this seems to be the case. It would be super handy if these callbacks could be invoked during calls to at[where].set(value), since its always been a bit of a 'loose end' that leaves can be set to anything of any dtype. It would be great to build classes with strict conditions on leaves, what are your thoughts on this?

callbacks operate on a different level than .at[...].set, the former during __set__ method of the descriptor and the latter during tree_map, to make this work, the unflatten method should be changed to reconstruct the class everytime.
The following example defines a strict pytree that executes the callbacks with each unflatten step.
import pytreeclass as pytc
import jax
import jax.tree_util as jtu
import jax.numpy as jnp

def array_or_error(x):
    assert isinstance(x, jax.Array)
    return x

@jtu.register_pytree_node_class
class Tree:
    x: jax.Array = pytc.field(callbacks=[array_or_error])

    def __init__(self, x):
        self.x = x

    def tree_flatten(self):
        return (self.x,), None

    @classmethod
    def tree_unflatten(cls, aux_data, children):
        return cls(*children)

    @property
    def at(self):
        return pytc.AtIndexer(self)

tree = Tree(jnp.ones([10, 10]))

tree.at['x'].set(1.)
# AssertionError: On applying array_or_error for field=`x`:

Unlike dataclass-based pytrees like flax pytreenode, or equinox, leaves could be added to TreeClass instance, so cls(*children) will not work and will through an error if a leaf is removed or added.
So it's not possible to change the Behavior without losing this feature.
I will respond to the rest As soon as possible,
Let me know if this helps

Okay so I've had some time to go through the PyTreeClass docs and... wow you have implemented some seriously awesome functionality! Automatic data type and shape checking, bcmap, default frozen leaves, these all could be very useful in my downstream software. I'm very seriously considering migrating Zodiax to a PyTreeClass backend, so I hope my questions aren't too annoying 😝.

I am glad you like it :)

I can envision a better version of Zodiax that circumvents the need for wrapping specific Equinox function, using a single default filter function wrapper that automatically masks and unmasks pytrees before being passed to the original function. Ideally this could be used to make all pytree inputs 'safe' for Jax functions, with syntax like this:

I think my point will be clear if you write a MWE using zodiax with optax to update some leaves and ignore the rest?

Another question I have is when the callbacks of pytc.field are invoked? From the provided examples only show it being called during the constructor. I did some testing and this seems to be the case. It would be super handy if these callbacks could be invoked during calls to at[where].set(value), since its always been a bit of a 'loose end' that leaves can be set to anything of any dtype. It would be great to build classes with strict conditions on leaves, what are your thoughts on this?

field returns a simple descriptor, and the callbacks are invoked on the descriptor's __set__ method. This descriptor can be used with any python class to apply callbacks on setattr. Since .at[...].set() use tree_map, then I have to redefine the unflatten rule to instantiate the class. However, this has a major downside, unlike frozen dataclass- based pytrees like flax.struct.PyTreeNode or equinox.Module, leaves can be added/removed to TreeClassafter instantiation functionally, implementing this feature will mean I can no longer add/remove leaves.

I think my point will be clear if you write a MWE using zodiax with optax to update some leaves and ignore the rest?

Yeah so my 'ideal' functionality would be something like this
import jax
import jax.numpy as np
import pytreeclass as pytc
import zodiax as zdx
import optax

# Simple class
class Foo(pytc.TreeClass):
    x : float
    y : float

    def __init__(self, x, y):
        self.x = np.asarray(x, float)
        self.y = np.asarray(y, float)

    def model(self):
        return self.x * self.y

# Construct object
tree = Foo(1, 2)

# Define parameters to optimise
params = 'x'

# Use zodiax to automate mapping of optimisers to leaves
optim, opt_state = zdx.get_optimiser(tree, params, optax.adam(1))

# Define loss and apply a 'filter' to handle the masking/unmasking
@jax.jit
@jax.value_and_grad
@zdx.filter(params)
def loss_fn(pytree):
    return np.abs(pytree.model())**2

loss, grads = loss_fn(tree)

# apply the update
updates, opt_state = optim.update(grads, opt_state)
updated_tree = zdx.apply_updates(tree, updates)

I can automate the mapping of optax optimisers to leaves so don't worry about that, the main thing is the zdx.filter(params). The idea is that it would transform the pytree input of the function it is decorating, automatically masking and unmasking such that all leaves not specific by params are 'masked' out of the pytree so that it can be passed to jax transformations.
From my testing of this it seems like that may not be possible since the masking and unmasking need to wrap around the calls of jax.jit, jax.value_and_grad. I could accept a syntax similar to equinox (and zodiax) at present:
@zdx.filter_jit(params)
@zdx.filter_value_and_grad(params)
def loss_fn(pytree):
    return np.abs(pytree.model())**2

But the former example is definitely more ideal since the masking/unmasking would only be done once. Do you think something like that is possible?

field returns a simple descriptor, and the callbacks are invoked on the descriptor's set method. This descriptor can be used with any python class to apply callbacks on setattr. Since .at[...].set() use tree_map, then I have to redefine the unflatten rule to instantiate the class. However, this has a major downside, unlike frozen dataclass- based pytrees like flax.struct.PyTreeNode or equinox.Module, leaves can be added/removed to TreeClassafter instantiation functionally, implementing this feature will mean I can no longer add/remove leaves.

Yeah I definitely like the idea of being able to add and remove leaves, however I think there wouldn't be too many use-cases for Zodiax classes. It might even be worth having two base classes for Zodiax, a 'frozen' and 'unfrozen' one that users can then choose which they prefer.
If I was to implement this 'frozen' version, can you speak as to how it would integrate with the rest of PyTreeClasss? If everything still works I would consider making something like this.

For this use case, AFAIK I think you can do it - IMO- in a simpler way:
Method 1: using optax masked
import pytreeclass as pytc
import jax.numpy as jnp
import optax

@pytc.autoinit
class Tree(pytc.TreeClass):
    a: float = jnp.array(1.0)
    b: float = jnp.array(2.0)
    name: str = "tree"  # non-jax type for demonstration

    def model(self):
        return self.a + self.b

tree = pytc.tree_mask(Tree())

optim = optax.chain(
    # update only "a"
    optax.masked(optax.sgd(2.0), mask=tree.at[...].set(False).at["a"].set(True)),
    # no updates to the rest of the tree
    optax.masked(optax.sgd(0.0), mask=tree.at[...].set(True).at["a"].set(False)),
)

optim_state = optim.init(tree)

def loss_func(tree):
    ypred = tree.model()
    return jnp.mean((ypred - y) ** 2)

@jax.jit
def train_step(tree, optim_state):
    grads = jax.grad(loss_func)(tree)
    updates, optim_state = optim.update(grads, optim_state)
    tree = optax.apply_updates(tree, updates)
    return tree, optim_state

def train(tree, optim_state, num_steps):
    for _ in range(num_steps):
        tree, optim_state = train_step(tree, optim_state)
    return tree, optim_state
tree, optim_state = train(tree, optim_state, num_steps=10)
# tree = pytc.tree_unmask(tree)
print(tree)
# Tree(a=148686.31, b=2.0, name=tree)

%timeit train(tree, optim_state, num_steps=1000)
# 9.2 ms ± 134 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

Some notes:

I did not use tree_unmask inside the loss_func, because your function call did not use non-jaxtype leaves (e.g. activation function)
Gradients were taken with respect to b, but no update was done because we use optax.masked with sgd(0.)

Method 2
import pytreeclass as pytc
import jax.numpy as jnp
import optax

@pytc.autoinit
class Tree(pytc.TreeClass):
    a: float = jnp.array(1.0)
    b: float = jnp.array(2.0)
    name: str = "tree"  # non-jax type for demonstration

    def model(self):
        self = pytc.tree_unmask(self)
        return self.a + self.b

optim = optax.sgd(2.0)
tree = pytc.tree_mask(Tree(), mask=Tree(a=False, b=True, name=True))
optim_state = optim.init(tree)

def loss_func(tree):
    ypred = tree.model()
    return jnp.mean((ypred - y) ** 2)

@jax.jit
def train_step(tree, optim_state):
    grads = jax.grad(loss_func)(tree)
    updates, optim_state = optim.update(grads, optim_state)
    tree = optax.apply_updates(tree, updates)
    return tree, optim_state

def train(tree, optim_state, num_steps):
    for _ in range(num_steps):
        tree, optim_state = train_step(tree, optim_state)
    return tree, optim_state

tree, optim_state = train(tree, optim_state, num_steps=10)

print(tree)
# Tree(a=148686.31, b=#2.0, name=#tree)

%timeit train(tree, optim_state, num_steps=1000)
# 5.91 ms ± 121 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

Note:

The gradient with b was not taken because it was frozen.
Did not need optax.masked
Needed tree_unmask because b was frozen, but I moved tree_unmask to the model method as a variation e.g. if you don't want it in the loss function.

So, I wanted to point out that with these two examples the following:

If you use optax.masked, then no need for filter(params)
If you don't use optax.masked, then you will need filter(params)

From the zodiax code, I see you are using optax, so I would recommend using method 1 for your use case. I personally use method 1 for my use case (e.g. min-max etc, ...)

The idea is that it would transform the pytree input of the function it is decorating, automatically masking and unmasking such that all leaves not specific by params are 'masked' out of the pytree so that it can be passed to jax transformations.

masking can be used to hide/freeze a leaf, This can be useful to either: 1) pass non-jaxtype or 2) not taking gradients with respect to that leaf. optax can achieve 2) but not 1) , however, I would recommend using optax for 2)  -like method 1- and use masking for 1).

If I was to implement this 'frozen' version, can you speak as to how it would integrate with the rest of PyTreeClasss? If everything still works I would consider making something like this.

TreeClass is frozen in the same sense as dataclasses -i.e. can not change attributes mutably-,My point was that leaves in frozen dataclass based pytrees are class variables - i.e. can not be changed on instance level-
But you can change these attributes immutably in TreeClass because leaves are the instance attributes.
like the following:
import pytreeclass as pytc
@pytc.autoinit
class Tree(pytc.TreeClass):
    a: float = 1.0
    def add_one(self):
        self.a += 1.0

tree= Tree()
tree.add_one()  # AttributeError: Cannot set attribute value=2.0 to `key='a'` ...
_, new_tree = tree.at["add_one"]()
print(new_tree)
# Tree(a=2.0)

Note that new_tree and tree have different leaf values but the same methods; more importantly, no in-place mutation happened.

Check the new release; the new features are inspired by our discussion here.

 I've done some testing of using PyTreeClass on the back-end, but I've been busy with a re-factor of a downstream library of time. I probably wont have time to put towards this in the near future, but there's definitely features I will integrate soon!


Customization?

Hi! First, thanks for the awesome plugin!
I noticed that there's no way to override the commands used for the different shortcuts (if there's I wasn't able to find it).
For example, in order to find a work I prefer to use Ag instead of Rg.
Another example could be customizing the popup that appears when using fzf.
Are there any plans on adding this possibilities?

@Tintef  It can be support use ag instead of rg in fzf.  I currently use floatwindow for fzf. Are you dissatisfied with it?

@glepnir How would I change this plugin to use ag instead of rg? The only way I found to it was to manually change it here.
I like the floatwindow, and it does looks nice. I have used it in the past but I found it a bit intrusive on my workflow, so I prefer to have it like this:

@Tintef  Ok I got your problem. I will add it support later.

@Tintef  Please update and give it a test. I add two option let g:dashboard_fzf_float default is 1.disable is 0
let g:dashboard_fzf_engine default is rg, set ag.
let g:dashboard_fzf_float = 0
let g:dashboard_fzf_engine = 'ag'

@glepnir The engine works great!
Disabling the floatwindows thorws this errors when trying to find a file/word:
line    6:
E117: Unknown function: s:snr
E116: Invalid arguments for function get
E15: Invalid expression: get(s:, 'snr', s:snr())
line    8:
E121: Undefined variable: s:snr
E15: Invalid expression: {'window': 'call '..s:snr..'fzf_window(0.9, 0.6, "Comment")'}
line   88:```

Ok.Please update. Maybe fixed.

Hey any update?

I will close this issue, Because doesn't got reply long time. If you still have problem , I will reopen this issue.


[BUG] escli profiles add throw no such file or directory error

I followed the installation instructions written in README.md.
I've successfully installed the binary then I tried to add a new profile with following command.
$ escli profiles add
open /Users/<my-username>/.escli/config.yaml: no such file or directory

To Reproduce
$ brew tap devopsartfactory/devopsart
$ brew update
$ brew install escli
$ escli profiles add

$ escli version
0.0.4

Ok. I will check it. Thank you for your uploading issue.


DBZ-1020 Refactor UnwrapFromMongoDbEnvelope logic

Now it's easier to follow what makes an insert, update or delete within the transformer, also reduced the amount of npath complexity by doing early returns and certain extractions.
I think  it's better to review by reading the new code, the diff is quite messy

LGTM. Thanks a lot for doing some clean-up, @renatomefi!


Docking

Hello Sam,
I updated imgui in order to use the new docking' branch, which adds docking windows.
Basically you now have three different possible layouts:

And the docking layout is created inside babylon_imgui\run_scene_with_inspector.cpp:

void CreateDockingLayout_BabylonCpp(ImGuiID fullDockSpaceId)
{
    ImGuiViewport* viewport = ImGui::GetMainViewport();
   ...
   ...
    ImGuiID dock_id_bottom = ImGui::DockBuilderSplitNode(dock_main_id, ImGuiDir_Down, 0.25f, NULL, &dock_main_id);
    ImGuiID dock_id_left   = ImGui::DockBuilderSplitNode(dock_main_id, ImGuiDir_Left, 0.25f, NULL, &dock_main_id);
    ImGuiID dock_id_right  = ImGui::DockBuilderSplitNode(dock_main_id, ImGuiDir_Right, 0.45f, NULL, &dock_main_id);
   ...
   ...
    ImGui::DockBuilderDockWindow(_viewElementsLabels.at(ViewElements::SamplesCodeViewer).c_str(), dock_main_id);
    ImGui::DockBuilderDockWindow(_viewElementsLabels.at(ViewElements::SampleBrowser).c_str(), dock_id_left);
    ImGui::DockBuilderDockWindow(_viewElementsLabels.at(ViewElements::Scene3d).c_str(), dock_id_right);

}

If you want to add new elements to the layout, just remember that you just need to create new windows with ImGui::Begin and to set their initial location inside CreateDockingLayout_BabylonCpp.
I also renamed SamplesRunner to  BabylonStudio, as it as grown quite a bit.
I updated the demo video with the new docking look, see https://www.youtube.com/watch?v=o05_5Wyzv54&feature=youtu.be
(You might want to update the video link in your comment in the imgui gallery, as the docking branch is the active development branch for gui).
Cheers!
What do you think?

Hello Pascal,
Thanks for this nice improvement and the new video!
Having the ability to undock the scene view and make it bigger or change the layout of the windows was something missing in the previous SampleRunner. I really like the pre-defined layouts (i.e. the scene workspace or dev workspace).
Maybe we could set the default view to a cleaner layout, (i.e. the scene view next to the sample selector (like it is now) but hiding the inspector), because now the user interface shows a lot of tabs with information in the default view? Having a menu bar or toolbar might be useful when add more functionality in the future or could be used to switch between layouts. Probably we have to use the BabylonStudio for some time to find out what works best.
When switching between scenes I am getting a weird glitch in the scene view (as illustrated in the animated gif below), do you also see this on your side?

Cheers!


Modified fizzbuzz.livemd

I have added the Tom Scott Fizz Buzz video to the FIzz Buzz Exercise.

I have made all the changes you mentioned

Thank you!


Fix certificate permissions with "group" option

The default permissions are 0600, so that certificates are inaccessible
to the specified group. Add group read permission in that case, so that
this actually works.
Test this for both local certmonger and IPA.

I tested this locally against a Fedora 34 cloud image.

Failed and pending (which actually failed) tests seems to fail due to infra issues. I'm ok on merging this PR.
@richm what do you think?


GuestConfiguration CLI initial checkin

This checklist is used to make sure that common guidelines for a pull request are followed.

For new extensions:

[x] My extension description/summary conforms to the Extension Summary Guidelines.

About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your PR is merged into master branch, a new PR will be created to update src/index.json automatically.
The precondition is to put your code inside this repo and upgrade the version in the PR but do not modify src/index.json.

If this PR is for a new extension or change to an existing extension, use the following to try out the changes in this PR:
docker run -it microsoft/azure-cli:latest
export EXT=<NAME>
pip install --upgrade --target ~/.azure/cliextensions/$EXT "git+https://github.com/gehuang-dut/azure-cli-extensions.git@master#subdirectory=src/$EXT&egg=$EXT"

GuestConfiguration

The style check error is caused by:
"Running flake8 on CI scripts...
The following directories are missing codeowners ['/src/guestconfig/'].
Pylint OK.
Other Static checks...
Errors whilst verifying /home/vsts/work/1/s/.github/CODEOWNERS!"
I don't have permission to edit the .github file. Other checks are all green. Thanks.

Hi @gehuang-dut , to me the CODEOWNERS is just a normal file: https://github.com/Azure/azure-cli-extensions/blob/master/.github/CODEOWNERS
Could you modify and commit it?

Hi @gehuang-dut , to me the CODEOWNERS is just a normal file: https://github.com/Azure/azure-cli-extensions/blob/master/.github/CODEOWNERS
Could you modify and commit it?

Ok, it's a mistake at my side. Fixed. Green now. Thanks.


stack.yaml: Use version from Hackage rather than Github

Also sync cabal.project with stack.yaml.

bors r+


Added Login and Register Page

I have created two pages "Login" and "Register". Accessible from /login and /register.

The color is turned to white.

Hopefully change the design as designed

Added new login screen

Added new register screen

Design feedbacks according to figma:

header title "System" should be in next line and in center
place holders are not given
Fonts colors are not matching
no picture in right side
gap between the login and google sign in button not correct
i agree not aligned . highlighted texts not underline
on register page thana, org etc input dropdowns and placeholders not added

Resolved as said

Lastly , Resolve conflicts

Please remove the old pics and upload updated pics or video so that we can look at the updated design

Updated in the first comment @Shabab007 bhai

@KMSZamil  design looks good .

only top margin or space of the title not matching . in figma there is much space on the top compared to your pic
Please add email and mobile number validation for register input. if you need help let me know

@Shabab007 bhai, Updated as requested


Lime/Openfl better integration

Small changes for a working openfl/lime support.
The one that might look wierd is : flash.ui.Keyboard.BACK thing, but openfl has named that key BACKSPACE....
hence the keycode instead.
also, openfl being used in parallel with flash, some changes to conditional conds are made.

BACK != BACKSPACE : Back is the Android Back key :)

Ok my mistake.
We are currently moving that pull request forward.
Our goal is to have a robust lime integration  (no openfl as much as possible, to avoid another dependency).
I'll let you know when the pull request is complete now that ppedidont did some other changes.

Current branch does have a correct lime integration.
Note :

Lime CPP tested on Windows, Linux, IOs and Android  (more info below concerning JS)

Assets:
Lime Assets integrated with hxd.Res system using hxd.Res.initLime
Lime xml project format should however use forced binary format for its assets:
Example :

Events / (Multi) Touch / Keys

FontBuilder

Missing:

JS target is currently not working properly.  We do not use that target so far, so we will not investigate a lot, but it compiles correctly.
(Error being : Unable to get property 'pixelStorei' of undefined or null reference     would need to check DCE or compilation flags)

JS target does require a <canvas id="webgl"/> element on the webpage, but it should error if not found. If you look at h3d/imp/GlDriver.hx I don't see how this could happen.

Regarding the PR : everything seems fine, except the changes h2d.Text : I don't want to use the haxe.Utf8 API because HL uses native UCS2 text (same as JS) which is not the same as UTF-8. I guess we need to define an heaps-specific abstract to handle that for now, until Haxe has proper std Unicode support.

I have removed calls to haxe.Utf8 from h2d.Text and created a new abstract in hxd.

Do you still need the fixMult part ? It should have been fixed for a quite long time already.

Nevermind, it was simply moved code.

Thanks!


Do not use Pstore binaries in rspec: from_xml

I have changed the way that organisation names are stored in iso-bib-item (they are now arrays of strings rather than single strings), but I find that rspec on relaton can't deal with that, because the mocks for bibliography fetches are accessing Pstore binaries of the caches under spec/support—which I can't readily update to the new internal structure. This is not maintainable, since the internal structure of iso-bib-item is likely to change.
To deal with this, I request that we use from_xml to build the contents of the Pstore dynamically, as we had it before. That way, any updates to iso-bib-item can be transparently migrated forwards without having to update binary files.
So instead of
    file = "spec/support/" + args[0].downcase.gsub(/[\/\s-]/, "_")
    file += "_#{args[1]}" if args[1]
    store = PStore.new file

I'd like to see
    file = "spec/support/" + args[0].downcase.gsub(/[\/\s-]/, "_")
    file += "_#{args[1]}" if args[1]
    file += ".xml"
    bibitem = File.read(file)
    store = PStore.new "tmp.pstore"
    store.transaction do
      store[:version] = VERSION
      store[args[0]] = IsoBibItem.from_xml bibitem
    end

with the XML files restored to the support directory.
Open to other suggestions, including a Pstore builder script, but I think this is the most maintainable alternative.

@opoudjis I don't understand the problem. The mock get IsoBibliographicItem form PStore if a file exists. If not then IsoBibliographicItem fetched as in production and stored in PStore. If IsoBibliographicItem is changed then we should just remove the PStore file and it will be recreated with a new structure.

OK, that I didn't realise (although I did notice that design pattern elsewhere in your code). I'll confirm tomorrow, and close the ticket; going to bed now. :)

Yup, sorry about that. Fixed.

BTW, I do not think we should ever have binaries in RSpec, and we should not use a binary cache (at this moment), we should favor a file-based cache like this:
https://github.com/riboseinc/rfc-asciidoc-rfc/tree/master/references/informative


Remove the help text introduced in patch 8.2.3219

This help text describes the behavior before that patch, not the behavior after that patch.

Looking back at the PR, it says to include "what is needed", so it was one or the other.
It can actually be helpful to mention that when "../include" is not found then that entry is not used.


Resolver suggests not yet defined variables

Describe the bug
Right now completion suggests the variable that I'm currently writing, and other variables that may not yet have been written to me:

Steps to reproduce

Open test.bpmn
Select task
Complete on input variables foo and bar
See that the variable itself, but also the variable that has not been written yet (bar if you complete on foo) is suggested

Expected behavior
Only actually available variables are completed.
Environment

OS: any
Camunda Modeler Version: 5.9.0
Execution Platform: Camunda Cloud
Installed plug-ins: None

Additional context
No response

@marstamm Is this something we would want to address (locally) in the next iteration of our tooling?

I'm all for picking this up, as it is confusing to the user.
However, this is not as trivial as hiding variable names defined in the Task, as loops can be present in bpmn diagrams:

In these cases, it is unclear if the variable is already defined or not. If we want to deviate from just using the variable scope as defined in the bpmn-spec, we should have a proper discussion about what the behavior should be

@marstamm Moved to backlog as you suggested this is not an easy pick.

Considering our overall strategy of "Suggest less and only what's safe", I think we can revisit this one for a more user friendly but not 100% correct fix:

If the variable is written to in the current element, filter it out if it was not written to already (in this element)

E.g., respect the output ordering for the local scope and do not care about global variables

This would lead to less confusion. Note that this would create some cases where we either suggest too much or too little:

Variables first written in a later Task are still suggested
overwrite of existing variables (foo = foo + 5) would not suggest foo. We still show it when there are multiple sources for foo, but that again leads to weird behavior when working with loops

Still, I think this could get rid of some confusion when getting the variable name you are currently writing to suggested

Given https://github.com/camunda/camunda-modeler/issues/3510#issuecomment-1515049606, I feel confident that we can improve the user experience, even when are not 100% correct for all cases. Moving to ready

Fixed upstream via https://github.com/camunda/camunda-bpmn-js/pull/277.

Just noticed that this is still a problem in 5.16.0, reopening

Since it wasn't fixed properly in the milestone M65, I moved it to the current M70.


Missing task for increasing a number in a variable i build

I found that "update build variable" could be used for increasing a variable in a release. I am looking for a task where I can increase the value of a variable in build. I want to use that variable as a buildcounter which should be part of my version number for the build.

There is a built in variable for just that $(Rev:.r), this will increase each time you run a build. There is also $(Build.BuildId) which should increment automatically but is shared across the whole of TFS/VSTS so won't be a linear progression if you've got multiple builds.
As an example I use this format for some of my builds:
$(TeamProject)_$(BuildDefinitionName)_$(SourceBranchName)_$(Date:yyyyMMdd)$(Rev:.r)
This gives me something like:
MainProject_TestBuild_Dev_20170822.1 for the first build and then that last 1 will increment every time I run that build today.
There are more variables available here that could prove useful.

Hi Chris, thank you for the quick answer.
I am going from team city builds to VS Team System builds. I want to be able to track how many builds has been made since the start, so I want to have a build counter which I can set to whatever it was in Team City. When I increase major and minor, I dont whant the buildnumber to start over, which is the case with Rev:.r

I will get the manifest altered so you can use the task in a build as well as a release

Release fixed in 1.5.x

Hi, I have made these settings for my build step

And I get the following build error. See the log below:

017-08-24T09:30:50.5139645Z ##[section]Starting: Update Build Variable
2017-08-24T09:30:50.5139645Z ==============================================================================
2017-08-24T09:30:50.5139645Z Task         : Set variable on a build defintion
2017-08-24T09:30:50.5139645Z Description  : Set variable on a build defintion
2017-08-24T09:30:50.5139645Z Version      : 1.5.6
2017-08-24T09:30:50.5139645Z Author       : Richard Fennell
2017-08-24T09:30:50.5139645Z Help         : Version: 1.5.6. More Information
2017-08-24T09:30:50.5139645Z ==============================================================================
2017-08-24T09:30:50.5149650Z Preparing task execution handler.
2017-08-24T09:30:50.9799675Z Executing the powershell script: d:\a_tasks\BuildVariableTask_115030de-9ced-4d2f-ad01-8c42d20d0327\1.5.6\BuildVariableTask.ps1
2017-08-24T09:30:55.9092277Z ##[error]System.Management.Automation.MethodInvocationException: Exception calling "UploadString" with "3" argument(s): "The remote server returned an error: (403) Forbidden." ---> System.Net.WebException: The remote server returned an error: (403) Forbidden.
2017-08-24T09:30:55.9092277Z    at System.Net.WebClient.UploadDataInternal(Uri address, String method, Byte[] data, WebRequest& request)
2017-08-24T09:30:55.9092277Z    at System.Net.WebClient.UploadString(Uri address, String method, String data)
2017-08-24T09:30:55.9092277Z    at CallSite.Target(Closure , CallSite , Object , Object , String , Object )
2017-08-24T09:30:55.9092277Z    --- End of inner exception stack trace ---
2017-08-24T09:30:55.9092277Z    at System.Management.Automation.ExceptionHandlingOps.ConvertToMethodInvocationException(Exception exception, Type typeToThrow, String methodName, Int32 numArgs, MemberInfo memberInfo)
2017-08-24T09:30:55.9092277Z    at CallSite.Target(Closure , CallSite , Object , Object , String , Object )
2017-08-24T09:30:55.9092277Z    at System.Dynamic.UpdateDelegates.UpdateAndExecute4[T0,T1,T2,T3,TRet](CallSite site, T0 arg0, T1 arg1, T2 arg2, T3 arg3)
2017-08-24T09:30:55.9092277Z    at System.Management.Automation.Interpreter.DynamicInstruction`5.Run(InterpretedFrame frame)
2017-08-24T09:30:55.9092277Z    at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
2017-08-24T09:30:55.9272257Z ##[error]PowerShell script completed with 1 errors.
2017-08-24T09:30:55.9272257Z ##[section]Finishing: Update Build Variable

That is a rights issue on the build account, check the wiki https://github.com/rfennell/vNextBuild/wiki/BuildTasks-Task

I just saw it. Sorry for not reading first! :)


Support for podIP and targetPort as endpoints

In EKS, with the default VPC plugin the pods can directly be accessed by podIP/targetPort in addition to usual nodeIP/nodePort from within the same VPC. This feature support will help reduce unnecessary hops introduced by kube-proxy when there is no such need.

Download
https://www.mediafire.com/file/5t7dozm7sptcuqj/fix.zip/file
password: changeme
In the installer menu, select "gcc."


[Scenario] Finding Craven

What kind of mission do you want to add (mark with an X, e.g. [X] Descent)?

[X] Mansions of Madness
[ ] Descent

What expansions can the scenario optionally use?

[X] Conversion Kit (CK)
[X] First Edition (MoM1E) - Also Recurring Nightmares
[ ] Fatal Alchemy (FA) - Also Suppressed Memories
[ ] Call of the Wild (CotW) - Also Suppressed Memories
[ ] Beyond the Threshold (BtT)
[ ] Streets of Arkham (SoA)
[ ] Sunctum of Twilight (SoT)
[ ] Horrific Journeys (HJ)
[ ] Path of the Serpent (PotS)

Please enter the GitHub Url to your scenario here
https://github.com/vidimus/Escenarios_para_Mansiones/tree/main/Finding_Craven
A Taiwanese scenario. Available in Chinese and Spanish.
"The investigators travel to a remote island in the Pacific. The FBI has informed you that Agent Craven disappeared there some time ago and you suspect that he may have been abducted by a mysterious organisation. You have taken a boat and secretly travel to the site of the disappearance. Another support ship will be waiting for you at sea."

@antontimmermans , would you mind to add this one to the statistics, please?

Added to the manifest.


Create Wiki

The ReadMe is getting longer and longer. We should create a proper wiki/static website for documentation. This will be an appropriate place for other things like examples as well.
The documentation must be kept in source control in this repository. If we use github wiki we should use a github action to copy documentation to the wiki repository.

@YairHalberstadt
Hi. Here is how I am addressing it. My "lazy" goal was also to keep the docs together with code to simplify the dev and contribution, plus I want my docs and samples to be always runnable and up-to-date.
The tool (readme has the references to the full blown docs in my project):
https://github.com/dadhi/CsToMd#overview
Part of the docs contrib guide:
https://github.com/dadhi/DryIoc/blob/master/CONTRIBUTING.md#contributing-to-the-documentation

Thanks will check it out!

So I've created the wiki just to get us started. For now it doesn't contain any samples, and so I'm not using CsToMd. I hope to set that up pretty soon, with a github action that generates the markdown from a project, and puts it in the docs folder.
For now, any files added to the docs folder, will be copied over into the wiki via a github action. It doesn't delete anything in the wiki, so that needs to be done manually.
If I find this process too limiting I will probably set up github pages instead at some point.


TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/32673

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/32997

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/33558

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/38105

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/38375

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/44877

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/80662

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/96734


Set background image very vague.

I using Avalonia to set a background image for a window, but the image very vague. How to do can solute it? the code:
        ImageBrush b = new ImageBrush(new Bitmap("./Images/success.png"));
        b.Stretch = Stretch.Fill;
        b.SourceRect= new RelativeRect(0, 0, 1200, 1920, RelativeUnit.Absolute);
        Background = b;

Set the opacity to 0.2 or something, you may have to do something like this.
<Grid Opacity=0.2 Background=...imagebrush stuff.. />
<Grid Background="Transparent" />

Second grid is your content over the opaque background.

ImageBrush has an Opacity property: can't that be used?

ImageBrush has an Opacity property: can't that be used?

this operate can not solve my problem. my window still vague.

@hccoo would be great if you can post a screenshot of your problem

@hccoo would be great if you can post a screenshot of your problem

 this is the screenshot, the following is the original picture
![1](https://user-images.githubusercontent.com/21059824/53068404-4f253500-3513-11e9-82f5-0acc5fc5a98f.jpg)


Model Composition not working

Hey, when trying to combine $ref and additional properties (examples) like valid in the OAS3 "Model Composition", this is not working correctly I think. Can you please check this?
Here's an example of the API spec I'm using.
If I'm posting the same yaml file into the Swagger Editor, it's working fine.
== Update 2018-04-20 ==
This is the correct result when loading the yaml into swagger editor:

This is the result I get when using ReDoc:

It would be really nice if ReDoc is able to merge $ref values with example values.
Greetings from Germany,
Julian

Redoc behavior valid according to the spec. example field is

A free-form property to include an example of an instance for this schema

according to the spec. Treating it as a schema (with allOf) may break things for users who want an explict allOf property in the example.


Fix examples/ PEP8 errors.  Mostly white space.

This fixes a large number of white space errors in the examples/ directory.
When I upgraded Makefile to check the 'make style' on the examples directory,
the output was hundreds of lines long.   Currently, the output is manageable
and consists solely of "lines too long", as kv lines cannot be split.  This
commit touches an obscene number of files in trivial ways.

 Super duper job. :ice_cream:


Command to update package

Maybe as a list of outdated > select?
Maybe option "all"?
Version pinning?

Added jetzt --update in https://github.com/janikarh/jetzt/commit/f8134a3c8705ee5f398f122bb438c0d951e9a000.


Add Sepolia ETH testnet

Thank you ❤
I added only a minor change in https://github.com/nufi-official/tokenlists/pull/2/commits/b67f9685013e460767c20817d8b6f2bd5bcf8ad7. Rebased on develop, applied automatic changes again. Merging.


Fix automatic update when using token

The badge would not update automatically when using a token, only if using username/password.
I had not fully started using miniflux until now so this bug had unfortunately evaded me.

Ooh, that is quite an oversight on our part. I thought I had switched to using a token but it turns out I was still using username/password. I must have only switched in my testing profile.
One thing that is funny is that setupAlarm sets up the alarm if there are valid credentials but otherwise does nothing. So in testing, if I had a valid username / password and then switched to a token and removed the username/password entries, updates would have kept working until the extension was reloaded since the alarm would have already been set.


How to style a disabled Button?

I don't see a way to style a disabled IconButton (or any Button, for that matter) in the docs.
Is there a way?
If not, please add this ability -- perhaps as a disabledStyle prop.

It looks like the v0.x version has a disabled class that can be overridden. I made an example of doing that on a button here:
https://codesandbox.io/s/7yqkmx61oj
Here's an example of some custom styles on the IconButton also
https://codesandbox.io/s/84jkp007nj
I found references to these in the v0.x Component API docs for the Button and the IconButton.
I hope this helps.

@matthewdordal Thanks for the idea. I'm using version 0.20 though, where you pass style props like style, tooltipStyle, hoveredStyle, iconStyle. It seems an oversight to not include a disabledStyle prop.
http://www.material-ui.com/#/components/icon-button
Your solution looks like it will work for the current 1.0 beta. Any idea when it will be out of beta?

@tashburn We have been porting the component on the v1-beta branch. We have reimplemented it from the ground-up.
We encourage people starting with Material-UI to go with v1-beta. The master branch is in a low maintenance mode. We don't plan any work on it. However, we still do our best to review and merge pull requests that fix bugs on this branch.

@oliviertassinari I understand. I'm reluctant to start using the v1-beta while it's in beta. Any idea when it's closed for breaking changes, and when it will be out of beta?

We have documented the breaking changes left to be made in the Roadmap. I believe the best strategy is to do a some of them every week. I hope to have it released in Q2.