adding transaction isolation level to relief transaction locking.

This request provide fix for DSCS-2283 to change the transaction level for multi value json search in AmberVertexQuery to prevent read locks.

@jrixon @snezanam @ninhnguyen @MingtaoSun @yetti?

👍 👍 👍

Are you sure this is going to make any difference or are we just making random changes in the hope that it will?
If you look at the actual deadlocks, the insert locked the whole properties table. So having a read isolation is not going to cause those to be unlocked..

LATEST DETECTED DEADLOCK
2016-06-17 08:59:46 7f66552cd700
*** (1) TRANSACTION:
TRANSACTION<PHONE_NUMBER>9, ACTIVE 94 sec inserting
mysql tables in use 3, locked 3
LOCK WAIT 31 lock struct(s), heap size 6544, 254 row lock(s), undo log entries 57
MySQL thread id 53608175, OS thread handle 0x7f6656d35700, query id 574276592 winch.nla.gov.au <IP_ADDRESS> amberdb Sending data
INSERT INTO property (id, txn_start, txn_end, name, type, value) SELECT p.id, p.s_id, 0, p.name, p.type, p.value FROM sess_property p, sess_vertex v WHERE p.s_id = @txn AND v.s_id = @txn AND v.id = p.id AND v.state = 'NEW'
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 109 page no 3372840 n bits 328 index property_name of table amberdb.property trx id<PHONE_NUMBER>9 lock_mode X locks gap before rec insert intention waiting
Record lock, heap no 2 PHYSICAL RECORD: n_fields 2; compact format; info bits 0
0: len 5; hex 616c696173; asc alias;;
1: len 6; hex 0000184e656c; asc    Nel;;
*** (2) TRANSACTION:
TRANSACTION<PHONE_NUMBER>1, ACTIVE 232 sec fetching rows
mysql tables in use 3, locked 3
287445 lock struct(s), heap size 45430312, 118489639 row lock(s)
MySQL thread id 52558308, OS thread handle 0x7f66552cd700, query id 574203294 winch.nla.gov.au <IP_ADDRESS> amberdb Sending data
INSERT INTO vp (id)
SELECT p.id
FROM property p
WHERE p.txn_end = 0
AND p.name = 'alias' AND p.value REGEXP x'226E6C612E6F682D323437352D303134362D303030302D733122'
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 109 page no 3372840 n bits 328 index property_name of table amberdb.property trx id<PHONE_NUMBER>1 lock mode S
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
0: len 8; hex<PHONE_NUMBER>6d756d; asc supremum;;
Record lock, heap no 2 PHYSICAL RECORD: n_fields 2; compact format; info bits 0
0: len 5; hex 616c696173; asc alias;;
1: len 6; hex 0000184e656c; asc    Nel;;
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 109 page no 3589378 n bits 864 index property_txn_end_idx of table amberdb.property trx id<PHONE_NUMBER>1 lock mode S waiting
Record lock, heap no 291 PHYSICAL RECORD: n_fields 2; compact format; info bits 32
0: len 8; hex<PHONE_NUMBER>000000; asc         ;;
1: len 6; hex 00009897f3fa; asc       ;;
*** WE ROLL BACK TRANSACTION (1)

@brycollins from reading above, it looks like the 2nd transaction<PHONE_NUMBER>1 is holding the lock, as the 1st Transaction<PHONE_NUMBER>9 looks to be created later than the 2nd transaction, and is holding less records than the 2nd transaction.  The second transaction has the query matches the multi-value json search, so I added the transaction level read-committed for consistent (non-locking) reads.

Yeah, Transaction 2 had been active 232 sec while transaction 1 had been waiting on acquiring the lock for 94 sec.
Stopping the query in transaction 2 from locking the table would have prevented this deadlock.

I still don't see how thats going to help.
While both queries are in separate transactions, they are pretty much single statement transactions.
READ_COMMITED will return the locks for any items NOT matching the scan, AFTER the statement finishes.
Given that it takes a long time to do this query, the statement did not finish before the deadlock hit.
This has something to do with the indexes not fulling covering all entries, so a full table scan is involved.
This is whats creating the row level locking. Again, another side effect of the blobs in properties, they can't really be indexed.
Unless I'm reading the doco wrong...


ci: use GOFLAGS, enforce gofmt -s.

This PR sets up a global GOFLAGS to avoid needing to repeat it for each command.
It also changes the script to exit on the first error, and to enforce that all non-vendored .go files satisfy gofmt -s without diffs. This will make sure all code is consistently formatted and help contributors
do the right thing by default.
(Matches the zcrypto PR: https://github.com/zmap/zcrypto/pull/195 - I'll leave it up to you folks if you'd like to add similar checks to the other zmap repositories, zcertificate, etc)

This is ready for :mag: as of f968b61


Fix validation-mixin causing "Uncaught TypeError: base is not a constructor" error

This PR fixes the following console error preventing the checkout from functioning properly:
TypeError: base is not a constructor
    at $.widget (widget.js:104:25)
    at validation-ext.js:36:11
    at mixins.js:105:22
    at Array.forEach (<anonymous>)
    at applyMixins (mixins.js:104:16)
    at mixins.js:129:36
    at Object.execCb (require.js:1696:33)
    at context.execCb (resolver.js:156:31)
    at Module.check (require.js:883:51)
    at Module.<anonymous> (require.js:1139:34)

Looking at other validator mixins, it looks like the target widget was missing as argument and return value. This is also shown in the examples at https://developer.adobe.com/commerce/frontend-core/guide/validations/custom-rules/
After applying these changes the error is gone and checkout works again.

@marcoveeneman Thank You for this PR, its merged.


Fix for cannot install npm version newer than 6.1.0

Also replaced the depending deleted library with a forked one to be able to build.

@nullivex   - will you produce just one more version?

@osher Absolutely, my apologies for the lag here. Let me check through this.

@petercsirmaz @osher Thank you for this and your patience!

Thanks for taking the time @nullivex if we may ask when might you find time for a new release on choco?

@nullivex @petercsirmaz @osher bump?

I should be able to complete the release today. Thanks for asking!

Release completed!

@nullivex  Cheers for sorting that out, looks like it's under moderation pending additional steps from maintainers before it will be available on the nodist feed.

@Dashue for Chocolately this is true, I am waiting on them to approve the package. The release is available from the releases page on this repo in the meantime.

They have provided feedback, there is a new requirement for Chocolately packages. I will work on getting this addition done today.

Cheers. I ended up following the instructions on how to install packages that are in moderation so worked around it. It seems to be working fine, was just able to install npm 6.9. Thanks again for this!

@Dashue you are very welcome and glad to hear its working. I just submitted the package that should conform to their newer standards. I am hopeful they will approve that tomorrow and the community can roll on!


Use package mgr to query Windows services

Go has a service manager package. Initial research shows it might be a viable alternative to using the current wmi package and might help with memory usage problems we see on one of our test machines.
https://godoc.org/golang.org/x/sys/windows/svc/mgr
Develop some test code to determine if using this package works and if so, implement it in the check_service command. Use a command line switch for the user to select the old wmi method or this service manager package method at run time. This will allow for testing the service manager package when deployed and fall back to wmi if needed.

This sounds great. Let's make sure we test on 32 bit and 64 bit architectures as well. I've got a few test machines spun up when you're ready


raftstore: use new approximate keys function when result is 0 (#5403)

cherry-pick #5403 to release-2.1

Signed-off-by: root<EMAIL_ADDRESS>What have you changed?
Please explain in detail what the changes are in this PR and why they are needed:

Summarize your change
When the new get_region_approximate_keys_cf function return 0, the get_region_approximate_keys function will downgrade to use the old get_range_entries_and_versions function. For this situation, 0 should be returned.

Describe any limitations of the current code (optional)
Add a test but the the test also passed using old get_range_entries_and_versions function.

What is the type of the changes?
Pick one of the following and delete the others:

Bugfix (a change which fixes an issue)

How is the PR tested?
Add a test function raftstore::coprocessor::split_check::keys::tests::test_region_approximate_keys_sub_region.
Does this PR affect documentation (docs) or should it be mentioned in the release notes?
No
Does this PR affect tidb-ansible?
No
Refer to a related PR or issue link (optional)
#5306

/run-all-tests


Change PostgreSQLContainer wait behaviour (fix #317)

I've changed the PostgreSQLContainer to use the WaitingConsumer for waiting for Log output, instead of performing SQL queries. It's basically the same code as in LogWaitStrategy, but I've figured we'll refactor the JdbcDatabaseContainers at a later stage, to make use of a WaitingStrategy?
This fixes #317

@rnorth Since you're already performing a refactoring of the Browser containers to use WaitStrategy in this PR, should I refactor this implementation to also rely on a WaitStrategy?

Using LogMessageWaitStratgy works great, I wonder if PostgreSQLContainer still needs to extend JdbcDatabaseContainer now?

@kiview that's a good point - would be interesting if we can break the inheritance hierarchy. @asafm  has kinda persuaded me to give that approach a chance :)
That said, it's not that easy to put the genie back in the bottle - we'd just need to consider whether this would cause any breaking changes (e.g. people using PostgresContainer and using methods that might cease to exist on its public API, like mounting volumes, etc).
I think it would be worth considering that as a separate change from this PR. I am, in general, meaning to kick off conversations about what the next iteration of the API will look like soon, so it's something we could consider at that stage.

@rnorth I'd also like to perform this refactoring in a seperate PR.
Regarding the failing codacy check:
I'm personally fine with using package scoping, if package scoping is all that's needed. I can however of course change the constants back to public scoping if you prefer it that way.

This looks good to me - no worries about the codacy warning; that's for guidance only and I agree with you on it.
Will merge!


How to change topbar height?

Two part question:
Question #1:
In F5, this was super easy! There was a specific SASS variable to change the topbar height.
Now in F6, it looks like there's not even half of the variables present anymore. I'm not exactly sure how to change the topbar height with SASS now!
I checked the _topbar.scss file, and there's no SASS variables in there either.
Is there an option in FoundationPress?
Also, are there any other topbar options anywhere else? F6 seems so sparse and it's making things more difficult than F5.
Question #2:
How exactly do you contain the navigation to grid? I've tried "contain-to-grid" in a couple places and it doesn't work!
Any help would be appreciated!

Check out this: http://zurb.com/university/lessons/how-to-re-create-top-bar-with-foundation-6
To mess with the height you only need to change the top-bar class's padding. To "contain-to-gird" just wrap everything inside of the top-bar div with a row div.

Thank you!

Although this issue is closed I'm going add how I achieve this through scss.
$topbar-height: 35px;
.top-bar .menu a {
	height: $topbar-height;
	line-height: $topbar-height;
}
.top-bar .menu > li:not(.menu-text) > a {
    padding: 0 1rem;
}

Hopefully this can help someone who needs a quick fix. Just increase (or decrease) the $topbar-height variable to whatever height you want.


CI: Add tests in msys2/mingw builds

The statically linked test binaries runs successfully but the dynamically linked ones do not. The 0xc0000135 error code in test means the program can not find a DLL. I have to copy libstdc++-6.dll, libgcc_s_seh-1.dll and libwinpthread-1.dll files from $MSYSTEM/bin to build/upnp/test folder for those tests.

gtest shouldn't be needed, I guess I have to get mingw to work here to check what's going on.

According to Process monitor, the dynamically linked tests can find libixml.dll and libupnp.dll from build directory. But it can not find those previously mentioned dlls. Here is the a portion of the output from Process Monitor:
test-upnp-init.exe	5852	QueryOpen	F:\pupnp\bin\upnp\test\libgcc_s_seh-1.dll	NAME NOT FOUND
test-upnp-init.exe	5852	CreateFile	C:\Windows\System32\libgcc_s_seh-1.dll	NAME NOT FOUND	Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a
test-upnp-init.exe	5852	CreateFile	C:\Windows\System\libgcc_s_seh-1.dll	NAME NOT FOUND	Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a
test-upnp-init.exe	5852	CreateFile	C:\Windows\libgcc_s_seh-1.dll	NAME NOT FOUND	Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a
test-upnp-init.exe	5852	QueryOpen	F:\pupnp\bin\upnp\test\libgcc_s_seh-1.dll	NAME NOT FOUND
test-upnp-init.exe	5852	QueryOpen	F:\pupnp\bin\ixml\libgcc_s_seh-1.dll	NAME NOT FOUND
test-upnp-init.exe	5852	QueryOpen	F:\pupnp\bin\upnp\libgcc_s_seh-1.dll	NAME NOT FOUND
test-upnp-init.exe	5852	QueryOpen	F:\pupnp\bin\upnp\test\%PATH%\libgcc_s_seh-1.dll	PATH NOT FOUND

This looks like the exakt opposite of what I expected. Could you post the upnp/test/CTestTestfile.cmake, maybe this gives some insights.

This one? CTestTestfile.cmake.txt

Exactly, you see the PATH=.... in 8? I guess %PATH% is not what mingw expects, you can edit this file directly, if this works, you could mask this line with if (MSVC) and add an else with the value that works for you.

Maybe mingw is smart enough to find it's stuff if you remove that whole path setting thing, would make the fix easier, too.

Thank you for the hint. I have limited that environment searching function for MSVC only and added the recently built DLLs path with ctest command. Now tests passed in mingw.

Does
D:/a/pupnp/pupnp/upnp/src/threadutil/ThreadPool.c:172:9: warning: implicit declaration of function 'gettimeofday'; did you mean 'mingw_gettimeofday'? [-Wimplicit-function-declaration]
172 |         gettimeofday(&now, NULL);
|         ^~~~~~~~~~~~
|         mingw_gettimeofday`
Need to be addressed?

That looks like a different topic to me. I am thinking to address that in different pull request with those string and I/O redefinition warnings. Just some ifdefs.


Add PhotoInfo.owner for shared photos

ZGENERICASSET.ZCLOUDOWNERHASHEDPERSONID points to ZCLOUDSHAREDALBUMINVITATIONRECORD.ZINVITEEHASHEDPERSONID

ZGENERICALBUM.ZCLOUDOWNERFULLNAME, ZCLOUDOWNERHASHEDPERSONID

ZGENERICASSET.ZSHAREORIGINATOR might be helpful as well

This will get the person's full name for shared photo with a given UUID
SELECT ZGENERICASSET.ZUUID, ZCLOUDSHAREDALBUMINVITATIONRECORD.ZINVITEEFULLNAME, ZCLOUDSHAREDALBUMINVITATIONRECORD.ZINVITEEHASHEDPERSONID
FROM ZCLOUDSHAREDALBUMINVITATIONRECORD
JOIN ZGENERICASSET ON ZGENERICASSET.ZCLOUDOWNERHASHEDPERSONID = ZCLOUDSHAREDALBUMINVITATIONRECORD.ZINVITEEHASHEDPERSONID
WHERE ZGENERICASSET.ZUUID = "83EE2759-6875-43A6-99C8-184FBE51AC9C"
ORDER BY ZCLOUDSHAREDALBUMINVITATIONRECORD.Z_PK
LIMIT 1

I have a beta implementation of this working.  Just need to add tests.  I've added a new way to easily add new properties to PhotoInfo without having to modify the code in photosdb which is very ugly and complex.  Basically, added a way to execute a sql query against the one property you're interested in which makes it much easier to add new properties like this in the future.  For example, the ZCLOUDSHAREDALBUMINVITATIONRECORD has a "ZISMINE" column which is true if the photo is yours versus someone else's that was shared with you.  This would be useful in addition to the owner name.

So this works for photos that people have shared to albums you invited them to but no for photos where someone else invited you to the shared album.
In the latter case, it looks like I'll need to get name from ZGENERICALBUM where ZGENERICALBUM is JOINed on ZCLOUDOWNERHASHEDPERSONID

For the latter case, this looks like it'll work:
SELECT ZGENERICALBUM.ZCLOUDOWNERFULLNAME
FROM ZGENERICALBUM
JOIN ZGENERICASSET ON ZGENERICASSET.ZCLOUDOWNERHASHEDPERSONID = ZGENERICALBUM.ZCLOUDOWNERHASHEDPERSONID
WHERE ZGENERICASSET.ZUUID = "9AAA73E1-1BCC-431D-BFD2-1D2C4A037A81"

It would be good to have a way to do this in a single SQL query but that may be beyond my SQL abilities

I think this will work:
WITH case1 AS
(
SELECT ZGENERICALBUM.ZCLOUDOWNERFULLNAME as OWNER_FULLNAME
FROM ZGENERICALBUM
JOIN ZGENERICASSET ON ZGENERICASSET.ZCLOUDOWNERHASHEDPERSONID = ZGENERICALBUM.ZCLOUDOWNERHASHEDPERSONID
WHERE ZGENERICASSET.ZUUID = "983B75C9-6991-41A7-9DCE-43E9DAA15C28"
),
case2 AS
(
SELECT
ZCLOUDSHAREDALBUMINVITATIONRECORD.ZINVITEEFULLNAME AS OWNER_FULLNAME
FROM ZCLOUDSHAREDALBUMINVITATIONRECORD
JOIN ZGENERICASSET ON ZGENERICASSET.ZCLOUDOWNERHASHEDPERSONID = ZCLOUDSHAREDALBUMINVITATIONRECORD.ZINVITEEHASHEDPERSONID
WHERE ZGENERICASSET.ZUUID = "983B75C9-6991-41A7-9DCE-43E9DAA15C28"
ORDER BY ZCLOUDSHAREDALBUMINVITATIONRECORD.Z_PK
LIMIT 1
)
SELECT * FROM case1
UNION
SELECT * FROM case2 WHERE NOT EXISTS (SELECT * FROM case1)

PhotoInfo.owner (and AlbumInfo.owner) were implemented in v0.42.85.  I tried a new way of implementing photo properties to make the code more maintainable but after doing some benchmarking, not sure about this as a long term solution.
osxphotos works but querying the sqlite database that Photos uses to store data about the library.  Normally, all the data needed by osxphotos is read at startup (when the PhotosDB() class is instantiated) and stored in in-memory data structures (python dict) which are then accessed by PhotoInfo and other classes as needed to retrieve the information.  This means the PhotosDB class is quite complex and I've got to keep track of all the various data structures.
For implementing the owner property,  I added the ability to execute a query  against the database and return the results as well as the ability to load a sql query from disk as a template.  This makes the implementation of owner and future properties very easy as one doesn't need to understand the complex and ugly code in PhotosDB to add new features--you just need to craft a SQL query against the Photos database, the structure of which is documented in the osxphotos wiki.
Unfortunately, I didn't realize how slow this would be.  For example, against my library of 30536 photos, this means looking up owner makes 30536 sqlite queries and this is slow.   I've taken steps to speed it up -- e.g. the SQL query that's loaded of disk is memoized so it only gets loaded once, the result is memoized so repeated lookups of owner don't have to hit the database, etc but it's still really slow.  Benchmarking results follow. Not sure if I should keep this implementation as this may not be a frequently used property or refactor this to load all the owner info at startup along with the other pre-loaded data.  The advantage here is it can be done in a single query instead of 30k+ separate queries.
python bench.py
Loading database
Getting photos
Got 30536 photos
Getting photos with title
Got 62 in 0.025536060333251953 seconds
Getting photos with owner
Got 6350 in<PHONE_NUMBER>5881042 seconds
Getting photos with owner again
Got 6350 in 0.06004810333251953 seconds

bench.py:
import osxphotos
import time

print(f"Loading database")
db = osxphotos.PhotosDB()

print(f"Getting photos")
photos = db.photos(images=True, movies=True)
print(f"Got {len(photos)} photos")

print(f"Getting photos with title")
start_t = time.time()
new_photos = [p for p in photos if p.title]
stop_t = time.time()
print(f"Got {len(new_photos)} in {stop_t-start_t} seconds")

print(f"Getting photos with owner")
start_t = time.time()
new_photos = [p for p in photos if p.owner]
stop_t = time.time()
print(f"Got {len(new_photos)} in {stop_t-start_t} seconds")

print(f"Getting photos with owner again")
start_t = time.time()
new_photos = [p for p in photos if p.owner]
stop_t = time.time()
print(f"Got {len(new_photos)} in {stop_t-start_t} seconds")

Also, the current implementation is not 100% correct. It returns "(null) (null)" for some photos.

Looks like pre-loading the owner info is definitely the way to go.  Simple test below shows that all owner info can be pre-loaded in 0.25 seconds (on my 2013 macbook pro, it already takes ~15 seconds for osxphotos to load the entire database so this is negligible):
Loading database
Getting photos
Got 30536 photos
Loading owner info
owners = 2365
owners = 6350
Got 6350 owner records in 0.2652130126953125 seconds

import osxphotos
import time

print(f"Loading database")
db = osxphotos.PhotosDB()

print(f"Getting photos")
photos = db.photos(images=True, movies=True)
print(f"Got {len(photos)} photos")

owners = {}
print(f"Loading owner info")
sql_owner_case_1 = """
    SELECT DISTINCT ZGENERICASSET.ZUUID as UUID, ZGENERICALBUM.ZCLOUDOWNERFULLNAME as OWNER_FULLNAME
    FROM ZGENERICALBUM
    JOIN ZGENERICASSET ON ZGENERICASSET.ZCLOUDOWNERHASHEDPERSONID = ZGENERICALBUM.ZCLOUDOWNERHASHEDPERSONID
    WHERE OWNER_FULLNAME != '(null) (null)'
"""
sql_owner_case_2 = """
    SELECT DISTINCT
    ZGENERICASSET.ZUUID AS UUID, ZCLOUDSHAREDALBUMINVITATIONRECORD.ZINVITEEFULLNAME AS OWNER_FULLNAME
    FROM ZCLOUDSHAREDALBUMINVITATIONRECORD
    JOIN ZGENERICASSET ON ZGENERICASSET.ZCLOUDOWNERHASHEDPERSONID = ZCLOUDSHAREDALBUMINVITATIONRECORD.ZINVITEEHASHEDPERSONID
"""
start_t = time.time()

results = db.execute(sql_owner_case_1)
for row in results:
    owners[row[0]] = row[1]

print(f"owners = {len(owners)}")

results = db.execute(sql_owner_case_2)
for row in results:
    owners[row[0]] = row[1]
print(f"owners = {len(owners)}")
stop_t = time.time()
print(f"Got {len(owners)} owner records in {stop_t-start_t} seconds")

This is more like it...new implementation that pre-loads the necessary data
>>> import time
>>> start_t = time.time(); owners = [(p.uuid, p.owner) for p in photos if p.owner]; stop_t = time.time()
>>> stop_t - start_t
0.07246017456054688
>>> len(owners)
6350
>>>

Fixed in v0.42.88.  Still need to apply the same fix to album.owner

Fixed (again) in v0.42.89


Can we add speaker separation function

For example, in a multiplayer meeting, distinguish the content of each speaker. Then output speaker1,2,3,4,5,6
@sandrohanea

This is a diarization based on multiple wave streams (assuming each speaker will be on a different stream): https://github.com/sandrohanea/whisper.net/tree/main/examples/Diarization
However, if the speakers are not separated to different streams, you'll need to rely only on: https://github.com/sandrohanea/whisper.net/issues/161
Either use that experimental tiny diarize (which tells you when speaker is changing as a different token) or pre-separate the audio using another tool and use whisper.net only for speech to text => and not diarization.


WIP deploy schematic

Checklist

Issue number for this PR: #nnn (required)
Docs included?: (yes/no; required for all API/functional changes)
Test units included?: (yes/no; required)
In a clean directory, yarn install, yarn test run successfully? (yes/no; required)

Description

Code sample

FYI @kirjs I rearranged things a bit and got it compiling. Haven't tested yet. Have a couple more cleanup tasks.

Cool, lmk if there's any way I can help

@kirjs My main concern is depending on firebase-tools it will light up npm audit, as it has vulnerabilities (mostly due to needing to support Node 6 for at least the next couple weeks). Fine with me if it's an @next cut. If we do pull into prod release, we'll want standard coms for it being a development dep or to wait until firebase-tools drops Node 6... though perhaps the management APIs can give us what we need. Thoughts @davideast?

@kirjs in the meantime, feel free to check out getting your tests ported. I just killed them for the moment but I put placeholders in... place.

@jhuleatt if you could give this a quick look over so we can merge it would be appreciated. It's had a bunch of eyeballs on it between myself, @kirjs, and @mgechev so the risk is low (beyond my prior expressed general concerns) so feel free to rubber stamp if nothing smells funny to you.

Just opened a PR (https://github.com/angular/angularfire2/pull/2081) to address @jhuleatt's comments.


Use default identifier in queued_search as can't handle others anyway

queued_search needs ids to be in form site.model.pk, which is the default haystack ID. However, the code is making use of get_identifier method which actually checks if a custom identifier method has been specified. In my case, I had one, and it returned ID's in the form of model:pk which is fine with Haystack but breaks everything in queued_search. This fix makes sure queued_search works while still using the id's that user wants in the index itself (as that is picked by Haystack in the end anyway, as long as you pass an instance of an object and not a string).

:(


[Epic] Detox 20 Breaking Changes

Description
This is a meta-PR to put miles with all the planned major and breaking changes and roll them out when ready.
Implemented:

[x] https://github.com/wix/Detox/issues/2469
[x] https://github.com/wix/Detox/pull/3289

[x] check specs docs, because the feature looks historically underdocumented... 🤔

[x] https://github.com/wix/Detox/pull/3286
[x] JS: always boot emulators in read-only mode (even for a single instance): d77450f366f35ee2a18a302f480ea66096eec0e5
[x] JS: set required Node.js version to be >=12.10.0 82c0e7a45ba829f81d3c4379e7c174cc64428be5
[x] https://github.com/wix/Detox/pull/3304
[x] https://github.com/wix/Detox/pull/3310
[x] https://github.com/wix/Detox/pull/3326
[x] https://github.com/wix/Detox/pull/3332
[x] #3352
[x] #3349
[x] #3350
[x] #3355
[x] #3351
[x] #3394

In progress:

[x] #3397
[ ] #3354

Planned:

[ ] #3353

Postponed

Android: Usage of ActivityTestRule instead of ActivityScenarioRule – @d4vidi says that due to the planned multiple apps breaking change, we need to postpone it – otherwise, users will need to fiddle with Android files twice every migration.

Will Detox v20 will support react-native 0.68.2+?

@RohovDmytro https://github.com/wix/Detox/issues/3344


[TEST] Implement test case test_migration_with_restore_volume

What's the test to develop? Please describe
Currently test case test_migration_with_restore_volume have skeleton but script not implement yet.
Describe the items of the test development (DoD, definition of done) you'd like

[x] Implement script
[x] Verify on pipeline

Additional context
N/A

Test case now working properly on pipeline, close this ticket and below were test result for reference

master-head
v1.3.x
v1.2.x


Add support for Viacoin [chain-integration]

Description
Add support for Viacoin. Chain intergration.
old pull request: https://github.com/TrustWallet/wallet-core/pull/274
Testing instructions

Types of changes

Checklist

[ ] Prefix PR title with [WIP] if necessary.
[x] Add tests to cover changes as needed.
[ ] Update documentation as needed.

I just force pushed a commit.
used
git -A
git commit -S --amend
git push -f
Unsure how I can these Azure tests running

CI fixed

We have added codecov recently, could you please fix test coverage? Thanks

Please add tests to CoinTests.cpp, that should fix the Codecov failure.

Will work on it.

This should fix it: https://github.com/TrustWallet/wallet-core/commit/fc98868134ccda04a2ee57905e21b9285b9e307a

So it's all good now?
Or am I supposed to git cherry pick it and push an extra commit to this pull request?
Or just edit it in my editor and do git add - A
And git commit -S --amend?
Or can this pull request be merged now?

Please rebase on latest master.

Oke I just rabased.
CI tests are running (build #2401 if I'm right)

Codecov still fails

@romanornr can you DM me on telegram @vikmeup, I would need a blockbook url node from you.


MSFT_xServiceResource.psm1: Fix Custom DSC Resource Kit PSSA rule violations

Details of the scenario you tried and the problem that is occurring
There's a number of script analyzer issues getting reported in MSFT_xServiceResource.psm1. These should be fixed.
Verbose logs showing the problem
[00:04:45]     Context MSFT_xServiceResource.psm1
[00:04:45]       [+] Should pass all error-level PS Script Analyzer rules 100ms
[00:04:49]       [+] Should pass all required PS Script Analyzer rules 3.64s
[00:04:49]       [+] Should pass all flagged PS Script Analyzer rules 246ms
[00:04:49]       [+] Should pass any recently-added, error-level PS Script Analyzer rules 73ms
[00:04:51]       [+] Should not suppress any required PS Script Analyzer rules 2.04s
[00:04:52] Custom DSC Resource Kit PSSA rule(s) did not pass.
[00:04:52] The following PSScriptAnalyzer rule 'DscResource.AnalyzerRules\Measure-ParameterBlockParameterAttribute' errors need to be fixed:
[00:04:52] MSFT_xServiceResource.psm1 (Line 206): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 210): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 214): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 218): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 222): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 225): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 229): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 232): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 236): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 240): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 244): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 247): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 250): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 429): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 433): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 437): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 441): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 445): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 448): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 451): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 455): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 459): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 463): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 467): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 470): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] MSFT_xServiceResource.psm1 (Line 473): A [Parameter()] attribute must be the first attribute of each parameter and be on its own line. See https://github.com/PowerShell/DscResources/blob/master/StyleGuidelines.md#correct-format-for-parameter-block
[00:04:52] For instructions on how to run PSScriptAnalyzer on your own machine, please go to https://github.com/powershell/PSScriptAnalyzer
[00:04:52] Common Tests - Custom Script Analyzer Rules will not fail unless you opt-in.
[00:04:52] To opt-in, create a '.MetaTestOptIn.json' at the root
[00:04:52] of the repo in the following format:
[00:04:52] [
[00:04:52]      "Common Tests - Custom Script Analyzer Rules"
[00:04:52] ]
[00:04:52]       [+] Should pass all custom DSC Resource Kit PSSA rules 880ms

Version of the DSC module that was used ('dev' if using current dev branch)
dev

I got this one.


[Docs Site] Fix mixed-case + spaces in Tab Switcher labels

Fixes the tab switcher usage on https://developers.cloudflare.com/workers-ai/fine-tunes/loras/ taking you to the top of the page since it caused exceptions in the JS that handles click events.

we called lower on non-JS/TS labels but the JS didn't search with a lower-cased selector

we didn't wrap the label in quotes meaning labels with spaces (like "Workers AI SDK") wasn't a valid selector

Fixes can be tested here


Allow to run downstream layers of excluded layers when using Terragrunt *-all

When using exclude-dir, terragrunt will not run the downstream layers of the excluded layers.
This makes perfect sense for apply-all and destroy-all, however it should not really matter for plan-all since the state is not altered between the plans.
Further to #868, this is a proposal to allow to run the downstream layers of excluded layers anyways.
Our use case is for testing on a CI server:

because reasons, we cannot plan some layers in CI (insufficient permissions), however we can plan other layers which depend on them.

@brikis98 I would have some questions.

Do we want this behaviour to be triggered via the flag introduced in #891, terragrunt-ignore-dependency-order? Or another flag?
I can see two ways of implementing this:

in running_modules.go, just conditionally avoid the call to removeFlagExcluded in toRunningModules.
avoid flagging the dependant modules in the first place in module.go

I lean towards the first option, but checking before starting :)

Just to make sure I understand, you're running plan-all --exclude-dir foo... And within folder foo, there are several modules, some of which you want to exclude, and some which you don't?

Just to make sure I understand, you're running plan-all --exclude-dir foo... And within folder foo, there are several modules, some of which you want to exclude, and some which you don't?

Not exactly, our infra looks something like:
prod
├── app
│   ├── secrets.tfvars.enc
│   └── terraform.tfvars
├── data-store
│   ├── secrets.tfvars.enc
│   └── terraform.tfvars
├── networking
│   ├── secrets.tfvars.enc
│   └── terraform.tfvars
└── terraform.tfvars

app depends on data-store which depends on networking (via  dependencies {    paths = ["../networking"]  }).
Because of permission reasons (that we hope to solve at some point, but cannot yet), our CI system cannot plan, say, the data-store layer (it needs AWS permission that we don’t want to grant ; or it relies on a submodule that we don’t want the CI system to be able to clone, etc.).
I thus want to plan all layers except data-store.
Currently, plan-all --exclude-dir data-store will exclude data-store and app, since app depends on data-store.
The new behaviour would be to run the plan against app anyways.

Thx for the explanation!

Currently, plan-all --exclude-dir data-store will exclude data-store and app, since app depends on data-store.

So when you run terragrunt plan-all --exclude-dir data-store in the prod folder, it also excludes app? And I guess that's because app depends on data-store, so Terragrunt is trying to be conservative, and makes the assumption that if we're not processing data-store, we won't succeed in processing anything that depends on it?
Now I think I understand your question here:

Do we want this behaviour to be triggered via the flag introduced in #891, terragrunt-ignore-dependency-order? Or another flag?

--terragrunt-ignore-dependency-order should definitely NOT skip the app folder. I'm not convinced we need yet another flag that does that too... Probably best to wait and see if a use case for that comes up, but for now, changes just the --terragrunt-ignore-dependency-order behavior.

I can see two ways of implementing this:

I'm not sure on the best approach, as this code has been updated many times since I last worked on it! I'd recommend doing a TDD style approach here, making sure we have test cases that verify both options: i.e., that app is excluded and is not excluded, depending on the --terragrunt-ignore-dependency-order flag... And once those tests are passing/failing as expected, try out the two implementation options you considered to see how complicated they turn out to be, and if they pass tests.

So, I wanted to implement this, and started by adding some tests for removeFlagExcluded in its current behaviour. Filed this as #912
And doing, that I discovered that:

So when you run terragrunt plan-all --exclude-dir data-store in the prod folder, it also excludes app? And I guess that's because app depends on data-store, so Terragrunt is trying to be conservative, and makes the assumption that if we're not processing data-store, we won't succeed in processing anything that depends on it?

…actually no, app is not excluded − I got confused because the pattern I used for excluding the layer I did not want just happened to also match a layer that depended on it ^__^"
So, while an argument could be made that Terragrunt should be conservative and not try to process downstream dependencies of excluded modules, that is not the current behaviour, and that issue is thus invalid :-)

Hmmm, so that’s for removeFlagExcluded.
However, re-reading module.go it really seems to me that https://github.com/gruntwork-io/terragrunt/blob/master/configstack/module.go#L128-L133 should mark the dependencies as excluded too… :thinking:

However, re-reading module.go it really seems to me that https://github.com/gruntwork-io/terragrunt/blob/master/configstack/module.go#L128-L133 should mark the dependencies as excluded too… thinking

But that’s covered by TestResolveTerraformModulesTwoModulesWithDependenciesExcludedDirsWithDependency so I guess I’m just missing something ^_^ (sorry for thinking aloud here!)

However, re-reading module.go it really seems to me that https://github.com/gruntwork-io/terragrunt/blob/master/configstack/module.go#L128-L133 should mark the dependencies as excluded too

It's using canonical paths—i.e., absolute paths with all relative paths resolved—so wouldn't app correspond to exactly one path / module on the whole file system?

However, re-reading module.go it really seems to me that https://github.com/gruntwork-io/terragrunt/blob/master/configstack/module.go#L128-L133 should mark the dependencies as excluded too

It's using canonical paths—i.e., absolute paths with all relative paths resolved—so wouldn't app correspond to exactly one path / module on the whole file system?

I have a module called app and another called application-roles − the latter is definitely excluded when using --terragrunt-exclude-dir app

Oh, that must be because it's doing a string.Contains check instead of an equality check: https://github.com/gruntwork-io/terragrunt/blob/master/configstack/module.go#L201. That's probably not a side effect we want. Since the --terragrunt-exclude-dir supports unix-style globs, it seems like --terragrunt-exclude-dir 'foo' should be an exact match on foo, whereas --terragrunt-exclude-dir 'foo*' should match foo, foobar, etc...


Fixing chef task to now call Get-ServiceEndpoint

Fixing the chef task to now call the Get-ServiceEndpoint cmdlet instead of the Get-ConnectedServiceDetails cmdlet.
Making the fix in the chef generic task as well.

Looks fine.


WIP: Continuous Fuzzing Integration with Fuzzit

Please do not merge yet. This is work in progress.
The contribution was discuss in the mailing-list with @htuch , @mattklein123 .
Description:
This will introduce another platform (apart from oss) fuzz that
will run the long-running fuzzers as well as will introduce
"sanity fuzzers" that will run the accumlated corpus and crashes
on every Pull-Request to detect bugs early-on in the development
cycle.
Risk Level:
Low - as this will introduce only another step in CircleCI where the fuzzers will be uploaded to Fuzzit and the heavy lifting will be there.
Testing:
No code is added just a CI code in Circle
Docs Changes:
Will be updated later.
Release Notes: None
[Optional Fixes #Issue] None
[Optional Deprecated:] None

@htuch looks like I get some no-credits error at CircleCI? any particular reason for this to happen? I can test my changes.

@lizan any idea what's up with this no credits error?

/retest

@htuch thx. just an update. look like it works now. I'll update when the PR will be ready for review. should be soon. cheers.

@htuch looks like I have a problem and maybe you can direct me to someone more familiar with the bazel build system that can give me a few hints.
I did manage to compile the targets inside a docker natively. Essentially what I did is a minor modification of the OSS-Fuzz script so it can work in a docker natively and not docker-in-docker.
The problem that I'm facing is that Circle doesn't support --cap_add SYS_PTRACE which for some reason that I don't totally understand is needed to compile the current libFuzzer targets (I Thought it is only needed when you run the target and not compile them).
I tried running it on Azure where it looks possible to add those capabilities but got some different errors.
Anyway looks like the OSS-Fuzz script is also not part of the build system and a lot of configuration are inside the bash script outside the bazel. I think it will be best to add those configuration to the bazel build.

@yevgenypats what does the compiler complain about when building the libfuzzer targets without SYS_PTRACE? This seems odd for sure.

@htuch if you run the following commands (taken from README)
git clone https://github.com/google/oss-fuzz.git
cd oss-fuzz
python infra/helper.py build_image envoy
python infra/helper.py build_fuzzers --sanitizer=address envoy <path to
   envoy source tree>

This will build the fuzzers and will place them in build/out/ which is fine and I took the same (just without creating a docker in a docker, so I did some minor modification).
Anyway you will see in the output and also in the infra/helper.py script that it is spawning the docker with SYS_PTRACE capabilities just to build the targets. I don't know if there is a way to get rid of this and I'm not sure why it is needed in the first place because from my understanding the capability is needed for LSAN only in runtime but if I run the docker without this capability the build fails.
Also, most of the logic of building the fuzzers is in this file and build.sh which is kind of outside the envoy repo and the bazel script which makes it a bit hard to maintain and understand IMHO. It will be the best if it will be possible to build the fuzzers via bazel build <command> which unfortunately I don't know how to do this without the right direction.

@yevgenypats why aren't we just building under Bazel without the infra/helper.py stuff? I thought the idea is we generate a driverless variant of the binary with bazel build directly, and then do some link to produce a libfuzzer binary?
Basically, we want to build this https://github.com/envoyproxy/envoy/blob/master/bazel/envoy_test.bzl#L109 and then link in libfuzzer in a target that depends on it.

@htuch I'm not pretty sure why we need to link it after bazel build? why not compile it directly with -fsanitize=fuzzer which links it directly with libFuzzer?
The infra is doing is exactly that it essentially calling bazel build and then linking but this needs for the unknown to me reason the SYS_PTRACE permissions.

@yevgenypats yes, you can use this flag as part of the build of the libfuzzer target. I'd drop the infra/helper.py scripts, this is a bit of a red herring. If you can just extend the Bazel I pointed at with an additional target that has -fsanitize=fuzzer and appropriate driver, we're set.

@htuch so looks like it doesn't work probably I'm missing something. here are the exact steps to reproduce:
apt update && apt install vim
docker run -it docker run -it envoyproxy/envoy-build:8246167b9d238797cbc6c03dccc9e3921c37617d /bin/bash
git clone envoy
vi ./bazel/envoy_test # here change -lFsanitizeEngine -fsanitize=fuzzer and also add ['-fsanitize=fuzzer'] in copts in the same target.
export CC=clang-8
export CXX=clang++-8
bazel build //test/server:server_fuzz_test_driverless

This fails with the following errors (snippet):
ERROR: /envoy/test/server/BUILD:223:1: Linking of rule '//test/server:server_fuzz_test_driverless' failed (Exit 1) clang-8 failed: error executing command /usr/bin/clang-8 @bazel-out/k8-fastbuild/bin/test/server/server_fuzz_test_driverless-2.params

Use --sandbox_debug to see verbose messages from the sandbox
/usr/bin/ld.gold: error: /usr/lib/llvm-8/lib/clang/8.0.1/lib/linux/libclang_rt.fuzzer-x86_64.a(FuzzerDataFlowTrace.cpp.o): invalid section group 9 refers to earlier section 3
/usr/bin/ld.gold: error: /usr/lib/llvm-8/lib/clang/8.0.1/lib/linux/libclang_rt.fuzzer-x86_64.a(FuzzerDataFlowTrace.cpp.o): invalid section group 10 refers to earlier section 4
/usr/bin/ld.gold: error: /usr/lib/llvm-8/lib/clang/8.0.1/lib/linux/libclang_rt.fuzzer-x86_64.a(FuzzerDataFlowTrace.cpp.o): invalid section group 11 refers to earlier section 5
/usr/bin/ld.gold: error: /usr/lib/llvm-8/lib/clang/8.0.1/lib/linux/libclang_rt.fuzzer-x86_64.a(FuzzerDriver.cpp.o): invalid section group 96 refers to earlier section 6
/usr/bin/ld.gold: error: /usr/lib/llvm-8/lib/clang/8.0.1/lib/linux/libclang_rt.fuzzer-x86_64.a(FuzzerDriver.cpp.o): invalid section group 97 refers to earlier section 7
/usr/bin/ld.gold: error: /usr/lib/llvm-8/lib/clang/8.0.1/lib/linux/libclang_rt.fuzzer-x86_64.a(FuzzerDriver.cpp.o): invalid section group 98 refers to earlier section 9
/usr/bin/ld.gold: error: /usr/lib/llvm-8/lib/clang/8.0.1/lib/linux/libclang_rt.fuzzer-x86_64.a(FuzzerDriver.cpp.o): invalid section group 99 refers to earlier section 10

Looks like all the additional flags defined here and here (It took me long to find out where it's exaclty located) are necessary but then again it only compiles with -lFsanitizeEngine and then again it need the SYS_PTRACE capability

@htuch Good News! Looks like the integration is ready for review, as the compilation now doesn't require SYS_PTRACE:)
You can go ahead and sign-up at fuzzit and I'll add you to the Envoy organisation. We currently contributed 20 CPUs.
The Logic is the following:

running "sanity" tests on every PR which will include running the fuzzers through all the accumulated (merged) corpus and crashes.
running nightly long-running fuzzing jobs with the latest binary from master. Each target will accumulate it's corpus and crashes (it will not reset every day it will just update the fuzzing binary).

Update: this is wip and hopefully will be ready by next week. cheers.

@yevgenypats @asraa has made some progress on Bazel linking with libfuzz, maybe you can take a look at her WiP branch.

Yep! I've linked Yevgeny to
https://github.com/asraa/envoy/commit/aae9db91a1e9da5e07d3a9ac821aac438f30925a
But I'm almost done linking in a reproducible way (with -fsanitize=fuzzer)

@asraa Can you contribute the last fix we discussed regarding the separating the ubsan asan config:) much appreciated!

@htuch @lizan looks like the fuzzit integration is finally ready!!

fuzzit regression now will run on every PR inline with CI.
long running fuzzing job will be run on a daily basis from master.

This is ready for final review.
One of you have to sign-up at https://app.fuzzit.dev so I can add you to envoyproxy account so you can take over. once you take over you need to go to https://app.fuzzit.dev/envoyproxy/settings and copy the api key to CircleCI environment variable FUZZIT_API_KEY.
@asraa Thanks you so much for the help and the native bazel build for the fuzzers!!

ping @htuch all checks passed and I think I fixed all the comments:)

/wait
On Thu, Sep 5, 2019, 8:16 PM stale[bot]<EMAIL_ADDRESS>wrote:

This pull request has been automatically marked as stale because it has
not had activity in the last 7 days. It will be closed in 7 days if no
further activity occurs. Please feel free to give a status update now, ping
for review, or re-open when it's ready. Thank you for your contributions!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/envoyproxy/envoy/pull/7509?email_source=notifications&email_token=AD52CDXYTF7UF4DGT2Y35PDQIE5IHA5CNFSM4H7ICZEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6AACAY#issuecomment-528482563,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AD52CDTWZCG7BOXG4M5V47TQIE5IHANCNFSM4H7ICZEA
.

@htuch @lizan sorry for the delay. I was a bit overloaded with our release for Java Support:)
As far as I can tell there are some problems integrating with azure pipelines. The current one that I'm strangling with  is that it looks like the output from the build inside the docker are not available on the host. i.e I can't access the fuzzing binaries...
+ ls -la /home/vsts/work/1/a/tmp/_bazel_bazel/b570b5ccd0454dc9af9f65ab1833764d/execroot/envoy/bazel-out/k8-fastbuild/bin/test/common/access_log/access_log_formatter_fuzz_test_with_libfuzzer.runfiles/envoy/test/common/access_log/access_log_formatter_fuzz_test_with_libfuzzer
lrwxrwxrwx 1 vsts docker 167 Sep 17 07:53 /home/vsts/work/1/a/tmp/_bazel_bazel/b570b5ccd0454dc9af9f65ab1833764d/execroot/envoy/bazel-out/k8-fastbuild/bin/test/common/access_log/access_log_formatter_fuzz_test_with_libfuzzer.runfiles/envoy/test/common/access_log/access_log_formatter_fuzz_test_with_libfuzzer -> /build/tmp/_bazel_bazel/b570b5ccd0454dc9af9f65ab1833764d/execroot/envoy/bazel-out/k8-fastbuild/bin/test/common/access_log/access_log_formatter_fuzz_test_with_libfuzzer
++ realpath /home/vsts/work/1/a/tmp/_bazel_bazel/b570b5ccd0454dc9af9f65ab1833764d/execroot/envoy/bazel-out/k8-fastbuild/bin/test/common/access_log/access_log_formatter_fuzz_test_with_libfuzzer.runfiles/envoy/test/common/access_log/access_log_formatter_fuzz_test_with_libfuzzer
realpath: /home/vsts/work/1/a/tmp/_bazel_bazel/b570b5ccd0454dc9af9f65ab1833764d/execroot/envoy/bazel-out/k8-fastbuild/bin/test/common/access_log/access_log_formatter_fuzz_test_with_libfuzzer.runfiles/envoy/test/common/access_log/access_log_formatter_fuzz_test_with_libfuzzer: No such file or directory

Is there any easy way to solve this or should we go back to Circle where it's already worked?

@yevgenypats just run everything in docker like it did in Circle? i.e. ci/run_envoy_docker.sh 'ci/do_ci.sh bazel.fuzz && ci/run_fuzzit.sh local-regression' as the step?

I can try that but regression also run the test in a docker so the tests
will be reproducible. Will I have access from inside the docker?
On Tue, Sep 17, 2019, 11:28 PM Lizan Zhou<EMAIL_ADDRESS>wrote:

@yevgenypats https://github.com/yevgenypats just run everything in
docker like it did in Circle? i.e. ci/run_envoy_docker.sh 'ci/do_ci.sh
bazel.fuzz && ci/run_fuzzit.sh local-regression' as the step?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/envoyproxy/envoy/pull/7509?email_source=notifications&email_token=AD52CDUSDXDEATHNQIPVJT3QKE4WHA5CNFSM4H7ICZEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD65ZSSI#issuecomment-532388169,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AD52CDWIRX6R7JIJMUWEBF3QKE4WHANCNFSM4H7ICZEA
.

it should have same access, lmk if you hit any issue.

@lizan I think something was changed. what would be the right (automated) way to find the path to the fuzzing binary. I get this https://dev.azure.com/cncf/envoy/_build/results?buildId=11190&view=logs&jobId=d205ed5c-f2f1-5f43-d907-fe4cb51a95a5 now.

@yevgenypats Oh yeah bazel test no longer generate binary with RBE, so you need to modify ci/do_ci.sh to do bazel build in addition to bazel test (in this line: https://github.com/envoyproxy/envoy/blob/master/ci/do_ci.sh#L261), maybe just add a fuzzit CI_TARGET in the do_ci.sh script?

so I've add a CI_TARGET and looks like it progressing but I don't have acces to docker daemon. https://dev.azure.com/cncf/envoy/_build/results?buildId=11205&view=logs
where will be the right way to add access to the docker daemon?

@lizan good news,  I think this almost works. Now I want to do the following I want to run regression tests on every PR/commit and fuzzing only on pushes to master (or daily cron from master). what would be the right way to configure this in the AzurePipelines

@lizan good news! Looks like we are on track. Now you or @htuch need to sign up at https://app.fuzzit.dev so I can add you to envoy account so you can take over. Also once I add you to envoy account you need to copy FUZZIT_API_KEY to AZP environment variables.

@lizan you should now have access to https://app.fuzzit.dev/orgs/envoyproxy/settings

@lizan fixed the review:)

@htuch?

@htuch why this fails now ?https://circleci.com/gh/envoyproxy/envoy/279964?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link

/retest

@htuch passing!:)

@htuch @lizan sure thing, thank you! and feel free to ping me if there are any issues/changes you need.
also @lizan are you sure the FUZZIT_API_KEY is available because I see that the script didn't find it (I couldn't check it before merge as this is environment variable is not available for forked-PRs)
Also looks there are some crashes already https://app.fuzzit.dev/orgs/envoyproxy/dashboard
and of course feel free to RT https://twitter.com/fuzzitdev/status/1176246876821217283
Thanks!

@yevgenypats are these crashes limited in visibility? We would like to maintain embargo on any zero days and only have https://github.com/google/oss-fuzz/blob/master/projects/envoy/project.yaml#L3 see these.

@htuch yes of course, like we discussed. currently only @lizan has access I can add you to the email notifications but you need to sign-up at https://app.fuzzit.dev so I can add you to the envoy account so you will have access to the crashes and other data.

I set it right now, should be fixed in future builds

Great.
On Mon, Sep 23, 2019, 5:55 PM Lizan Zhou<EMAIL_ADDRESS>wrote:

I set it right now, should be fixed in future builds
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/envoyproxy/envoy/pull/7509?email_source=notifications&email_token=AD52CDUYCVHBV4TWPG5AYHTQLE3LVA5CNFSM4H7ICZEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7MMOZY#issuecomment-534300519,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AD52CDRJ4YHQ2IFSKL2UCEDQLE3LVANCNFSM4H7ICZEA
.

@yevgenypats now it is failing with
+ ./fuzzit create target --skip-if-exists --public-corpus envoyproxy/access-log-formatter
2019/09/23 23:20:23 Creating target...
2019/09/23 23:20:23 target can only contain lowercase characetrs, numbers and hypens


Fixed transpile_scoring documenation

There was an error in the documentation for the output of the transpile_scoring function, I switched it to the correct description of the output.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Get Username from UserID

Hey,
I would like to know if it's possible to get the username of profiles using their user ID which is downloaded by the Instaloader. I've never downloaded this profile but it had commented on my post (I have the comments JSON file) and now I can't find that account on Instagram. So any way to get the current username of the account using their user ID?

https://github.com/instaloader/instaloader/blob/9ffdf258a9441ca680864fb468facfa261df9491/instaloader/structures.py#L617
Profile.from_id

It worked. Thanks a lot!

I did not understand that exactly now. What exactly should you do with the code and what should you replace?


[QUESTION] When I try to create a PersistentVolumeClaim (PVC) to use Longhorn, I encountered an issue. I have two nodes in Longhorn and the replication factor is set to 1. However, I found that the Pod is unable to bind successfully with the PVC.

Hi：
When I try to create a PersistentVolumeClaim (PVC) to use Longhorn, I encountered an issue. I have two nodes in Longhorn and the replication factor is set to 1. However, I found that the Pod is unable to bind successfully with the PVC.
apiVersion: v1
kind: ConfigMap
metadata:
name: longhorn-storageclass
namespace: longhorn-system
labels:
app.kubernetes.io/name: longhorn
app.kubernetes.io/instance: longhorn
app.kubernetes.io/version: v1.5.3
data:
storageclass.yaml: |
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: longhorn
annotations:
storageclass.kubernetes.io/is-default-class: "true"
provisioner: driver.longhorn.io
allowVolumeExpansion: true
reclaimPolicy: "Retain"
volumeBindingMode: Immediate
parameters:
numberOfReplicas: "1"
staleReplicaTimeout: "30"
fromBackup: ""
fsType: "ext4"
dataLocality: "disabled"
kubectl  logs -f -n longhorn-system   longhorn-manager-x6xrh
<IP_ADDRESS> - - [18/Dec/2023:09:49:50 +0000] "GET /v1/volumes/pvc-2eb660ef-7878-4f8c-9ec3-bb4bbc06aacb HTTP/1.1" 200 6572 "" "Go-http-client/1.1"
time="2023-12-18T09:49:50Z" level=warning msg="HTTP handling error" func=api.HandleError.func1 file="router.go:21" error="unable to attach volume pvc-2eb660ef-7878-4f8c-9ec3-bb4bbc06aacb to <IP_ADDRESS>: node.longhorn.io "<IP_ADDRESS>" not found"
<IP_ADDRESS> - - [18/Dec/2023:09:49:50 +0000] "POST /v1/volumes/pvc-2eb660ef-7878-4f8c-9ec3-bb4bbc06aacb?action=attach HTTP/1.1" 404 314 "" "Go-http-client/1.1"

attachdetach-controller@
Message
AttachVolume.Attach failed for volume "pvc-2eb660ef-7878-4f8c-9ec3-bb4bbc06aacb" : rpc error: code = Internal desc = Bad response statusCode [404]. Status [404 Not Found]. Body: [message=unable to attach volume pvc-2eb660ef-7878-4f8c-9ec3-bb4bbc06aacb to <IP_ADDRESS>: node.longhorn.io "<IP_ADDRESS>" not found, code=Not Found, detail=] from [http://longhorn-backend:9500/v1/volumes/pvc-2eb660ef-7878-4f8c-9ec3-bb4bbc06aacb?action=attach]
Please help me.

Does it have anything to do with me changing the path below?

error="unable to attach volume pvc-2eb660ef-7878-4f8c-9ec3-bb4bbc06aacb to <IP_ADDRESS>: node.longhorn.io \"<IP_ADDRESS>\" not found"
Looks like you are trying to start the workload on a node which doesn't Longhorn component running. Can you provide:
kubectl get nodes and kubectl get nodes.longhorn.io -n longhorn-system?

Yes, it seems that only two nodes are running, and the node 233 is missing. However, how should I proceed with this? My understanding is that the storage should be on nodes 234 and 235, but can other nodes still be used? I don't quite understand.

seems that only two nodes are running, and the node 233 is missing. However, how should I proceed with this? My understanding is that the storage should be on nodes 234 and 235, but can other nodes still be used? I don't quite understand.

Hey, there are 2 parts:

If you want the workloads that use Longhorn volume on node 233, you must deploy Longhorn components on that node
Then, if you want Longhorn to use the disk on node 234 and 235 to store its data, you can add this label node.longhorn.io/create-default-disk=true to node 233, 235 AND Install Longhorn with the setting Create Default Disk on Labeled Nodes set to true.

This is the instruction that fit your use-case: https://longhorn.io/kb/tip-only-use-storage-on-a-set-of-nodes/#tell-longhorn-to-create-a-default-disk-on-a-specific-set-of-nodes

ots/Screenshot%20from%202023-12-19%2015-06-35.png

Yes, other nodes are functioning correctly. I just wanted to store data on nodes 234 and 235. I tried the solution you provided. However, I have some doubts. I set the option node.longhorn.io/create-default-disk=true for the nodes and then deleted the previous Longhorn configuration. After redeploying, I noticed that my defined storage paths became ineffective, and the default paths were being used instead."

@linux-golang
The appoarch  https://longhorn.io/kb/tip-only-use-storage-on-a-set-of-nodes/#tell-longhorn-to-create-a-default-disk-on-a-specific-set-of-nodes required to full uninstall Longhorn then re-install Longhorn. Did you full uninstall Longhorn (https://longhorn.io/docs/1.5.3/deploy/uninstall/)?

Thank you very much for your support.


Staging next

Motivation for this change
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

perl5.30.1-IPC-System-Simple-1.25 fails, cc @volth https://hydra.nixos.org/build/111420639

The failing perl package can be fixed on master.

perl5.30.1-IPC-System-Simple-1.25 fails, cc @volth https://hydra.nixos.org/build/111420639

but.. there should 1.26
nix-build -E '(import (builtins.fetchTarball https://github.com/NixOS/nixpkgs/archive/staging.tar.gz) {}).perlPackages.IPCSystemSimple'
does work and downloads 1.26 from cache.nixos.org

And even your 1.25 link https://hydra.nixos.org/build/111420639 shows success


Add verification of known emails to the Register section

A possible improvement focused on the Bluesky registration process was identified. Currently, the platform does not present any validation for the emails admitted during the registration section; and by mistake, it is possible that a user does not place the emails correctly. For this reason, a list of the most common email addresses is added that will help the user verify whether or not they made a mistake.

This isn't needed. You can update your email before you verify it.

Also. Your email "validation" restricts all possible email domains to 3 (Microsoft, Google, Yahoo), which is insane.

i've no idea where you're getting this list of email provider domains from but it's certainly not very up to date and i'm doubting the legitimacy of some of them

You're also removing basic validation checks and messing up library imports.

Thanks, but this isn't something that we're going to do, even with a list of valid domain names. For example, this would restrict me from signing up with a work email address, which is obviously legitimate but wouldn't pass this check.
The only check I think we could reasonably add here is to verify the user enters a valid TLD. Otherwise, email verification itself would need to take place (for example to address possible typos) outside of the client itself.


add a "source" cmd to hyprctl

Describe your PR, what does it fix/add?
adds a source cmd to hyprctl which lets you load configs through the socket
hyprctl source <path/to/conf>

hyprctl keyword source ?

oh 💀


Initial checkin for a new identity & policy contract family

THIS CONTRACT FAMILY IS WORK IN PROGRESS
Initial commit for a new contract family for identity and policy. An identity is a contract that manages a collection of hierarchically generated keys. As a result, a standard format for keys would involve <contract_id>//key2>/....
The policy support is incomplete. Documentation for (and serialization/deserialization of) verifiable credentials and verifiable presentations is available.

Superceded.


Added: ArrayLiteral#shuffle to macros

I tried to implement #sort too, but I just plain failed.

Thanks! I was going to ask for a spec, but shuffle is hard to test :wink:
Why did you fail on sort? I guess it's because there's no way to sort AST nodes. I think I'll implement it for some literals, like StringLiteral, NumberLiteral, etc., and for others it will give a compilation error. I'll do it latr.

I couldn't get crystal to compile, even with a basic return ArrayLiteral.new(elements.sort), and didn't understand how to prepare the block arguments when passing a block, and so on.
I thought about tests, but yeah, that's complicated to test!


Darwin builds both standalone and mac_gcc in CI, duplicating work

Problem
When looking at https://github.com/project-chip/connectedhomeip/runs/5681814706?check_suite_focus=true I saw that clang-tidy was tidying every single file 2x.
it seems reason is that the bulild generates both standalone and mac_x64_gcc outputs which seem similar enough that the same files are compiled.
Proposed Solution
Figure out if there is a difference what it is and only compile things once. Worst case, update filters in clang-tidy to only tidy once. Ideally we should only build one output to speed up these two steps by roughly 50%.

Linux has the same thing between the "Build" and "Linux Standalone" jobs...


Handle parent of geotables with subdomains

closes https://github.com/JuliaEarth/GeoStats.jl/issues/378

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (277baf6) 84.22% compared to head (6d327ff) 84.60%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master      #98      +/-   ##
==========================================
+ Coverage   84.22%   84.60%   +0.37%
==========================================
  Files          18       18
  Lines         818      838      +20
==========================================
+ Hits          689      709      +20
  Misses        129      129

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


 How can I reproduce?

export APP_ID=your-app-id
export APP_TOKEN=token-of-your-application
export ACCOUNT_SERVER_PROTO=https
export ACCOUNT_SERVER_USERNAME=your-oauth-client-id
export ACCOUNT_SERVER_PASSWORD=your-oauth-client-secret
export ACCOUNT_SERVER_URL=account.thethingsnetwork.org

before running the test

Coverage decreased (-0.01%) to 74.146% when pulling 2a8f1e47b5d45783d858dbd96266026727cbf696 on feature/non-nullable-fields into 39c2d385624d45fd70f2840ab88534c95a8a209e on develop.


Update references to Selenium v4 to point to stable release

Updating references to Selenium v4 pre-release versions in our docs to ensure they are pointing to the official release now that it is available: https://www.selenium.dev/blog/2021/announcing-selenium-4/
There are some Nuget.org links that had a specific version in the URL. I've removed these so that following the link will take the user to the latest release, which will be at least 4.0.0 as of today.
Also removed usages of the UseChromium property in the WebView2 automation article because that article is based on Selenium 4 only and UseChromium is no longer present there. I've left these usages in the main WebDriver article for now and will address them in a follow-up PR because that article still targets both v3 and v4 and detangling them will take a little more time.

Docs Build status updates of commit 6c7169a:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

microsoft-edge/webdriver-chromium/index.md
:white_check_mark:Succeeded
View

microsoft-edge/webview2/how-to/webdriver.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit c1ef2ec:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

microsoft-edge/webdriver-chromium/index.md
:white_check_mark:Succeeded
View

microsoft-edge/webview2/how-to/webdriver.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


About front 180° and front facing

Does front facing mean that -20<yaw<20 and -20<pitch<20 ?

The entire test set (All 360◦) ,
Front 180◦ (within 90)
Front facing (within 20)
This only apply to gaze360 dataset


Run example site as hugo module

This PR turns the theme into a hugo module and runs the example site with the theme as hugo module.

Hey @deining, Thanks for your contribution. I am merging this PR. Happy coding...


Implement Export to PDF

https://github.com/libharu/libharu

For ideas that might be good to graph:
I think that the application should give two sections to the PDF "Report". It should provide a summary page, with simple data such as:

Total Income vs. Total Expenses.
Percentage of money in assets/how assets are doing (if this is ever supported)
etc.

While the further pages should contain more advanced information and graphs, which could be useful for people who are using this application as a way to manage a company/more complex finances. These sort of more advanced statistics would be a good step forward if the aim of this application was to replace non FOSS accounting software.
An example of a "more advanced statistic" would be a break down of losses (e.g. inflation, spending and investment returns (if this is ever supported)).
Another option should be to remove hard financial figures, and replace them with percentages, allowing the user to share their financial data, without relieving their wealth, which would be a good extra feature.
I also thing that it would be a good idea to present this data in-app. Either way, I'll look into some more ideas
that would be good to implement.

I also thing that it would be a good idea to present this data in-app. Either way, I'll look into some more ideas that would be good to implement.

An initial page with graphs like a dashboard would be really great. One graph with my expenses as an example would help me see where I'm using more money in a faster way in comparison with other groups (medicines, travels, food, etc.).

You can track PDF progress on this branch: https://github.com/nlogozzo/NickvisionMoney/tree/pdf-reports
So far we have a basic blank page:

Test.pdf
If someone would like to open the PDF in LibreOffice Draw and create a mockup of how you would like everything drawn out that would be nice :)
I know @fsobolev is great at UI design but idk about Document Design :smile:

A question i would have for the PDF is if it should follow the GNOME Style? I personally think so, but at the same time applications like apostrophe don't, so I guess it is up for debate?

A question i would have for the PDF is if it should follow the GNOME Style? I personally think so, but at the same time applications like apostrophe don't, so I guess it is up for debate?

I don't really know how to create a "GNOME-Styled" PDF since all i can draw are lines and text.
The most i could do is use the default gnome font for the PDF.

Oh, that is understandable, but will you be able to plot graphs?

I should be able to although I may need to draw the whole graph manually. I'm using libharu as the pdf library, which I'm slowly learning as I go along.

Love drawing PDFs with C code :laughing:
//Initialize PDF
HPDF_Doc pdf{ HPDF_New([](HPDF_STATUS, HPDF_STATUS, void* data){ *reinterpret_cast<bool*>(data) = false; }, &success) };
//First Page
HPDF_Page page1{ HPDF_AddPage(pdf) };
HPDF_Page_SetSize(page1, HPDF_PAGE_SIZE_LETTER, HPDF_PAGE_PORTRAIT);
//First Page - Main Box
HPDF_Page_SetLineWidth(page1, 0.5);
HPDF_Page_Rectangle(page1, 10, 10, HPDF_Page_GetWidth(page1) - 20, HPDF_Page_GetHeight(page1) - 20);
HPDF_Page_Stroke(page1);
//First Page - Font
HPDF_Font fontTitle{ HPDF_GetFont(pdf, "Helvetica", nullptr) };
//First Page - Title
HPDF_Page_SetFontAndSize(page1, fontTitle, 11);
HPDF_Page_BeginText(page1);
HPDF_Page_MoveTextPos(page1, 14, HPDF_Page_GetHeight(page1) - 24);
HPDF_Page_ShowText(page1, std::filesystem::path(m_path).filename().c_str());
HPDF_Page_EndText(page1);
//First Page - Icon
HPDF_Image imageIcon{ HPDF_LoadJpegImageFromFile(pdf, pathToSymbolicIcon.c_str()) };
HPDF_Page_DrawImage(page1, imageIcon, HPDF_Page_GetWidth(page1) - 50, HPDF_Page_GetHeight(page1) - 50, 32, 32);
//First Page - Income
HPDF_Page_SetFontAndSize(page1, fontTitle, 9);
HPDF_Page_BeginText(page1);
HPDF_Page_MoveTextPos(page1, 20, HPDF_Page_GetHeight(page1) - 54);
HPDF_Page_ShowText(page1, std::string("Income: " + MoneyHelpers::boostMoneyToLocaleString(getIncome(), locale)).c_str());
HPDF_Page_EndText(page1);
//First Page - Expense
HPDF_Page_BeginText(page1);
HPDF_Page_MoveTextPos(page1,20, HPDF_Page_GetHeight(page1) - 70);
HPDF_Page_ShowText(page1, std::string("Expense: " + MoneyHelpers::boostMoneyToLocaleString(getExpense(), locale)).c_str());
HPDF_Page_EndText(page1);
//First Page - Total
HPDF_Page_BeginText(page1);
HPDF_Page_MoveTextPos(page1, 20, HPDF_Page_GetHeight(page1) - 86);
HPDF_Page_ShowText(page1, std::string("Total: " + MoneyHelpers::boostMoneyToLocaleString(getTotal(), locale)).c_str());
HPDF_Page_EndText(page1);
//Save and Close PDF
HPDF_SaveToFile(pdf, path.c_str());
HPDF_Free(pdf);

That, is an impressive amount of code.
On Sat, Nov 26 2022 at 11:21:29 -0800, Nick Logozzo
@.***> wrote:

https://user-images.githubusercontent.com/17648453/204105658-71af6b4b-d959-4bf8-966e-fa11848f0c60.png
Love drawing PDFs with C code 😆
//Initialize PDF
HPDF_Doc pdf{ HPDF_New([](HPDF_STATUS, HPDF_STATUS, void* data){
reinterpret_cast<bool>(data) = false; }, &success) };
//First Page
HPDF_Page page1{ HPDF_AddPage(pdf) };
HPDF_Page_SetSize(page1, HPDF_PAGE_SIZE_LETTER, HPDF_PAGE_PORTRAIT);
//First Page - Main Box
HPDF_Page_SetLineWidth(page1, 0.5);
HPDF_Page_Rectangle(page1, 10, 10, HPDF_Page_GetWidth(page1) - 20,
HPDF_Page_GetHeight(page1) - 20);
HPDF_Page_Stroke(page1);
//First Page - Font
HPDF_Font fontTitle{ HPDF_GetFont(pdf, "Helvetica", nullptr) };
//First Page - Title
HPDF_Page_SetFontAndSize(page1, fontTitle, 11);
HPDF_Page_BeginText(page1);
HPDF_Page_MoveTextPos(page1, 14, HPDF_Page_GetHeight(page1) - 24);
HPDF_Page_ShowText(page1,
std::filesystem::path(m_path).filename().c_str());
HPDF_Page_EndText(page1);
//First Page - Icon
HPDF_Image imageIcon{ HPDF_LoadJpegImageFromFile(pdf,
pathToSymbolicIcon.c_str()) };
HPDF_Page_DrawImage(page1, imageIcon, HPDF_Page_GetWidth(page1) - 50,
HPDF_Page_GetHeight(page1) - 50, 32, 32);
//First Page - Income
HPDF_Page_SetFontAndSize(page1, fontTitle, 9);
HPDF_Page_BeginText(page1);
HPDF_Page_MoveTextPos(page1, 20, HPDF_Page_GetHeight(page1) - 54);
HPDF_Page_ShowText(page1, std::string("Income: " +
MoneyHelpers::boostMoneyToLocaleString(getIncome(), locale)).c_str());
HPDF_Page_EndText(page1);
//First Page - Expense
HPDF_Page_BeginText(page1);
HPDF_Page_MoveTextPos(page1,20, HPDF_Page_GetHeight(page1) - 70);
HPDF_Page_ShowText(page1, std::string("Expense: " +
MoneyHelpers::boostMoneyToLocaleString(getExpense(),
locale)).c_str());
HPDF_Page_EndText(page1);
//First Page - Total
HPDF_Page_BeginText(page1);
HPDF_Page_MoveTextPos(page1, 20, HPDF_Page_GetHeight(page1) - 86);
HPDF_Page_ShowText(page1, std::string("Total: " +
MoneyHelpers::boostMoneyToLocaleString(getTotal(), locale)).c_str());
HPDF_Page_EndText(page1);
//Save and Close PDF
HPDF_SaveToFile(pdf, path.c_str());
HPDF_Free(pdf);
—
Reply to this email directly, view it on GitHub
https://github.com/nlogozzo/NickvisionMoney/issues/3#issuecomment-1328100600,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARSOFBJTXUARJWYFHTUHIADWKJPLTANCNFSM6AAAAAARSBKZFQ.
You are receiving this because you commented.Message ID:
@.***>

The PHP library FPDF has a lot of examples of scripts that you might find helpful as a reference/resource when making the pdf generation tool.
http://www.fpdf.org/en/script/ex20.pdf
http://www.fpdf.org/en/script/script20.php
http://www.fpdf.org/en/script/ex101.pdf
http://www.fpdf.org/en/script/script101.php
http://www.fpdf.org/en/script/index.php
http://www.fpdf.org/
While it is a completely different library in a completely different languish, I hope that you will find something there that might help you.

Love drawing PDFs with C code laughing

I can help you write a wrapper around that! It would be great if someone could make or find a beautiful template for financial information.
I'll be able to work on it after Dec 8th in case you haven't done it by then.

Love drawing PDFs with C code laughing

I can help you write a wrapper around that! It would be great if someone could make or find a beautiful template for financial information.
I'll be able to work on it after Dec 8th in case you haven't done it by then.

Thanks for the idea! I built a light wrapper for a PDF page and document that I am building as I go along. https://github.com/nlogozzo/NickvisionMoney/commit/2e22f48fe4978dc675462e25448f57cca8851729

Perhaps the diagram could also be drawn in the application using the DrawingArea widget.

report_template.odt
I know this is low-effort production but see if this inspires you. Public domain/CC-BY.

Here's some information that could be generated and used.
Basic, doesn't require new function
Income-Expense Ratio
Average Income/day
Average Expense/day
Net Income
Top Expense Groups

These could be calculated by the selected range
Extensions---would require new function(s) and may not be useful for some users
Overdraft | requires (saving) target
Debts | requires data type "debts" #90
Foreign Currency Balance | requires foreign currency (re-)support #128
Estimated Credit Interest | requires input of interest rate

There's 12 columns in the report template. The basics ones requires 7 columns (assuming Top Expense Group lists 3). Or we could use every other column.

The indent of the income and expense could be modified from -0.06 (both before-text and after) to -0.16 to fit 100 vertical bar for easier calculation. You'd need to change the indent of "income" and "expense" to -0.10 before-text as well.

Foreign Currency Balance | requires foreign currency (re-)support #128

Custom currency option shown in #128 is not about additional currency for an account, but about setting a currency that is not the one defined by a user locale. And even if we would add ability to show balance in multiple currencies, it's very problematic to convert (see https://github.com/nlogozzo/NickvisionMoney/discussions/130#discussioncomment-4362211).

And even if we would add ability to show balance in multiple currencies, it's very problematic to convert (see https://github.com/nlogozzo/NickvisionMoney/discussions/130#discussioncomment-4362211).

It's not about converting to local currency. That's another feature (Local currency equivalent). We could instead just use the currency sign (take £ as example) and display it like £300.00. I guess this is common in saving accounts?
And for the multiple currencies part, it could probably be a hidden group (sorry I only know a bit python so can't really help there)

Feature all most done! Just need to find a place to put receipt images.
(Fancy reporting and PDFs for reports will come in a future version. I'll make an issue to track that soon)

https://github.com/nlogozzo/NickvisionMoney/releases/tag/2023.1.0-rc1

I'm sorry, but is this feature implemented? I cannot find it in the app.

I'm sorry, but is this feature implemented? I cannot find it in the app.

Yes, it is, open an account -> "Actions" menu -> "Export to File" -> "PDF"

Thanks!

Maybe the PDF could include diagrams?
A pie chart that shows expenses for each group.
A line chart that shows expenses for each month.
A heat map that shows expenses for every day of the year.

Maybe the PDF could include diagrams?
A pie chart that shows expenses for each group.
A line chart that shows expenses for each month.
A heat map that shows expenses for every day of the year.

That will come with #289


[BUG] [scratchpads] Abnormal window position in special workspace with animation fromTop and formBottom

Pyprland version
Which version did you use? (copy & paste the string returned by pypr version)
2.4.0
Describe the bug
A clear and concise description of what the bug is.
When toggling a scratchpad in a special workspace, the window position is abnormal. And it only happens on the vertical.
https://github.com/user-attachments/assets/03eafcb0-2e4a-4c0f-b5b0-4f8dd15d5427
It seems the ture window still in the correct position, but it shows in a wrong position. I can draw it in the original position:
https://github.com/user-attachments/assets/22c64e8d-bbdc-4683-979f-de909171748c
To Reproduce
Steps to reproduce the behavior:
just toggle scratchpads in sepcial workspace.
Configuration (provide following files/samples when relevant):

pyprland.toml

[scratchpads.term]
animation = "fromTop"
command = "kitty --class kitty-dropterm"
class = "kitty-dropterm"
size = "75% 60%"
max_size = "1920px 100%"
margin = 50
multi = false

[scratchpads.filemanager]
animation = "fromLeft"
command = "dolphin"
size = "75% 70%"
margin = 50
multi = false
lazy = true
skip_windowrules = ["aspect"]

[scratchpads.music]
animation = "fromBottom"
command = "yesplaymusic"
class = "yesplaymusic"
size = "70% 60%"
lazy = true
multi = false

[scratchpads.obsidian]
animation = "fromRight"
command = "obsidian"
class = "obsidian"
size = "80% 80%"
lazy = true
multi = false

hyprland.conf

Additional context
Add any other context about the problem here.
Hyprland version:
Hyprland, built from branch main at commit 293e687389a19b369f312c5c335c9afe7c886be1  (renderer: Make shader time always count from zero (6903)).
Date: Wed Jul 17 04:03:10 2024
Tag: v0.41.2-43-g293e6873, commits: 4929

flags: (if any)

I also face this issue on my work setup, it can not be a Pyprland bug, it's something in Hyprland...
I couldn't find a way to work around it tbh... I think we need to wait for more stability in Hyprland.

Using no animations also spawn windows in the wrong positions and hiding/showing them does not resolve the issue.

Pretty sure this is Hyprland issue, do you use any monitor scaling or rotation? Thise is often a factor introducing some issues there...

The only scale factor that i have in my config is: special_scale_factor = 0.8 + no rotations. My hyprland config can be found here.

If you have a reproducible scenario, you can try opening a bug on hyprwm...

I'm closing this bug, I really don't think it is something in Pyprland...


docs(vue):  up-to-date devtools information

need to help users clarify their current status
also maybe write information about https://github.com/TanStack/query/issues/7228#issuecomment-2171894507
cc @DamianOsipiuk siema :)

It would be nice to reorder this page to promote component based devtools at the top, as they are more up-to-date and have more features.

It would be nice to reorder this page to promote component based devtools at the top, as they are more up-to-date and have more features.

could you change my PR as you would like ?

LGTM


Add NLT to ExtractorBindingPattern, ExtractorAssignmentPattern, and ExtractorMemberExpression

This adds [no |LineTerminator| here] assertions to ExtractorBindingPattern, ExtractorAssignmentPattern, and ExtractorMemberExpression to align with Option (2) as discussed in #24:

Introduce an NLT in ExtractorBindingPattern and ExtractorAssignmentPattern.

This is an alternative solution to the one implemented in #25.
The outcome of this change is as follows:

For
let x
(a) = b

the source is interpreted as let x; (a) = b; due to the NLT assertion and ASI.

For
let x
[a](b) = c

the source is interpreted as let x; [a](b) = c; due to the NLT assertion and ASI. Note that [a](b) is not a valid ExtractorAssignmentPattern as [a] is not a valid ExtractorMemberExpression and will result in an Early Error when applying static semantics for destructuring assignment.

For
let x[a]
(b) = c

the source is interpreted as let x[a]; (b) = c; due to the NLT assertion and ASI and results in a Syntax Error at parse time.

For
x
(a) = b

the source is interpreted as x(a) = b; when parsing AssignmentExpression, but will result in an Early Error when applying static semantics for destructuring assignment since ExtractorAssignmentPattern has an NLT restriction. Since this is enforced as a static semantics rule, ASI will not trigger.

For
x
[a](b) = c

the source is interpreted as x[a](b) = c; when parsing AssignmentExpression, but will result in an Early Error when applying static semantics for destructuring assignment since ExtractorAssignmentPattern has an NLT restriction. Since this is enforced as a static semantics rule, ASI will not trigger.

For
x
[a]
(b) = c

the source is interpreted as x[a](b) = c; when parsing AssignmentExpression, but will result in an Early Error when applying static semantics for destructuring assignment since ExtractorAssignmentPattern has an NLT restriction. Since this is enforced as a static semantics rule, ASI will not trigger.

Fixes #24
Related #25

LGTM - i would be shocked if anyone intentionally wrote code with parens on the LHS of an assignment in the current century, and I don't have much empathy for someone who puts the opening invocation paren on a separate line from the thing being invoked :-)

LGTM. I have the same lack of sympathy for anyone writing code like that (it's frankly nonsense that JS allows you to put whitespace between the function name and the parens...), but more importantly, even if the restrictions end up happening via different methods, it does look like the practical effect of this NLT is consistent with the current behavior. All of these examples are invalid today due to being invalid assignments or bindings, they'll just switch to becoming invalid due to static semantics for destructuring.


Fix code 228A

I don't know how can I fix the looping problem. Sometimes, counter is "countering" more than the expected.
For example, if input is 1 3 3 3, the expected answer is 2. However, in my code, returns the answer 3.

I solve this problem using Python, but I wanna do this using C

Nailed it! I wrote "for ... in ..." code in C


Unittests for clarity/data/*.py; introduces tests/conftest.py

Trying again as #88 was messy and confusing. Branched freshly from main and pulled over tests/data/test_{utils,HOA_tools_cec2}.py as well as tests/conftest.py, then added the try: ... except: ... statements I'd introduced to clarity/data/utils.py and clarity/data/HOA_tools_cec2.py.
All but two tests pass locally, the two that fail appear to be due to Torch trying to use my CPU and GPU (as mentioned in #97).
I've also made sure the .pre-commit-config.yaml uses up-to-date versions of each virtual environment (see comment).

Thanks @TuZehai :+1:


Cannot find reference 'GenericAssistant' in 'imported module neuralintents'

Its not finding generic assistant

@NeuralNine


Cannot update on Manjaro from AUR

This is the error:
Synchronizing package databases...
Starting full system upgrade...

Building ungoogled-chromium-bin...
Cloning into 'ungoogled-chromium-bin'...
==> Making package: ungoogled-chromium-bin 69.0.3497.100-5 (mié 10 oct 2018 00:33:58 CEST)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
  -> Downloading ungoogled-chromium_69.0.3497.100-2_linux.tar.xz...
curl: (3) Invalid file://hostname/, expected localhost or <IP_ADDRESS> or none
==> ERROR: Failure while downloading file://ungoogled-chromium_69.0.3497.100-2_linux.tar.xz
    Aborting...

same here. I bet there's something faulty at the PKGBUILD. I think 'hostname' has to be replaced with <IP_ADDRESS> or localhost. Maybe I'm wrong and the whole thing needs to be replaced by a valid url.

This isn't our PKGBUILD. Please report the error to that PKGBUILD's authors.


Add huntr.dev to SECURITY.md

As requested through the platform by @davidjgraph, this will point your security policy to huntr.dev

Changed file directly


Enhancement: Neuropixels ELT

The neuropixels etl will update modification date and rig_id if the model hasn't changed. We would like it to only update these values if other fields in the model change. This enhancement is implemented here: #47

Closing this as completed by #47


GEM-7489: Adds example scripts.

Example run-gfsh.sh
Example start-locator.sh
Example start-server.sh

I think you might be missing the point of these examples. These are examples of how to write scripts to directly start members or wrapper scripts around gfsh. We have customers that write these scripts because frankly gfsh is horrible for starting members in production. They aren't meant as product. We should make a product script... but that is for another fight.
So no, these script examples should not assert that certain properties, environment variables, etc. be set. What the customer wants to set in these are not our concern outside of them getting the bootstrapping correct.

That makes more sense.

LGTM.
Given that the start-server.sh does not contain locators property, we might add that to the README file, showing HOW to use the scripts. OR we have to add the locators property to the start-server script

Oh, good call. That would have been a smart thing to include in the examples. 🤦

Sadly locators addresses are not a first class configuration argument. 😢


Player joining is shown as {x2} in chat, for the first message

A player joined a server and saw "PlayerName joined the game {x2}". The logs don't show the {x2} and just the regular message.
There is no inidication what goes wrong.

A second player just reported the same issue.


update json/body parser example

some of these /examples files are hard to read, spaces for indentation, unnecessary or confusing code, I would update the json body parse example with a better readBody helper function

rename file


upgrade carvel image bundle from 0.1 to 0.11 to add auth setup

Signed-off-by: Jinheng Xu<EMAIL_ADDRESS>

The new image bundle has been pushed
docker pull projects.registry.vmware.com/kubeflow/kubeflow-carvel-testing:0.11
docker pull projects.registry.vmware.com/kubeflow/kubeflow-carvel-repo:0.11

Clusters that installed the previous version should be unaffected.
docker pull projects.registry.vmware.com/kubeflow/kubeflow-carvel-testing:0.11
docker pull projects.registry.vmware.com/kubeflow/kubeflow-carvel-repo:0.11


Created tasks-list module; moved jschema to components module

Checklist before merging

[x] I added an appropriate entry to CHANGELOG.md

Closes #666

Regarding the warning:

▲ [WARNING] Cannot find base config file "./.svelte-kit/tsconfig.json" [tsconfig.json]
../jsconfig.json:2:12:
  2 │   "extends": "./.svelte-kit/tsconfig.json",
    ╵              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

TL;DR: It disappears if you run npm install on the parent folder.
The build detects that there is a jsconfig.json file in the parent folder, so it understands that is currently building a subproject. It then checks if there is a file .svelte-kit/tsconfig.json in the parent folder and doesn't find it, so the warning is displayed. To remove the warning we could run npm install on the parent folder before executing the subproject build, because running npm install also generates the .svelte-kit folder.
However, as stated by a comment on related issue:

the warning is supposed to be harmless. It indicates that whatever build system you use (vercel, netlify, multistage docker builds, ...) didn't populate the .svelte-kit folder before invoking vite build.

So, since we don't really need to run npm install on the parent folder, I would keep our CI steps as they are, to avoid running an extra step only for removing the warning.


Logs subcommand

deployctl logs
Shows the logs of the given project.

To show the latest logs of a project:
  deployctl logs --project=helloworld

To show the logs of a particular deployment:
  deployctl logs --project=helloworld --deployment=1234567890AB

To show the logs of the production deployment:
  deployctl logs --project=helloworld --prod

USAGE:
    deployctl logs [OPTIONS] [<PROJECT>]

OPTIONS:
        --deployment=<DEPLOY_ID>  The id of the deployment you want to get the logs (defaults to latest deployment)
        --prod                    Select the production deployment
    -p, --project=NAME        The project you want to get the logs
        --token=TOKEN             The API token to use (defaults to DENO_DEPLOY_TOKEN env var)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Wow!

Any updates on this getting merged?

up