Add docs for scaling tips

Once backend is done reviewing we should get a docs reviewer as well

@cbron I have included most of you recommendations and commented on any others. I left a new commit to help with review the new updates. I will squash once you are ready to approve.

@moio addressed feedback, including providing explicit instructions on what users can do in regards to apps v1 and apps v2.

Want to re-review one last time, then will merge.


terminate called after throwing an instance of 'cv::Exception'

error: (-210:Unsupported format or combination of formats) Failed to parse onnx model in function 'ONNXImporter'.
Ubuntu18.04, opencv4.1.0.
ncnn model is running ok, but onnx model can not run, always failing. can you help me? somebody help me out?

sorry, my bad. my model file path is wrong. thought a dir is enough, end up wrong. should be more specific.


Migrate draft UG

Transfer our draft user guide on Google Docs to our actual user guide in MarkDown.
Draft UG : https://docs.google.com/document/d/1ZCrtN0wttxK84nucsdoWTojkNpHVkhvWDMtA_rU5ng8/edit#heading=h.m6qyqo3zbc1h

I think this issue is too big and general? Maybe each person can create an issue specific to their work e.g. my issue should be "Add feature list and command summary to user guide" since I'm doing first page of our google docs


Rename jbds to devstudio

Since JBDS is forbidden for some time and sometimes we called it devstudio anyway. Initially only wanted to change element IDs but since some of them are loaded from requirements.json, I just replaced all jbds with devstudio.

Codecov Report

Merging #587 into master will increase coverage by <.01%.
The diff coverage is 63.63%.

@@            Coverage Diff             @@
##           master     #587      +/-   ##
==========================================
+ Coverage   85.25%   85.25%   +<.01%
==========================================
  Files          25       25
  Lines        2374     2510     +136
  Branches      207      207
==========================================
+ Hits         2024     2140     +116
- Misses        350      370      +20

Impacted Files
Coverage Δ

browser/pages/confirm/controller.js
82.75% <100%> (+0.74%)
:white_check_mark:

browser/services/data.js
100% <100%> (ø)
:white_check_mark:

browser/model/devstudio-autoinstall.js
100% <100%> (ø)

browser/pages/start/controller.js
44.79% <25%> (-0.04%)
:x:

browser/model/devstudio.js
55.06% <64.51%> (ø)

browser/main.js
92.59% <80%> (ø)
:white_check_mark:

browser/services/platform.js
86.71% <0%> (-1.75%)
:x:

browser/directives/breadcrumb.js
10% <0%> (-1.12%)
:x:

browser/services/request.js
20% <0%> (-0.84%)
:x:

... and 18 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a26b31f...49f11e7. Read the comment docs.

:+1:

Rebased on top of the master and included into new PR https://github.com/redhat-developer-tooling/developer-platform-install/pull/588.


2020.3 Intellij is coming

Hi, any plan to add support for this version?

Yep, lombok plugin will be already included in the comming IntelliJ 2020.3


try-runtime: Lazy Download

see: https://github.com/paritytech/substrate/discussions/13109

related: chopsticks storage layer https://github.com/AcalaNetwork/chopsticks/blob/master/packages/chopsticks/src/blockchain/storage-layer.ts

I tried out this subway to see if it could handle the batch request that you are currently fixing in one MR.
But it did not work. However, it is good for caching repeated queries.

I tried out this subway to see if it could handle the batch request that you are currently fixing in one MR. But it did not work. However, it is good for caching repeated queries.

This could be useful - but worth keeping in mind that (unless using a remote node) the biggest bottleneck is merkelizing the keys when loading them in TextExternalities, rather than fetching them from the node

Also, I think Centrifuge's fudge might have the right inspirational piece of code that lets you have Ext be backed by a database path: https://github.com/centrifuge/fudge

something that encodes and sends the whole Trie

This is state sync. This downloads the entire trie at a given block. It is used by warp sync.

Also, I think Centrifuge's fudge might have the right inspirational piece of code that lets you have Ext be backed by a database path: https://github.com/centrifuge/fudge

What you mean by backed by a database path? Ext is in "normal mode" always operating on a db, but I assume you want a db to cache your queries?

This is state sync. This downloads the entire trie at a given block. It is used by warp sync.

Thanks @bkchr this sounds very worth looking into. Perhaps not for lazy download but for saving entire state to disk that can quickly be loaded up again on subsequent runs (existing snapshots created by 'create-snapshot' still take 300+ seconds on my M2 Max Macbook Pro to load, bc building the tri is so expensive).
I've been meaning to create an issue to improve the performance of 'snapshot' mode, will mention this as an option to consider.

why does it need to build trie? there is (almost) no such logic in chopsticks

why does it need to build trie? there is (almost) no such logic in chopsticks

This is a good question. The trie may even be disadvantageous it'll slow things down a lot and it's unlikely that there'll be many keys needed cached in-memory at any one time.
Today I'll start hacking on a simple storage layer that accesses storage from a node and stashes results in a flat in-memory LRU cache. No trie, just flat key-value.

I'll start hacking on a simple storage layer that accesses storage from a node and stashes results in a flat in-memory LRU cache. No trie, just flat key-value.

You can just use the OverlayedChanges?

(existing snapshots created by 'create-snapshot' still take 300+ seconds

If I'm reading the code correctly, this could be because key-values are inserted in the trie one at a time here:
https://github.com/paritytech/substrate/blob/2b91202014e718525f90c17eb0545e2f2a3261ff/primitives/state-machine/src/testing.rs#L137
If you pass multiple/all values at once, it should be much faster because the trie root would not need to be recalculated after each insertion.

(existing snapshots created by 'create-snapshot' still take 300+ seconds

If I'm reading the code correctly, this could be because key-values are inserted in the trie one at a time here:
https://github.com/paritytech/substrate/blob/2b91202014e718525f90c17eb0545e2f2a3261ff/primitives/state-machine/src/testing.rs#L137
If you pass multiple/all values at once, it should be much faster because the trie root would not need to be recalculated after each insertion.

I assumed they were being inserted one at a time for a reason. I'll try batch inserting, if it works it'll speed things up dramatically, thanks for the advice!

why does it need to build trie?

Ideally, we want the try-runtime-enabled runtime to be able to operate in a normal mode. In that case, one requirement would be to be able to respond to storage::root host function correctly, ergo using a trie.
If we remove this assumption, a whole lot of things will be easier, and as @liamaharon pointed in a comment, a flat key -> value map would suffice. We could achieve this by implementing trait Externalities for a new type that works mostly the same, is backed by HashMap, and panics if you ask it to calculate the state root. As @bkchr said, this might very well be just OverlayedChanges.

Ext is in "normal mode" always operating on a db, but I assume you want a db to cache your queries?

Not just for caching, but also as the first point of response. The scenario I have in mind is that you run try-runtime-cli with some sub-command, and instead of your state being an RPC endpoint or a snapshot file, you give the path to an existing (RocksDB/ParityDB) polkadot/substrate database, and it would initialize an Ext based on that, and execute the commands there.

and panics if you ask it to calculate the state root

That isn't required, you can just take all the key/value pairs and calculate the storage root :P No problem.

Not just for caching, but also as the first point of response. The scenario I have in mind is that you run try-runtime-cli with some sub-command, and instead of your state being an RPC endpoint or a snapshot file, you give the path to an existing (RocksDB/ParityDB) polkadot/substrate database, and it would initialize an Ext based on that, and execute the commands there.

This is basically an enhanced OverlayedChanges. OverlayedChanges is already caching writes and then only goes to the state if the key is not yet written. Basically it should be enough for you to keep the OverlayedChanges changes around. This also already supports calculating a storage root etc.

instead of your state being an RPC endpoint or a snapshot file, you give the path to an existing (RocksDB/ParityDB) polkadot/substrate database, and it would initialize an Ext based on that, and execute the commands there.

fwiw, ➕ for this approach. In the case of data gathering for analysis and block replays with different state for testing purposes this would be very useful.

instead of your state being an RPC endpoint or a snapshot file, you give the path to an existing (RocksDB/ParityDB) polkadot/substrate database, and it would initialize an Ext based on that, and execute the commands there.

fwiw, ➕ for this approach. In the case of data gathering for analysis and block replays with different state for testing purposes this would be very useful.

@gpestana are there cases where reading from a DB path would be helpful in ways that reading from a snapshot would not? Snapshots can be created from arbitrary block numbers and (shortly) will be able to be loaded almost instantly.
Update on this: I've had success writing directly to the ext OverlayedChanges (rather than backend) which allows for  close to instant snapshot loading. WIP PR: https://github.com/paritytech/substrate/pull/14057

FYI, I have created a dedicated issue to track the progress of allowing db-backed externalities to be used with the try-runtime-cli. Let's maintain the focus of this issue on lazy-download.
Upon introducing a DB path option, I think lazy-download is likely to be a preferred mode only when using a remote node and the state cannot fit in memory.

Node Location
State Size
Preferred Mode

local
reasonable
snapshot

remote
reasonable
snapshot

local
large
db-backed

remote
large
lazy-download

Although we should definitely eventually implement lazy-download, I wanted to note that it may not be as high priority as some other try-runtime-cli tasks, because most bases will be covered and I'm uncertain how prevalent use case of using it with a large remote node will be.

I think its still a high value issue - just currently resource starved.  We instead prioritized the versioned runtime upgrades to reduce the number of error sources when migrating.
Hopefully once that is done maybe @liamaharon (OOO) can pick it up or distribute to someone.

Yeah I've made decent progress with this and have a working PoC, I am planing to pick it up again once the higher prio runtime upgrade stuff is shipped.
@crystalin after I clean up my branch would you be interested in helping out finish this off?

That's awesome.
I'll definitely help testing it

Hey @crystalin, we've decided to deprioritise this for now to focus on some higher priority features.
I have a PoC branch here you're welcome to pick up if you need the feature.

Moved: https://github.com/paritytech/try-runtime-cli/issues/8


Correct typo in pydoc  for MinOrMaxGrad

Can one of the admins verify this patch?

I signed it!

Jenkins, test this please.


Enact permissions onboarding flow for orgs with 'invalid' permissions

Context
We deliberately omitted enabling safeguarding and diversity permissions for existing organisations when setting up their default permissions.
This made some relationship permissions 'invalid' as at least one provider in the relationship should have each permission enabled. If the managing user in the organisation has not amended the permissions they are likely still in this state. We identified 181 records with this issue.
This has the side effect of restricting the visibility of safeguarding/diversity information completely if neither provider has the permission enabled. Users who are not able to manage permissions will be unaware of why they cannot see this information.

Changes proposed in this pull request

Amend the conditions of ProviderSetup#next_pending_relationship to return 'invalid' relationships. This will direct the managing user through the permissions setup flow when they log in.
Unskip a test for next_pending_relationship which was relying on record order, this adds an order clause to ensure we retrieve the oldest records first.

Guidance to review

Link to Trello card
This PR covers the task When a user with manage org/user permissions logs in and permissions haven't been set up, prompt them to set them up
https://trello.com/c/UAsvSnET/2868-dev-changes-tweaks-for-permissions

Things to check

[x] This code does not rely on migrations in the same Pull Request
[x] If this code includes a migration adding or changing columns, it also backfills existing records for consistency
[x] API release notes have been updated if necessary
[x] New environment variables have been added to the Azure config

I think we'll also need to fix #provider_relationship_permissions_needing_setup in ProviderRelationshipPermissionSetupController?

@duncanjbrown I've had a go at using the same piece of code to identify relationships manageable by the current provider user and then filtering these as suggested https://github.com/DFE-Digital/apply-for-teacher-training/pull/3161/commits/d2a614101ade8ae868114e34e8cd2542cdc3ed94

Looking good — left one comment! (on a commit 🤦) https://github.com/DFE-Digital/apply-for-teacher-training/commit/d2a614101ade8ae868114e34e8cd2542cdc3ed94#r43406191

@duncanjbrown I've amended as per your suggestion, I omitted the sort because the underlying query orders by created_at so the first selected relationship will be the oldest.


Checkbox with indeterminate state

Describe the feature in detail (code, mocks, or screenshots encouraged)
It would be useful to have checkbox with indeterminate state.
What type of pull request would this be?
None
Provide relevant links or additional information.
No response

@Tihomir971 this is possible with via an HTML checkbox element and a tiny bit of Javascript:
https://css-tricks.com/indeterminate-checkboxes/
var checkbox = document.getElementById("some-checkbox");
checkbox.indeterminate = true;

In Svelte you would do something like this:
const elemCheckbox;
elemCheckbox.indeterminate = true;

<input bind:this={elemCheckbox} type="checkbox">

We cover our stance on this on the forms page, but we recommend you lean on native form inputs wherever possible. As such as would likely never abstract this to a component. However, you certain can if you wish!


Can't import Bestiary 3 - Ratfolk Chemist

Can't import statblock:
Ratfolk Chemist      CR 7
XP 3,200
Ratfolk alchemist 8 (Pathfinder RPG Advanced Player's Guide 26, Pathfinder RPG Bestiary 3 231)
N Small humanoid (ratfolk)
Init +9; Senses darkvision 60 ft.; Perception +13
Defense
AC 24, touch 15, flat-footed 21 (+4 armor, +1 deflection, +3 Dex, +5 natural, +1 size)
hp 63 (8d8+24)
Fort +9, Ref +14, Will +3; +6 bonus vs. poison
Offense
Speed 20 ft. (15 ft. in armor), fly 40 ft. (good)
Melee dagger +6/+1 (1d3-1/19-20)
Ranged bomb +13/+8 (4d6+4 fire) or
frost bomb +13/+8 (4d6+4 cold) or
light crossbow +12 (1d6/19-20)
Special Attacks bomb 12/day (4d6+4 fire, DC 18), swarming
Alchemist Extracts Prepared (CL 8th; concentration +12)
3rd—cure serious wounds, fly, heroism
2nd—barkskin, bull's strength, invisibility, resist energy, see invisibility
1st—comprehend languages, crafter's fortune[APG] (DC 15), negate aroma[APG] (DC 15), reduce person (DC 15), shield
Tactics
Before Combat The chemist imbibes her mutagen and extracts of barkskin and fly. She gives infusions of bull's strength and heroism to allies.
During Combat The chemist drops bombs while flying above the fray.
Base Statistics Without her mutagen and barkskin, the chemist's statistics are Init +7; Senses Perception +14; AC 19, touch 15, flat-footed 16; Ref +12; Ranged light crossbow +10 (1d6/19-20) or bomb +10/+5 (4d6+4 cold or fire); Dex 16, Wis 12; CMD 18; Skills Heal +8, Perception +14, Profession (herbalist) +12, Survival +8.
Statistics
Str 8, Dex 20, Con 14, Int 18, Wis 10, Cha 8
Base Atk +6; CMB +4; CMD 20
Feats Brew Potion, Improved Initiative, Lightning Reflexes, Point-Blank Shot, Precise Shot, Throw Anything
Skills Acrobatics +2 (-6 to jump), Appraise +11, Craft (alchemy) +17 (+25 to create alchemical items), Disable Device +13, Fly +4, Heal +7, Knowledge (arcana) +15, Knowledge (nature) +15, Perception +13, Profession (herbalist) +11, Spellcraft +11, Survival +7; Racial Modifiers +2 Craft (alchemy), +2 Perception
Languages Common, Draconic, Dwarven, Gnome, Undercommon
SQ alchemy (alchemy crafting +8), discoveries (fast bombs, frost bomb, infusion, precise bombs [4 squares]), mutagen (+4/-2, +2 natural armor, 80 minutes), poison use, rodent empathy, swift alchemy
Combat Gear elixir of truth, mutagen[APG], potion of cure serious wounds (3), potion of displacement, potion of invisibility, acid (2), antitoxin (2), smokestick (2), tanglefoot bag (2); Other Gear +1 studded leather, dagger, light crossbow with 10 bolts, cloak of resistance +1, dust of dryness, ring of protection +1, formula book[UE], tindertwig (10), 317 gp
Special Abilities
Alchemy +8 (Su) +8 to Craft (Alchemy) to create alchemical items, can Id potions by touch.
Bomb 4d6+4 (12/day, DC 18) (Su) Thrown Splash Weapon deals 4d6+4 fire damage.
Darkvision (60 feet) You can see in the dark (black and white only).
Fast Bombs (Su) Throw multiple bombs as a full-round action.
Fly (40 feet, Good) You can fly!
Frost Bomb (DC 18) (Su) Bomb deals cold damage and staggers foe next round (Fort neg).
Infusion Create an extract can be used by anyone but takes up a slot until used.
Mutagen (DC 18) (Su) Mutagen adds +4/-2 to physical/mental attributes, and +2 nat. armor for 80 minutes.
Point-Blank Shot +1 to attack and damage rolls with ranged weapons at up to 30 feet.
Poison Use You do not risk poisoning yourself accidentally while poisoning a weapon.
Precise Bombs (Su) Exclude up to 4 squares from splash damage of your bombs.
Precise Shot You don't get -4 to hit when shooting or throwing into melee.
Rodent Empathy Ratfolk gain a +4 racial bonus on Handle Animal checks made to influence rodents.
Swarming (Ex) Can share a square with another with swarming, if both att the same foe, he is flanked.
Swift Alchemy (Ex) Construct alchemical items in half the normal time.
Throw Anything Proficient with improvised ranged weapons. +1 to hit with thrown splash weapons.
Considered superior to spellcasting by many ratfolk, alchemy produces more items for sale and creates remedies that benefit the warren.
Ratfolk chemists are treasured members of ratfolk colonies. Not only do they provide the warren with excellent defensive capability, but their alchemical creations are valuable merchandise to sell or trade.

Used your latest module that was available as of this afternoon at about 2:30pm.  I used Hero Lab to produce this statblock. For greater accuracy, you might want to use one of the online resources. Hero Lab can truncate some of the text. All the other statblocks from Bestiary 3 imported -- a total of more than 86% imported!


Installation bug

Hi @WangPeng-Lab , perhaps there is an error in your pip version of scGCO. I use pip install scGCO but it throws out.

So maybe you should change from sklearn  into scikit-learn.

Thanks for your informations. I updated the scGCO.


On-disk size of data.kz is large when using multiple CREATE statements

Migrating this from our Discord channel as reported by a user. Basically, when a few thousand CREATE statements are sent via individual transactions, the on-disk size of the database directory is much larger (400 MB) compared to sending batched transactions (50 MB) via COPY. The question is about whether some sort of compaction process can be triggered to reduce the disk space usage when batched transactions are not possible.
Original message:

Is there a way to reduce the on-disk footprint of kuzu graph db? Can we trigger some sort of compaction on-demand? If I load a few thousand transactions, nodes and edges combined, my on-disk size grows to 400MB. Same transactions when batched create less than 50MB. However, batching is not practical in many cases, therefore is there a way to load transactions and then trigger compaction to reduce the disk footprint?

Questions we asked to clarify more:

"A few thousand transactions, nodes and edges combined": Is this for creation only? i.e., no deletions?
Batched meaning a few thousand creations within a single transaction?
Does most of the size difference come from data.kz file?

Clarifications:

Creation only, no deletion.
It was not a single batch... around 20 batches.
Size difference is all from data.kz file. That single file has most of the data.

Reproducible example:

Here is a test, that reproduces. Just two node types, create a 1000 instances of each. Once for batched and once  for non-batched, and print the directory sizes for each.

import kuzu
import os
import shutil
import pandas as pd
from tempfile import NamedTemporaryFile

def run_kuzu_query(query, conn):
    qR = conn.execute(query)
    return qR.get_as_df()

def init_kuzu_db(_kuzu_loc):
    if os.path.exists(_kuzu_loc):
        shutil.rmtree(_kuzu_loc)
    ddl1 = 'create node table ContainerNode(my_key STRING, name1 STRING, name2 STRING, name3 STRING[], count INT64, PRIMARY KEY (my_key))'
    ddl2 = 'create node table ContentNode(my_key STRING, name1 STRING, name2 STRING, name3 STRING[], val1 DOUBLE, val2 DOUBLE, val3 DOUBLE, x BOOLEAN, y BOOLEAN, z BOOLEAN, w STRING[], PRIMARY KEY (my_key))'
    ddl3 = 'CREATE REL TABLE ContContRel(FROM ContainerNode TO ContentNode)'
    buf_pool_gb = 1
    _kuzuDB = kuzu.Database(_kuzu_loc, buffer_pool_size=buf_pool_gb*(1024**3))
    conn = kuzu.Connection(_kuzuDB, num_threads=1)
    run_kuzu_query(ddl1, conn)
    run_kuzu_query(ddl2, conn)
    run_kuzu_query(ddl3, conn)
    return conn

def create_container_nodes_no_batch(conn):
    for i in range(1000):
        dml  = "CREATE(t:ContainerNode {{ my_key : '{0}', name1 : 'xyz', name2 : 'abc', name3 : ['pqr'], count : {1} }})".format(i, i)
        run_kuzu_query(dml, conn)

def create_content_nodes_no_batch(conn):
    for i in range(1000):
        dml  = ("CREATE(t:ContentNode {{ my_key : '{0}', name1 : 'xyz', name2 : 'abc', name3 : ['pqr'], val1 : {1}, val2: {2}, val3: 3.5, "
                "x: TRUE, y: FALSE, z: TRUE, w: ['random'] }})").format(i, i, 1.5*i)
        run_kuzu_query(dml, conn)

def create_container_nodes_batched(conn):
    df = pd.DataFrame(columns=['my_key', 'name1', 'name2', 'name3', 'count'])
    for i in range(1000):
        df.loc[len(df.index)] = [f'{i}', 'xyz', 'abc', ['pqr'], i]

    df.reset_index(drop=True, inplace=True)

    tf = NamedTemporaryFile(suffix=".parquet")
    with tf:
        df.to_parquet(tf)
        tf.flush()
        dml = f"""COPY ContainerNode FROM "{tf.name}" """
        conn.execute(dml)

def create_content_nodes_batched(conn):
    df = pd.DataFrame(columns=['my_key', 'name1', 'name2', 'name3', 'val1', 'val2', 'val3', 'x', 'y', 'z', 'w'])
    for i in range(1000):
        df.loc[len(df.index)] = [f'{i}', 'xyz', 'abc', ['pqr'], 1.0*i, 1.0*i, 3.5, True, False, True, ['random']]

    df.reset_index(drop=True, inplace=True)
    tf = NamedTemporaryFile(suffix=".parquet")
    with tf:
        df.to_parquet(tf)
        tf.flush()
        dml = f"""COPY ContentNode FROM "{tf.name}" """
        conn.execute(dml)

def get_dir_size(path):
    total = 0
    with os.scandir(path) as it:
        for entry in it:
            if entry.is_file():
                total += entry.stat().st_size
            elif entry.is_dir():
                total += get_dir_size(entry.path)
    return total

## Without batching
conn_no_batch = init_kuzu_db("/tmp/no_batch.kuzudb")
create_container_nodes_no_batch(conn_no_batch)
create_content_nodes_no_batch(conn_no_batch)

## With batching
conn_batched = init_kuzu_db("/tmp/batched.kuzudb")
create_container_nodes_batched(conn_batched)
create_content_nodes_batched(conn_batched)

print('Size without batching:\t', get_dir_size("/tmp/no_batch.kuzudb"))
print('Size with batching:\t\t', get_dir_size("/tmp/batched.kuzudb"))

By looking at the script, one hypothesis for the size difference is due to compression, a few thousand transactions might trigger re-compression of existing tuples, and right now we don't reclaim those space yet (will be added for sure later), while a single COPY statement doesn't trigger re-compression at all.
Will profile a bit more to verify if that's the case.

This is now solved with mvcc #3718


Feature Request - 2FA Notification script

Hi,
Would it be possible to get an additional command line option to notify when 2FA has expired via an external script? e.g. --notification-script /path/to/my/script/notify.sh
I currently use Prowl for iOS push notifications so don't have e-mail configured for external relay. My script would just contain:
curl https://api.prowlapp.com/publicapi/add -F apikey=$APIKEY -F application="icloud_photos_downloader" -F event="2FA Expired" -F priority="2" -F description="Two factor authentication has expired. Please renew."
This method could also be used with other push notification services such as Pushover, PushBullet, NotifyMyAndroid etc
Thanks.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Hey, that's a great idea! I like --notification-script. Would you be able to submit a PR with this change? I think you'll just need to add a @click.argument to icloudpd/base.py, and then add a notification_script argument at the bottom of def main(. Then you'll need to add:
import subprocess

# ...

subprocess.call([notification_script])

Hope that helps!

I think you've over-estimated my Python skills, which I can sum up in two words... 'hello world!' lol
I should be able to work it out, it may take a while though. I'll see what I can come up with this afternoon and report back.

Well I've got something that's working on my system...
https://github.com/boredazfcuk/icloud_photos_downloader/blob/master/icloudpd/base.py
Does that look right to you?

Awesome thanks testing that out! I merged those changes in and added a test for it. Released version 1.3.3 that includes the --notification-script option


[MINOR][DOCS][STREAMING] Minor docfix schema of csv rather than parquet in comments

What changes were proposed in this pull request?
Fix parquet to csv in a comment to match the input format being read.
How was this patch tested?
N/A (doc change only)

Test build #62565 has started for PR 14274 at commit 534c53f.

Test build #62565 has finished for PR 14274 at commit 534c53f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62565/
Test PASSed.

LGTM

Merged to master/2.0


update the screenshot funcs to match the behavior of chrome commands

Fixes #1220.

Here are the chrome commands:

It seems that  the behavior changed in Chrome 108.

The test failed because it tests with chromedp/headless-shell:latest,  which is chromedp/headless-shell:107.0.5304.107 as of now.


can use defaultdict in namecount.py

"Why you should learn Python" - nice read!  I noticed you can use a defaultdict in namecount.py:
names = {}
[...]
    if name in names:
        names[name] += 1
    else:
        names[name] = 1

This can be simplified to
from collections import defaultdict
[...]
names = defaultdict(int)
[...]
    names[name] += 1

So this would be the full namecount.py:
#!/usr/bin/env python3
import sys
from collections import defaultdict

def count_names():
    names = defaultdict(int)
    for name in sys.stdin.readlines():
        name = name.strip()
        names[name] += 1  # or combine this and the previous line

    for name, count in names.items():
        sys.stdout.write("{0}\t{1}\n".format(name, count))

if __name__ == "__main__":
    count_names()

Kind of a minor thing, but since the article is about the virtues of Python, I thought it'd be a nice addition.
(I'd fork and make a pull request, but looks like the repo has raw HTML rather than markdown or some such)

Hey, thank you for your feedback and suggestion :smile:

"Why you should learn Python" - nice read! I noticed you can use a defaultdict in

Someone has already pointed out that optimization and I actually included a defaultdict version at the end of the post (the EDIT section). Here's what it says:

As per jyf1987's suggestion,
here is a namescount.py version using collections.defaultdict: namescount.py using defaultdict.
I'm keeping the original one in the post, because the goal was to be more explicit, so that people new to Python would have less trouble understanding it.

(I'd fork and make a pull request, but looks like the repo has raw HTML rather than markdown or some such)

Oh yes, I'm using Jekyll. Basically you write your posts in Markdown (with some extensions, like syntax highlighters), then use Jekyll to compile it to HTML.
So I basically have 2 branches: source, where the Jekyll source resides and master, where I put the generated HTML.

Cool.  Missed the edit I guess.

So I basically have 2 branches: source, where the Jekyll source resides and master, where I put the generated HTML.
Different branch!  I figured it was Jekyll (does github pages support other site generators?), just couldn't find the source.

Different branch! I figured it was Jekyll (does github pages support other site generators?), just couldn't find the source.

I'm pretty sure you can use pretty much what you want, as long as it's HTML/JS/CSS.

Maybe I wasn't super clear, but GitHub doesn't generate the HTML, you have to do it manually (or use something like Travis CI integration to automate it)


Additional types for the AvatarSearchQuery struct

The minimum/maximum price and limit were visible on the API documentation but not implemented for some reason.
There are some hidden parameters like SalesTypeFilter. It's possible there are more but I'm not aware of them.

Looks great! The code is clean and follows conventions, and the documentation is written well also.
I may move the new Limit a bit since we already have a universal limit. I'm probably going to change the name to something like CatalogLimit. I'm then probably going to add a list of alternative limits to Limit's documentation. Also going to change the u32s to u64s just because it's what's used everywhere else.
But yeah, thanks for your contribution!

Just clarifying that the reason u32 used was because any values above would make the API return cached results.

Just clarifying that the reason u32 used was because any values above would make the API return cached results.

I'm a bit confused here, what do you mean by returning cached results? Does that mean that each page requested with the exact same parameters (ignoring cursor) is going to be the same? Is there a good example that I could run that shows this in action?

I'm not sure how it would work in a browser less context since I tested the parameters in the browser. However, what I did was queried the API with a parameter of minPrice as something normal like 1000. Then, I tried a very large number like<PHONE_NUMBER>. It showed the same results as if I inputted 1000 instead. I'm not sure what it would say if you didn't query the API before hand.

I'm not sure how it would work in a browser less context since I tested the parameters in the browser. However, what I did was queried the API with a parameter of minPrice as something normal like 1000. Then, I tried a very large number like<PHONE_NUMBER>. It showed the same results as if I inputted 1000 instead. I'm not sure what it would say if you didn't query the API before hand.

Alright ty for the heads up. I'll take a look at it when I'm back on pc and have the time

Here's a couple links to test. It might've just been my browser doing it since testing on Safari leads to an error.
https://catalog.roblox.com/v1/search/items/details?minPrice=2147000000
https://catalog.roblox.com/v1/search/items/details?minPrice=214700000
https://catalog.roblox.com/v1/search/items/details?minPrice=10

Alright, so I did a bit of testing. So the max u32 value is<PHONE_NUMBER>. The endpoint does appear to break at high amounts but given that it can go above<PHONE_NUMBER> slightly, it implies that the limit is probably something arbitrary and not the max u32 value.
However, I think there's a small but also decent chance that someone is going to use u64::MAX for the min/max price and not understand why it's not working, and it's unlikely they'll really need a number larger than<PHONE_NUMBER>, so I think I'm just going to change it back to u32 in the next update.


Homebrew and /usr/local

It has always bugged me that Homebrew installs itself into /usr/local and claims ownership over it instead of root:wheel. For the longest time, Homebrew’s git repository was installed directly in that path (nowadays in /usr/local/Homebrew). Nevertheless, Homebrew still insists on changing the ownership of /usr/local/bin and /usr/local/lib, which means that the installing user can install any binaries or libraries without additional privileges.
==> The following existing directories will be made group writable:
/usr/local/bin
/usr/local/include
/usr/local/lib
/usr/local/lib/pkgconfig
==> The following existing directories will have their owner set to my_admin_account:
/usr/local/bin
/usr/local/include
/usr/local/lib
/usr/local/lib/pkgconfig
==> The following existing directories will have their group set to admin:
/usr/local/bin
/usr/local/include
/usr/local/lib
/usr/local/lib/pkgconfig

Since /usr/local/bin takes precedence over /usr/bin and /bin (in /etc/paths), any malicious program could simply put binaries or libraries there and effectively ‘overwrite’ the ones installed in other system paths.
$ cat /etc/paths
/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin

I am of the opinion that this is a bad practice and it should be mentioned explicitly. I personally have installed Homebrew in ~/.homebrew, despite their advice against it. I have not come across any problems so far.

How do you install binary files that are already provided by MacOS (e.g. git or ruby) if homebrew does not take precedence over /usr/bin and /bin?

@hidi: export PATH=“/usr/local/bin:$PATH”

You recommend giving homebrew priority over /usr/bin and /bin. But that's exactly what this guide is supposed to prevent. So this cannot be a solution.


Remove color setting help text

There are some leftover text in the theme settings from the days of settings.html. This only confuses people since theme settings are coded differently now.

@cshold

:+1: thanks


Refactoring in pointer down event handler

This handler is now at 1000 LOC. I tried to simplify it by pulling things out. I'll follow up with further refactor later.
Reviewing each commit should be easy.

I fixed the tests, the last "else" late at night broke everything (stateful code... 🤮 )

Hi, thanks for your work. I've been trying to reduce the lines of App.tsx.
I haven't looked your work in deep, but do you think you could refactor so that the separated logic can be out of App.tsx?

@dai-shi I can move the code out of App.tsx in a follow up PR, yes.

Looking at the code, moving this out of App is not gonna be easy. We will probably have to refactor App to use hooks, then split the state, then have a hook that handles all the pointer interactions. As is the code is too tightly coupled to the component state.
But also the way the instance state is used means that this is gonna be pretty involved to correctly keep track of dependencies for the effect hooks.
An alternative is a mixin (old school) or moving everything over to Recoil to make it easier to share the state.
Folks with a better understanding of App might have a better shot at this than me.


Improve ability to change/add File icons without having to subclass

Right now if you want custom icons for files you have to subclass filer.models.filemodles.File. There is a lot of bloat if you need to just change the icons for many file types.
I've written my own subclass where I can support a bunch of different file types and get the right icons for them. See below...
class CustomFile(File):

    # The dict key is icon type and value is list of file extensions that match
    # Any new file types need icons put in the static in 32x32 and 48x48 sizes
    FILER_FILE_ICONS = getattr(settings, 'FILER_FILE_ICONS', {'file': {['*', ], })

    @property
    def _icon(self):
        for t in self.FILER_FILE_ICONS:
            if self.extension in t:
                return t
        return 'file'  # Default to file if we don't know the type

    @classmethod
    def matches_file_type(cls, iname, ifile, request):
        # the extensions we'll recognise for this file type
        filename_extensions = []
        for t in cls.FILER_FILE_ICONS:
            filename_extensions += cls.FILER_FILE_ICONS[t]
        ext = os.path.splitext(iname)[1].lower()
        return ext in filename_extensions

In settings.py:
FILER_FILE_ICONS = {
    'pdf': ['.pdf'],
    'doc': ['.doc', '.docx', '.dotx', ],
}

There is very few changes to do to support easier icon support without having to subclass just to support custom icons. Would you accept a PR for this?

Definitely!


collect policy event for the grafana datashboard

Signed-off-by: myan<EMAIL_ADDRESS>Refs: https://issues.redhat.com/browse/ACM-4099
TODO: consider the batch update for a better performance.
The collected data is like this:
$ table=event.local_policies
$ kubectl exec -it $(kubectl get pods -n hoh-postgres -l postgres-operator.crunchydata.com/role=master -o jsonpath='{.items..metadata.name}') -c database -n hoh-postgres -- psql -U postgres -d hoh -c "select * from $table"
              policy_id               |              cluster_id              |                                               message                                               |      reason      |
     source                |     created_at      |  compliance
--------------------------------------+--------------------------------------+-----------------------------------------------------------------------------------------------------+------------------+----------
---------------------------+---------------------+---------------
 1d858095-4689-4f47-869b-3d4761f9825b | b5fbe3fb-3059-4d28-badb-dbd9c7991e74 | Policy local-placement.policy-limitrange status was updated in cluster namespace kind-hub1-cluster1 | PolicyStatusSync | {"compone
nt": "policy-status-sync"} | 2023-05-23 12:06:24 | non_compliant
 1d858095-4689-4f47-869b-3d4761f9825b | b5fbe3fb-3059-4d28-badb-dbd9c7991e74 | Policy local-placement.policy-limitrange status was updated in cluster namespace kind-hub1-cluster1 | PolicyStatusSync | {"compone
nt": "policy-status-sync"} | 2023-05-23 12:06:17 | non_compliant
(2 rows)

/retest

Any special reason to select new kafka go lib https://github.com/Shopify/sarama? We use github.com/confluentinc/confluent-kafka-go for spec and status syncs.

Good question! The following are the reasons that I choose to use https://github.com/Shopify/sarama

The implementation for cloudevents kafka protocal is https://github.com/Shopify/sarama. Considering that we may fully use cloudevents as the global hub transport in the future, we choose to be consistent with its implementation here.
The Shopify/sarama has a larger user base and community support, with a history of being widely adopted.

It does not say that we have to choose sarama. Confluent-kafka-go is also a good choice, but based on the current situation, the two will coexist for a while

/retest


Index Memberof as Collections

Closes #133

Coverage increased (+0.03%) to 98.793% when pulling 61dbcfe6deef712d53c8b103a5fe3120bf77f41b on index_plum_titles into f5aaad8574f09316f005bd32d70a764a702546bf on master.


Fix contributor agreement link

GitHub is expanding the file name to https://github.com/elm/compiler/blob/master/.github/ContributorAgreement.pdf, which is a 404, so use the full URL instead.

This can be closed as the issue was first fixed in #2138 and then in efb6711757d40acee612f797c02cafca020cc073 the CLA process was thankfully abandoned altogether.


Count of Shipment Rates for Fedex test account is 0!

I just tried it with your address information and it worked fine. Currently, it looks like FedEx dev environment is having issues getting any other rate than Ground. Since I don't have your FedEx account information, I cannot tell if it is an issue with your account. If you set a break point in the FedExProvider.cs file at line 136 (I think), you can check any replies coming back. Here is a screenshot:

You can also download the files from FedEx and use it to make sure it is working.
https://www.fedex.com/us/developer/web-services/process.html?tab=tab2

We experienced this same bug, and we found that Fedex integration wasn't returning rate information because Fedex class provider was configured to use production URL as default.

closed by #51


feat: add TypeScript Support

Follows #180
Notes
I also moved from package-lock.json to yarn.lock, as it's what we're using in all other community modules.

@kevinmarrec thanks! I am not using TypeScript in my projects, is there anything else missing or is it save to be merged?

@dohomi It's safe to be merged :)

Ok :-)


fix flavor after moving to new version of cn-infra

Coverage increased (+0.3%) to 79.248% when pulling 256a9df80e3efd23b326610b21ba91924554d13a on rastislavszabo:master into 2deb2a759d0375e5f4cc4388cc571f3c3f75ce5f on contiv:master.


Coding guidlines

There should be a document about coding guidelines/best practices for tellurium (can be part of the release checklist wiki document).
Some short basic things like

if you write a function/class/module it has to be documented (one sentence + description of input and output arguments)
all variable names are camelCase
code should adhere to PEP8
how to handle logging, exceptions, warnings ? (no idea what I should do at this point to be consistent with how tellurium is handeling things)
...
Currently, things are very divers in the code base.

I would like to remove author names & create date from headers (This is unnecessary information and discourages changes, best described in
https://youtu.be/ZSFDm3UYkeE?t=864 (around 17.10)
Very interesting video and worth a watch for everybody contributing to open source projects.
Can you give me a okay if I can remove names and dates?
M

Some things I notices:

all classes should be new style classes i.e. be subclass of object

class SomeClass(object):
    pass

do imports on top of the module not within functions. The function imports can be packaged even if the import requirements are not full-filled. But will fail on run-time
run the tests after changes! And especially run the tests after packaging Tellurium. In addition do some minimal testing of the packaged version and before shipping. This will find things like #79. A simple

 import tellurium as te

would have revealed the problem.

write tests for things you implement. This will make your life so much easier in the long run.
A first test suite is implemented which can be run via

import tellurium.tests.test_runner as tetest
runner = tetest.TestRunner()
runner.te_passes_tests()

I added a document for coding guidelines to the Wiki
https://github.com/sys-bio/tellurium/wiki/Coding-Guidlines
Important points should be stated there.


KeyError: 'items' when trying to like timeline

Please follow the guide below

Issues submitted without this template format will be ignored.
You will be asked some questions and requested to provide some information, please read them carefully and answer completely.
Put an x into all the boxes [ ] relevant to your issue (like so [x]).
Use the Preview tab to see how your issue will actually look like.

Before submitting an issue, make sure you have:

[x] Updated to the latest version
[x] Read the README and docs
[x] Searched the bugtracker for similar issues including closed ones

Purpose of your issue?

[x] Bug report (encountered problems/errors)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections requests more details for particular types of issues, you can remove any section (the contents between the triple ---) not applicable to your issue.

For a bug report, you must include the Python version used, code that will reproduce the error, and the error log/traceback.
Paste the output of python -V here: Python 2.7.10
Code:
def get_timeline_medias(self, filtration=True):
    if not self.api.get_timeline_feed():
        self.logger.warning("Error while getting timeline feed.")
        return []
    return self.filter_medias(self.api.last_json["items"], filtration)

Error/Debug Log:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "ultimate.py", line 44, in like_timeline
    bot.like_timeline(amount=300 // 24)
  File "build/bdist.macosx-10.13-intel/egg/instabot/bot/bot.py", line 422, in like_timeline
    return like_timeline(self, amount)
  File "build/bdist.macosx-10.13-intel/egg/instabot/bot/bot_like.py", line 61, in like_timeline
    medias = self.get_timeline_medias()[:amount]
  File "build/bdist.macosx-10.13-intel/egg/instabot/bot/bot.py", line 309, in get_timeline_medias
    return get_timeline_medias(self)
  File "build/bdist.macosx-10.13-intel/egg/instabot/bot/bot_get.py", line 46, in get_timeline_medias
    return self.filter_medias(self.api.last_json["items"], filtration)
KeyError: 'items'

Describe your issue
Since the beginning of today I have been getting this error. I did not add any code. I just simply ... $python setup.py build... Then $Python setup.py install ... In the code root dir because my instabot was not taking into account the updated values for max_followers_to_follow=10000 and filter_business_accounts=False when I was running the ultimate schedule script. I recloned the repo and installed via pip but it still producers this error.
Tried another fresh install and even running the multi_script_CLI.py it crashes because of KeyError: 'items' when trying to like timeline. It appears to like from hashtag but is super buggy. See screenshot.

I had the same error, #689 solves my problem

I had the same error, #689 solves my problem

Okay, so I changed that one line 46 to
   return self.filter_medias(self.api.last_json["feed_items"], filtration)
and restarted the bot but it still doesn't work. Is there an extra step I am missing?

Is there any error message?

Yes, the KeyError: 'items' .
I think there may be something wrong with my environment though since I am getting these other strange behaviours such as this unnecessary filtering of medias for instance (see the image in the original post). I don't know how to reset it other than deleting the files and re-downloaded the repo but that didn't change anything.

Fixed #689 . Thanks to @aferook


Update bundled JDK to 21.0.1+12

There is no Windows release yet, check: https://adoptium.net/temurin/releases/?os=windows&arch=x64 for 21.0.1+12 to become available.

Windows build is now available


JSON Formatter is line-based

The JSON Formatter is generating output that looks like this:
{"prog":"relaton","message":"cache /Users/nickn/.relaton/cache/iso/: version is obsolete and cache is cleared.","severity":"INFO"}
{"prog":"relaton","message":"cache /Users/nickn/.relaton/cache/jis/: version is obsolete and cache is cleared.","severity":"INFO"}
{"prog":"relaton","message":"cache /Users/nickn/.relaton/cache/ogc/: version is obsolete and cache is cleared.","severity":"INFO"}
{"prog":"relaton-iso","message":"Is not recognized as a standards identifier.","severity":"INFO","key":"ISO 3696aaaa"}

But if I'm to machine-read this, I have to assume one line per entry, and I don't think I can or should assume that. The JSON output should be a list of objects, in order to guarantee correct processing:
[
{"prog":"relaton","message":"cache /Users/nickn/.relaton/cache/iso/: version is obsolete and cache is cleared.","severity":"INFO"},
{"prog":"relaton","message":"cache /Users/nickn/.relaton/cache/jis/: version is obsolete and cache is cleared.","severity":"INFO"},
{"prog":"relaton","message":"cache /Users/nickn/.relaton/cache/ogc/: version is obsolete and cache is cleared.","severity":"INFO"},
{"prog":"relaton-iso","message":"Is not recognized as a standards identifier.","severity":"INFO","key":"ISO 3696aaaa"}
]

If you do action this, let me know, so I can update the parsing of this in metanorma-standoc

@opoudjis this because the logger continuously writing messages to an IO object, so it doesn't know when it should close square brackets. You need to read messages line by line:
# SETUP

# it's possible use file name or IO object
io = StringIO.new

# create logger and add it to the pool
logger = Relaton::Logger::Log.new(io, formatter: Relaton::Logger::FormatterJSON)
Relaton.logger_pool[:string_io] = logger

# WRITE MESSAGES

# Relaton gems writes messages to all loggers in the pool
# you can write messages to the loggers with
Relaton.logger_pool.warn "message1", "prog name", my_key: "key1"
Relaton.logger_pool.warn "message2", "prog name", my_key: "key2"

# READ MESSAGES

# split the string in the IO by new lines and parse each line
io.string.split("\n").map { |m| JSON.parse m }
 => [{"prog"=>"prog name", "message"=>"message1", "severity"=>"WARN", "my_key"=>"key1"}, {"prog"=>"prog name", "message"=>"message2", "severity"=>"WARN", "my_key"=>"key2"}]

# CLEAN LOGGERS

# you can remove all messages from all IO objects, so next time you will read only new messages
Relaton.logger_pool.trucate

You can guarantee that each message is a line long?
Ok, closing the ticket.

You can guarantee that each message is a line long?

@opoudjis it's possible to create a message with new lines, so let's use regex:
io.string.split(/(?<=})\n(?={)/).map { |m| JSON.parse m }
 => [{"prog"=>"prog name", "message"=>"message1", "severity"=>"WARN", "my_key"=>"key1"}, {"prog"=>"prog name", "message"=>"message2", "severity"=>"WARN", "my_key"=>"key2"}]


[Q] Is it possible to get rid of unnecessary Maybe in generated types?

Hi! It's really good to see that we have a library for building Telegram clients in Haskell.
I have the question: is it possible to get rid of unnecessary Maybe in generated types?
For example, every field in BotInfo is Maybe. As I know, it's safe to remove Maybes from fields whose description doesn't contain may be null.

Also BotInfo.description doesn't have a documentation string. That's because in tl file it's annotated as @param_description

Thank you, and sorry for the delayed response.
I tried to do this at first, but I don’t think we can fully rely on handwritten documentation. It may contain vague descriptions like 'pass null if'. Wrongly added or removed clarifications may cause runtime errors.

Also BotInfo.description doesn't have a documentation string. That's because in tl file it's annotated as @param_description

Thanks for pointing this out!


SparseCategoricalFocalLoss with class_weight ?

Hm..maybe this is weird question.
But can be used  SparseCategoricalFocalLoss with defining class_weight (for example in model.fit fn) together? Or its bad idea?
Thanks for answer.   :)

I think it could be a good idea to try if you have a good reason to weight your classes, but I haven't tried it myself.


guildCreate doesn't get triggered on init

Currently there's no way to get guilds on launch with ChannelManager and GuildManager being LimitedCollection with size = 0 without fetching them with client.guilds.forge(exampleGuildId).channels.fetch() (ChannelManager) and client.guilds.forge(exampleGuildId).fetch() (GuildManager) as example.
client.on("raw",console.log) shows, that we receive GUILD_CREATE events (with channels and threads) without getting them on guildCreate or guildUpdate (shardConnect works for this option, but will do it only on sharding).
I was collecting parentId's of guild channels and thread parentIds, so there's no way to get them at launch without fetching.
Code:
...

const categories = new Collection<Snowflake, Snowflake>() // { channel => category }
const threads = new Collection<Snowflake, Snowflake>() // { thread => channel }

// categories and threads are always synced:
client.on("channelCreate",this._onChannelCreate);
client.on("channelUpdate",this._onChannelUpdate);
client.on("channelDelete",this._onChannelDelete);
client.on("threadCreate",this._onThreadCreate);
client.on("threadDelete",this._onThreadDelete);
client.on("threadUpdate",this._onThreadUpdate);
client.on("threadListSync",this._onThreadSync);

const channels = await this.client.guilds.forge(config().constants.trustedGuild).channels.fetch(); // can't receive on guildUpdate or guildCreate
// same with threads

channels.forEach(channel => {
   if (!channel.parentId || !channel.isText()) return;
   this.categories.set(channel.id, channel.parentId)
})

...

messageCreate: (leveling.channelMultipliers contains { Snowflake: number } with channel/category ids and multiplier values)
const channelId = message.channel.isThread() ? this.threads.get(message.channelId) : message.channelId;
const categoryId = channelId ? this.categories.get(channelId) : null;

const channelMultiplier = leveling.channelMultipliers[channelId!] ?? leveling.channelMultipliers[categoryId!] ?? 0;

console.log(`${channelId}'s multiplier is ${channelMultiplier}`);

guildCreate and guildUpdate should work correctly after the client is ready. The initial GUILD_CREATE events that are sent to fill up the cache at login are not emitted by the discord.js guildCreate event, they are handled internally to decide when the client should become ready.
If you're looking for the initial GUILD_CREATE events, you have to use shardConnect or raw.
Is this what you're asking? im not sure im understanding the issue very well.

@timotejroiko Yeah, I thought of maybe implementing a new before-ready guild create event, because with shardConnect I still won't get thread objects, only guilds. So raw is only way to go with there :(

thank you!


ProducerAsync return task which never completes

Quick fix for https://github.com/confluentinc/confluent-kafka-dotnet/issues/406.
Several points:

Decided to return NoError instance for error instead of null, because this property is always filled
Not sure about Timestamp, so used NotAvailable type

thanks @andreycha - is this the problem reported in #545 as well? I was aware of the issue that you've resolved in this PR, what I'm trying to work out is if there is something else going on as well.

@mhowlett no, these are different scenarios. In #545 delivery is enabled. Here we have rather logical mistake, and in #545 I believe it's something related to sync context, because it only hangs in unit tests.

I am also facing this issue when running my code in EKS some times ProducerAsync return task which never completes
but same code running fine in Windows environment. below is my Producer config.
var config = new ProducerConfig
{
//   GroupId = "" + ConfigurationManager.AppSettings["GroupId"],// "ARIFILES",
BootstrapServers = "" + ConfigurationManager.AppSettings["bootstrapserver"],
QueueBufferingMaxMessages = 300,
BatchNumMessages = 10,
CompressionLevel = 2,
Acks = Acks.All,
MessageTimeoutMs = 5000
};


Adding GuidConverter which will handle binding of Guid/Guid? type parameters

Fixes #521
Adding a new IConverter implementation to handle binding of Guid or Guid? type parameters.
Sample usage for binding to Guid? parameter type.
[Function("GetRecords")]
public static HttpResponseData GetRecords(
    [HttpTrigger(AuthorizationLevel.Anonymous, "get", "post",Route = "records/{clientId}")] HttpRequestData req,
    Guid? clientId,
    FunctionContext executionContext)
{
   // do something with clientId value.

    var response = req.CreateResponse(HttpStatusCode.OK);
    response.WriteString($"ClientId nullable:{clientId}");

    return response;
}

Out of curiosity, would it make sense to make IConverter public so that we can provide our own implementations for custom types?

@kzryzstof yes, and that is the plan. This just needs to go through a round of API reviews as no changes can be made once this becomes public, but that is the intention (same with a number of other APIs and extensibility points currently internal)

@fabiocav Great! Thanks :)


feature request: add MPD user service

MPD (music player daemon) is probably a service that lots of people (including me) would like to run for specific users. In your blog you describe that managing user services is a big priority for home-manager, so I wanted to ask if you would be interested in implementing it (and maybe also modipy, if that's possible).

@throwup - this module should work if you set services.mpd.dataDir to where your mpd data is.
@rycee - What is your opinion on importing the <nixpkgs> set and reusing the options provided in it? It could save on rewriting/copy+pasting options and keeping better compatibility with upstream.

@ayyjayess Unfortunately it is not entirely safe to import the options from <nixpkgs> although I entirely understand the motivation. The reason is that, if the module options change in Nixpkgs then the evaluation of the Home Manager modules may break.
That said, we have made a few exceptions for some modules that are imported as-is into Home Manager. These modules are very central to the NixOS module system so I imagine them to be very stable. I'm not as certain about the mpd module, though, so I would be a bit reluctant about doing the same for the mpd options. Sadly.

@ayyjayess Unfortunately the import <nixpkgs/nixos> {} in your gist won't work for everybody, because it could fail trying to find a configuration.nix (for me because I'm using nixops, and probably for Darwin people too), so I think the options need to be duplicated unfortunately

I created a pull request here: #311
It probably needs work but it gets it going

I've merged the #311 module and am therefore closing this. Thanks @rembo10 for creating the module and @throwup for opening the issue! 🙂


Change the shutdown procedure to deregister fabio from the registry and then shutdown the proxy

The current shutdown procedure shuts down all proxies and then de-registers Fabio from the service registry. This may cause some requests to fail while a Fabio instance is shutting down. This PR reverses the order and introduces a de-register grace period, causing Fabio to wait for a configurable period after de-registering from a registry and then shutting down the proxies. This way no new requests will be sent to the stopping instances, while giving time to running requests to finish.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

I like the idea of changing the shutdown procedure order.  As for the configurable grace period, there is already a proxy.shutdownwait .
https://fabiolb.net/ref/proxy.shutdownwait/
Would that solve your issue?

It doesn't serve the same purpose. The shutdownwait gives time to currently in-flight requests/connections finish, while new ones are declined. In the best case proxy.shutdownwait and the grace period will work together for pretty much 0 dropped requests. The other option is to reuse the shutdownwait config.

So after digging through the code, it looks like tcp, grpc, and https+tcp+sni proxies will all wait until the context expires before returning.  However, the http proxy does not, it will wait at most up to the context deadline but if all listeners finish with connections before that, it returns early.  So as much as I hate the inconsistency here in fabio, I don't think it's worth it to change existing behavior.
So if you could add some documentation to

the fabio.properties file in the comments, following the other examples in there
add a markdown document with similar documentation inside of docs/content/ref

I will get this merged.  Thank you!

Done!

Thank you for your contribution!


It's even better

Hello Michel!
I've looked at this project and found it even more clever than your map one.
Such a beautiful thing!
It's a pure UNIX philosophy I guess, so minimalistic and so functional programs.
Thanks for that!

Thanks @alex-bender! I'm very glad you like it :-)


Add Abstrackr and Colandr to table

I will add 2 issues for Colandr feature extractor & classifier

Many thanks for adding the information!


[WIP] Sources integration

@patilsuraj767 - Currently, we are currently working with cost management team related to OpenShift integration so we will figure out whether we need ROS-specific Sources or can we reuse Cost Management Sources?
Since RHIN-293 moved in backlog last week, can we close this PR?
Reopen it if needed in future?

@kgaikwad Yes we can close this PR for now.


Use (chan struct{}) instead of (chan bool)

... when the message content doesn't matter, but only the event.

Nice, didn't know that.


fix(ci): stop notifing about failing merge queue builds

Backport failed for releases/v0.3, because it was unable to cherry-pick the commit(s).
Please cherry-pick the changes locally and resolve any conflicts.
git fetch origin releases/v0.3
git worktree add -d .worktree/backport-4469-to-releases/v0.3 origin/releases/v0.3
cd .worktree/backport-4469-to-releases/v0.3
git switch --create backport-4469-to-releases/v0.3
git cherry-pick -x 4bf420982a9b5b446b587db826e156581755e535

Seems already backported.


Feature Request / Question - Mocking with proxies

This fails because the connection is opened to a proxy URL instead of the specified in the entry
from http import HTTPStatus

import pytest
import requests
from mocket import Mocketizer
from mocket.exceptions import StrictMocketException
from mocket.mockhttp import Entry

# setup session
session = requests.Session()
session.proxies = {
    "http": "http://proxy.url/",
    "https": "https://proxy.url/",
}

# mock request to target url
Entry.single_register(
    Entry.GET,
    "http://target.url/",
    status=HTTPStatus.OK,
)
with Mocketizer(strict_mode=True):
    res = session.get("http://target.url/")

# StrictMocketException: Mocket tried to use the real `socket` module.

We can mock the proxy URL instead

Entry.single_register(
    Entry.GET,
    "http://proxy.url/",  # we mock to proxy url
    status=HTTPStatus.OK,
)
with Mocketizer(strict_mode=True):
    res = session.get("http://target.url/")

assert res.url == "http://target.url/"

# Success!

However, then it also passes with other URLs

Entry.single_register(
    Entry.GET,
    "http://proxy.url/",  # we mock to proxy url
    status=HTTPStatus.OK,
)
with Mocketizer(strict_mode=True):
    res = session.get("http://not-the-target.url/")  # different url

assert res.url == "http://target.url/"  # fails

# AssertionError:

I'd prefer to do something like this
Entry.single_register(
    Entry.GET,
    "http://target.url/",
    status=HTTPStatus.OK,
    proxies={"http": "http://proxy.url/", "https": "https://proxy.url/"},
)
with Mocketizer(strict_mode=True):
    res = session.get("http://target.url/")

with Mocketizer(strict_mode=True):
    with pytest.raises(StrictMocketException):
        res = session.get("http://not-the-target.url/")  # not the right URL

with Mocketizer(strict_mode=True):
    with pytest.raises(StrictMocketException):
        res = requests.get("http://target.url/")  # not through the proxy

Hi @janrito, I did not get the step below:
However, then it also passes with other URLs
I assume you said Mocket did not fail, but then what about the assert?
Why asserting against the previous step's URL?

This said, I am not sure Mocket should know about proxies, because at socket level you are opening a connection against the proxy, and that's what you want to mock.

Example:
My code uses proxy P to call websites W1 and W2.
I want to test that I am calling W1, but I am telling Mocket to forget about that because there is a proxy in the middle. So, after a refactor, I may have a piece of code that calls W2 instead.
My recommendation, since HTTP[S] proxies should work transparently for you - you simply assume the proxy works properly - and not relevant when testing, I'd say you should test forgetting about proxies.

Hi @janrito, I am sorry I did not get the step below: However, then it also passes with other URLs
I assume you said Mocket did not fail, but then what about the assert? Why asserting against the previous step's URL?

it's meant to symbolise a typo. In the context of a test, it shoudl fail if I don't request the correct URL.
...

This said, I am not sure Mocket should know about proxies, and that's because at socket level you are opening a connection against the proxy, no matter what URL you are telling the proxy to hit for you.

I get what you mean: mocket just cares about the connection and when using proxies, i'm always opening a connection to the proxy URL and not the final destination.

I get what you mean: mocket just cares about the connection and when using proxies, i'm always opening a connection to the proxy URL and not the final destination.
The point is that Mocket knows nothing about the destination URL, it only knows about the actual connection.


Projection support for embedded models

This is rebased and updated version of the #65 PR.
Couple of notes:

Had to refactor how child model datas are created
There is one missing test case for handling deletions because of updates.

Address some of the comments and left replies to the rest.


Win rates and lower_is_better improvements

Added average win rates for model-v-scenario tables and improved how we handle lower_is_better in the process.
Specifically:

lower_is_better can now be None which means that we should not be comparing models across this metric.
Created a HeaderCell dataclass that contains some additional metadata, including lower_is_better to make Tables a bit more standalone (will use these to keep the plotting code separate from any involved summarize.py logic)
Implemented a compute_row_win_rates that takes a table and returns the win rates for each row, computed based on all the columns where lower_is_better is not None.
Added a add_win_rate parameter to create_group_table which we set true for the model-v-scenario tables.
Populated the schema with lower_is_better: True (this was the default before, but now the default is None). (@rishibommasani a quick skim would be appreciated)

Fixed comments 🔥
still waiting for tests but feel free to take a look @percyliang @rishibommasani


refactor: except external directories as workdir as well

This change allows us to give workdir directories like git repositories to runtime

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 13.529%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
23

Relevant Lines:
170

💛  - Coveralls


Play video only when element is fully visible in the ui

Currently video is played even when video is partially viewable.
I want to only play video when video is fully viewable.
Is there any callback function I can use for this?
import React from 'react';
import { Waypoint } from 'react-waypoint';

class  AutomaticVideoPlayer extends React.Component {
  constructor(props) {
    super(props);
    this.videoElementRef = React.createRef();
  }

  handleEnterViewport() {
    const videoRef = this.videoElementRef;
    console.log(videoRef);
    videoRef && videoRef.play();
  };

  handleExitViewport() {
    const videoRef = this.videoElementRef;

    videoRef && videoRef.pause();
  };

  render() {
    return (
      <Waypoint onEnter={this.handleEnterViewport} onLeave={this.handleExitViewport}>
        <video ref={this.videoElementRef} playsInline autoPlay loop muted {...this.props} />
      </Waypoint>
    );
  }
}

export default AutomaticVideoPlayer;

Hi @svankireddy-cartera! I would probably look into using IntersectionObserver for your use-case. Since you're using React, you could look at https://www.npmjs.com/package/react-intersection-observer

@trotzig  thank you for suggestion


Bug when matching single payload 'match'

Hiya, nice engine, I'm about to start using it and I love the simplicity. Hey, I found a one strange thing in rule behaviour - I can't get a single match clause on payload value to work, here's what works:
payload:
{
  test-field: test,
  test-field2: test2
}

rule:
  trigger-rule:
  and:
    - match:
        type: value
        value: test
        parameter:
          source: payload
          name: test-field
    - match:
        type: value
        value: test2
        parameter:
          source: payload
          name: test-field2

here's what doesn't work:
payload:
{
  test-field: test,
  test-field2: test2
}

rule
  trigger-rule:
    match:
        type: value
        value: test
        parameter:
          source: payload
          name: test-field

rules get parsed and accepted ok for me, and logical and with 2 of them are ok, but a single match doesn't seem to work - see if you had any idea what's that about pls. greetings

Hmm.  I think your 2-test YAML is probably invalid.  The and stanza should be indented under trigger-rule.  With the way you're written it, there is an empty trigger-rule, so it will probably execute regardless of the payload.  See the YAML configs for our test suite here.


Create Announcement with all fields inputted but name and description are nil

I've just tried the app for the first time and looking for issues to contribute.
When I first started the app, because there's no seed data of Announcement so I tried to create a record.

When checking in the rails console, I saw the name and description are nil now, but the publish_from and publish_until are correct.

Expected

Create Announcement successfully with all input fields

Hello! I believe these values are stored in the string_translations table
bc of the mobility gem. When you go to the home page, do you see the
announcement displayed?
On Sat, Apr 3, 2021, 4:24 PM Hai Tran @.***> wrote:

I've just tried the app for the first time and looking for issues to
contribute.
When I first started the app, because there's no seed data of Announcement
so I tried to create a record.
[image: Screen Shot 2021-04-04 at 3 16 57 AM]
https://user-images.githubusercontent.com/42526152/113490488-758c6b80-94f4-11eb-9d91-069403debe9e.png
When checking in the rails console, I saw the name and description are
nil now, but the publish_from and publish_until are correct.
[image: Screen Shot 2021-04-04 at 3 17 18 AM]
https://user-images.githubusercontent.com/42526152/113490530-da47c600-94f4-11eb-975d-1bc3bb194b6e.png
Expected

Create Announcement successfully with all input fields

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/rubyforgood/mutual-aid/issues/924, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB2BMBMDAUPLN5G65D6PNADTG52QLANCNFSM42KTBK6A
.

@maebeale I do after approving the announcement. I've never used the Mobility gem before so it's a little weird when looking at the console like this and can still retrieve the attribute 😅
I confirmed that it's completely normal. Thank you for pointing it out. :bow:

If you add .i18n, it'll pull the "translations"


Angular 15 doesn't work

Summary
Angular 15 changed the way karma works. Integrating it even more in angular cli. See https://levelup.gitconnected.com/angular-15-new-must-know-features-ae392a2baf2d#ae16
Stryker config

{
  "$schema"<EMAIL_ADDRESS>  "_comment": "This config was generated using 'stryker init'. Please see the guide for more information: https://stryker-mutator.io/docs/stryker-js/guides/angular",
  "mutate": [
    "src/**/*.ts",
    "!src/**/*.spec.ts",
    "!src/test.ts",
    "!src/environments/*.ts"
  ],
  "testRunner": "karma",
  "karma": {
    "configFile": "karma.conf.js",
    "projectType": "angular-cli",
    "config": {
      "browsers": [
        "ChromeHeadless"
      ]
    }
  },
  "reporters": [
    "progress",
    "clear-text",
    "html"
  ],
  "concurrency": 8,
  "concurrency_comment": "Recommended to use about half of your available cores when running stryker with angular",
  "coverageAnalysis": "perTest"
}

The current workaround is to create a karma.conf.js file yourself in the root of your project:
// karma.conf.js
module.exports = function (config) {
  config.set({
    basePath: '',
    frameworks: ['jasmine', '@angular-devkit/build-angular'],
    plugins: [
      require('karma-jasmine'),
      require('karma-chrome-launcher'),
      require('karma-jasmine-html-reporter'),
      require('karma-coverage'),
      require('@angular-devkit/build-angular/plugins/karma')
    ],
    client: {
      jasmine: {
        // you can add configuration options for Jasmine here
        // the possible options are listed at https://jasmine.github.io/api/edge/Configuration.html
        // for example, you can disable the random execution with `random: false`
        // or set a specific seed with `seed: 4321`
      },
      clearContext: false // leave Jasmine Spec Runner output visible in browser
    },
    jasmineHtmlReporter: {
      suppressAll: true // removes the duplicated traces
    },
    coverageReporter: {
      dir: require('path').join(__dirname, './coverage/date-picker'),
      subdir: '.',
      reporters: [
        { type: 'html' },
        { type: 'text-summary' }
      ]
    },
    reporters: ['progress', 'kjhtml'],
    port: 9876,
    colors: true,
    logLevel: config.LOG_INFO,
    autoWatch: true,
    browsers: ['Chrome'],
    singleRun: false,
    restartOnFileChange: true
  });
};

Fixed with #4054


Ensure sig handler routine returns on shutdown, turn it off in most tests

I noticed that when running the test suite, there would be a file
server/log1.txt left. This file is created by one of the config
reload test. Running this test individually was doing the proper
cleanup. I noticed that the Signal test that was checking
that files could be rotated was causing this side effect.
It turns out that none of the config reload tests were disabling
the signal handler (NoSigs=true), and since the go routine would
be left running, running the TestSignalToReOpenLogFile() test
would interact with an already finished test.
I put a thread dump in handleSignals() to track all tests that
were causing this function to start the go routine because NoSigs
was not set to true. I fixed all those tests. At this time, there
are only 2 tests that need to start the signal handler.
I have also fixed the code so that the signal handler routine select
on a server quitCh that is closed on shutdown so that this go routine
exit and is waited on using the grWG wait group.

Coverage increased (+0.1%) to 92.338% when pulling e47c91c0696f68b93d9180d4328762c06c0416ca on turnoff_signal_handler_in_some_tests into e8b524aed6da4091892045fa80bd1b2ee2df5529 on master.

LGTM, one minor spelling nit.

LGTM


Support Xbox One Console

The Xbox 1 console has the DLNA support, so Connect SDK should be able to work with it. Need to test that.

Xbox's response XML contains tags with prefixes different from expected:
<?xml version="1.0"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><SOAP-ENV:Body><m:GetVolumeResponse xmlns:m="urn:schemas-upnp-org:service:RenderingControl:1"><CurrentVolume xmlns:dt="urn:schemas-microsoft-com:datatypes" dt:dt="ui2">12</CurrentVolume></m:GetVolumeResponse></SOAP-ENV:Body></SOAP-ENV:Envelope>

e.g., SOAP-ENV:Envelope instead of s:Envelope and m:GetVolumeResponse instead of u:GetVolumeResponse. The parsing should be fixed.

Fixed the DLNA XML parser regarding the namespace in tags.

Continuing testing, there is a lot of volume subscription errors like this:
2015-01-14 09:48:53.146 Connect SDK Sampler[22102:590468] -[DLNAService valueForVolumeKey:atChannel:inResponse:] [Line 234] Unexpected contents for volume notification ((null) object)
2015-01-14 09:48:53.146 Connect SDK Sampler[22102:590468] Subscribe Vol Error A generic error occured: Could not find volume in subscription response

However, most of the notifications are not related to volume at all, so no volume subscription callback should be called. The issue is with the subscription URLs. For Sonos we use:
CALLBACK: <http://<IP_ADDRESS>:49291/MediaRenderer/RenderingControl/Event>\r\n
CALLBACK: <http://<IP_ADDRESS>:49291/MediaRenderer/AVTransport/Event>\r\n

and for Xbox:
CALLBACK: <http://<IP_ADDRESS>:49291/upnphost/udhisapi.dll?event=uuid:0f4810a6-4fb4-4fdf-8acc-81e751e7ec8a+urn:upnp-org:serviceId:RenderingControl>\r\n
CALLBACK: <http://<IP_ADDRESS>:49291/upnphost/udhisapi.dll?event=uuid:0f4810a6-4fb4-4fdf-8acc-81e751e7ec8a+urn:upnp-org:serviceId:AVTransport>\r\n

DLNAHTTPServer checks the URL path only when deciding which callbacks to call, so all notifications go through all the callbacks, which is wrong.

Fixed the subscriptions issue.

Merged into dev. Commit https://github.com/ConnectSDK/Connect-SDK-iOS/commit/5fd5f1a25feb40b46db5c4e10126b0b5fde74d7e


Do not encode raw pointers in bytecode

When codegening string initialization, BytecodeGenerator serializes and encodes a pointer allocated from ExecutionContext into the bytecode here.  ExecutionContexts are per-query, meaning these plans cannot be reused across queries since the plan can outlive the ExecutionContext that created it.
But, more generally, encoding raw pointers into bytecode makes the TBC unit un-portable across process restarts, machines etc. Truly insidious and should always be avoided.

Fixed in #994.


deps: bump vllm-tgis-adapter to 0.4.0

changelog for 0.4.0
https://issues.redhat.com/browse/RHOAIENG-11591

Smoke test failed because the build05 cluster is failing to schedule our test pod, see internal discussion on slack

Smoke test failed because the build05 cluster is failing to schedule our test pod, see internal discussion on slack

Thanks @dtrifiro for pointing to the thread


feat(ci): add nightly job

Add a cron schedule for the build_and_push job.
Closes #69 (nice)

Awesome this is great! Thank you!


Harden containers with read-only root filesystems

Is your feature request related to a problem? Please describe.
Strict security policies prefer read-only containers, as to avoid potential changes to the mission-critical areas of container during runtime (such as binary/executable files).
Describe the solution you'd like
Container Security Context should be configured to enforce securityContext.readOnlyRootFilesystem: true.
Describe alternatives you've considered

Additional context
Running as non-root already mitigates bulk of risk. This request is being opened for evaluation by team, for additional layer of protection.
/kind feature

Duplicate of #5925

GH doesn't have "Duplicate" available as close reason. Using "Not planned" to signal this is not implemented.

Changing status to "Completed" as 6d206795c70efc70395147493b296582807c4a21 implemented the change. This is simply pending a 1.14 release.


Fix: query vector copy out of bounds when using InnnerProduct

Signed-off-by: wusonglin<EMAIL_ADDRESS>issue: #253
diskann uses norm L2 to replace InnerProduct when using SSD index. It adds an extra dim at the end, this would cause a  array out of bounds issue.

/lgtm

/approve
/lgtm


beforeSend is not called for Fatal Exceptions on Android

Environment
Version: 0.2.0
Platform: Android
I'm trying to filter fatal exceptions, so that they are not reported to sentry. I use the beforeSend hook to filter out the events:
    Sentry.init(context) {
        ...
        it.beforeSend = { event ->
                if (event.level == SentryLevel.FATAL) {
                    // Ignore fatal errors
                    null
                } else {
                    event
                }
            }
    }

Steps to Reproduce

Setup Sentry and configure a beforeSend callback
Throw exception in Activity onCreate, which crashes the android app
App closes, but beforeSend is not called
Crash event is shown in sentry UI

Expected Result
beforeSend is called for fatal exception, so I can filter or modify the events before they are send to sentry.
Actual Result
I can not filter and modify the event before it is send to sentry.

Thanks @Legion2 .
We'll check it and come back to you.
@buenaflor have you seen that before? or @romtsn ?

Thanks @Legion2 . We'll check it and come back to you. @buenaflor have you seen that before? or @romtsn ?

no, first time seeing this. I guess we have to investigate with debugger attached to see what's going on

@Legion2 before the event is sent after the event is "ready" and enriched.
It depends on the platform:
iOS is after the app is restarted
Android usually before the process dies, but it can be on restart as well.

@Legion2 hi, thanks for bringing up the issue. What kind of error are you throwing in your activity? It's definitely weird that beforeSend is not being triggered. I'm trying to reproduce this issue but it seems to be working on my end. Can you show a code sample of your activity?, just to be sure.

Initially a Unable to start activity ComponentInfo {...} androidx.fragment.app.Fragment$InstantiationException: Unable to instantiate fragment ...: could not find Fragment constructor caught my attention, because it was logged to sentry and had the tag mechanism: UncaughtExceptionHandler and handled: false in sentry, however we use the above beforeSend to filter such exceptions. (Use case: We provide an sdk which is integrated in 3rd party apps, we do not want to catch all errors of the 3rd party app and send them to our sentry, because they are not actionable for us)
When I tried to reproduce this in a debug version of a demo app with a debugger attached and breakpoint set in the beforeSend hook I used error("test") to throw an exception in the onCreate of the main activity to reproduce a fatal exception. During the testing I could never trigger the breakpoint in beforeSend callback.

@Legion2 could you prepare a little demo repo that reproduces this issue?
Other than that we do not officially support using Sentry within other sdks.

Closing this due to inactivity

I think this issue is related to double initialization of SentrySDK, we are using React Native Sentry SDK and Kotlin Multiplatform SDK at the same type and they both initialize the Android SDK and there by overriding each others settings.
@buenaflor I was not able to find any documentation about that case and assume that it is not supported.

@Legion2 most likely yes that's the case. and yes the SDKs are not written with these cases in mind unfortunately

I found this flag today (https://docs.sentry.io/platforms/react-native/manual-setup/native-init/?original_referrer=https://www.google.com/) and hope this will fix most of the issues we have.


Try to update to or install 1.0.9 via Foundry VTT and get "Installation failed: Not Found".

There's something wrong with staging this version of the module.  Can't update or install via "Update All" or "Install Module".

This was found in the modules folder:

Please try again, that error was there for a small timeframe yesterday


HIVE-2492: Increase buildah memory for Konflux

In order to build hiveutil, we need a Konflux build step with higher available memory.

/override ci/prow/security

/lgtm
/approve


Can it be tested in bend / jsdom instead of Karma + PhantomJS?

should be possible to load angular + ng-describe and test under jsdom

could not :( jsdom is hard to debug


Ruby 2.7.0 installation fails on Catalina.

In file included from bigdecimal.c:13:
./bigdecimal.h:122:1: error: redefinition of 'rb_array_const_ptr'
rb_array_const_ptr(VALUE a)
^
../.././include/ruby/ruby.h:2191:1: note: previous definition is here
rb_array_const_ptr(VALUE a)
^
In file included from bigdecimal.c:13:
./bigdecimal.h:139:1: error: static declaration of 'rb_sym2str' follows non-static declaration
rb_sym2str(VALUE sym)
^
../.././include/ruby/ruby.h:1827:7: note: previous declaration is here
VALUE rb_sym2str(VALUE);
      ^
bigdecimal.c:108:1: error: static declaration of 'rb_rational_num' follows non-static declaration
rb_rational_num(VALUE rat)
^
../.././include/ruby/intern.h:184:7: note: previous declaration is here
VALUE rb_rational_num(VALUE rat);
      ^
bigdecimal.c:120:1: error: static declaration of 'rb_rational_den' follows non-static declaration
rb_rational_den(VALUE rat)
^
../.././include/ruby/intern.h:185:7: note: previous declaration is here
VALUE rb_rational_den(VALUE rat);
      ^
linking shared-object -test-/arith_seq/extract.bundle

the full error log as gist

https://github.com/rbenv/ruby-build/issues/1505#issuecomment-752274233 may help this.


pip install fails with missing dependency for wheel on Ubuntu 18.04

Describe the bug
When following the installation instructions for Linux, using Ubuntu 18:04 (from the docker ubuntu:18.04 docker image) I get a failure when running setup.py for bdist_wheel
To Reproduce
Steps to reproduce the behavior:

Create docker image using the following dockerfile:
FROM ubuntu:18.04
WORKDIR /tern
RUN apt-get update
RUN apt-get install -y git docker python3 python3-venv
RUN git clone  https://github.com/vmware/tern.git
Build the docker image:
docker build . -t tern-docker-image
Launch the docker image:
docker run -it tern-docker-image
Execute the following commands in the docker image to see the error:
cd /tern
python3 -m venv ternenv
source ternenv/bin/activate
cd tern
pip install .
Error in terminal

Building wheels for collected packages: PyYAML
  Running setup.py bdist_wheel for PyYAML ... error
  Complete output from command /tern/ternenv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-g5yh14jj/PyYAML/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpb8ckvy42pip-wheel- --python-tag cp36:
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help

  error: invalid command 'bdist_wheel'

Expected behavior
It should install the dependencies.
Environment you are running Tern on
Docker image ubutnu:18.04
Workaround
Execute: pip install wheel before executing pip install .

@davidgibsonadi Thanks for filing an issue! Tern won't run in a Docker container unless it is privileged and has access to the host machine at this time. Try following these instructions: https://github.com/vmware/tern#getting-started-with-docker

Thanks @nishakm  sorry I should have looked there first :)

Also, using pip3 install . rather than pip install . should get rid of the bdist_wheel error.


Use nova-kit/nova-packages-tool to reduce breaking change to laravel-nova requiring new build.

laravel-nova will be served via external window.LaravelNova and no longer requires asset rebuild whenever mixins contains any changes.
nova-kit/nova-packages-tool will bump new version after Nova made any minor/patch release before user can received the updated version.
dist/js/field.js has been reduced from 161 KB to 32.7 KB

Signed-off-by: Mior Muhammad Zaki<EMAIL_ADDRESS>

@Muetze42 can we merge this? current version is broken with following error:


feat(cientos): 178 v-light-helper

This will not close the issue 178

@JaimeTorrealba dunno if it's possible but add an update call helper.update() everytime there is a change in the light. Otherwise the helped will remind in the same place if you move the light or change the color

Interesting, I could use it with the renderLoop, or in the update directive cycle

With the renderloop is the same as the request animation frame, will not be performance problems right?
Option 2 it will only trigger when a reactive property (Vue) of the element changes (which is not always the case)

Interesting, I could use it with the renderLoop, or in the update directive cycle

With the renderloop is the same as the request animation frame, will not be performance problems right?
Option 2 it will only trigger when a reactive property (Vue) of the element changes (which is not always the case)

I will go for the second option since the update is not like the controls.update() which needs to be inside of the renderLoop


simple tests

address #9
example for minimal tests for hole, sph_process, and sos_triangle via test_hole.sh script and stored reference data
test is simple diff between generated data (with fixed RANSEED) and reference
TODO: incorporate docs/hole_test
TODO: add CI / GH actions
TODO: possibly split

Please feel free to take over the PR and improve/change it!

The diff does not work at the moment. One needs to filter lines in the output files that contain PREFIX.


Add o3 ipxe scripts

https://progress.opensuse.org/issues/133160

As mentioned it would likely be safer to append the new entries instead of inserting them before Tumbleweed. However, we can also just try whether it breaks tests or not.

do you happen to have a reference to a test that actually uses this?
Grepping through os-autoinst-distri-opensuse it seems that tests that perform PXE seem to bring their own supportserver.
So I would simply see if anything breaks.

Maybe no tests are using this anymore. I just remember that last time I temporarily broke the setup people were complaining :-)

If tests break due to a changed order I think the test is broken and should be fixed instead. After all, a human would also just select what it needs and not "the second entry" ;)


Continue email registration in another session or on another device

Hello, guys! Need some advice about email registration (sign up with by email) flow.
Basically we call Amplify.Auth.signUp(username: .., password: .., options: ..)
and after that Amplify.Auth.confirmSignUp(for: .., confirmationCode: ..).
But in case an app was closed before the confirmation code was entered, it seems like context is lost and we unable to continue the registration precess with that email. When I call Amplify.Auth.signUp second time, I just have an error "An account with the given email already exists", there is no any flag about email is used but I can continue with confirmSignUp step.
How would you suggest me to handle this situation? Maybe I missed something?
Thank you,
Sergey

Can you please elaborate your following comment?

there is no any flag about email is used but I can continue with confirmSignUp step.

Are you able to confirm sign up after the error is thrown by utilizing the AWSMobileClientError usernameExists(message: String) or userNotConfirmed(message: String)?
But if you can elaborate the question a little more. That would be really nice.

Hi @harsh62, yes, sorry for confusing.
I mean the flow like this:

I want to register for the first time with email and password and call Amplify.Auth.signUp
Now I should enter confirmation code but I just close my app instead
Later I decide to continue registration and open app again. For this point, as I understand, the only option is to call Amplify.Auth.signUp again, but this call returns error "An account with the given email already exists". I would expect here some sign or flag that given email not just already exists but also needs confirmation and I should go to confirmSignUp step.

So how could I continue the registration with that email in such situation?
Is there a way to determine current status(step) of registration for given email? For example for the case when I start registration on the one device and want to enter a confirmation code on another device.

@harsh62, could you provide any advise on this please?

@harsh62 or @hellopandaxx  Any solution or workaround for this?

Hello,
In this case, you'd need to call Amplify.Auth.confirmSignUp() with the username and confirmationCode to confirm the user in Step 3.

@hellopandaxx sorry for responding late to the question..
If you tried signing up a user and did not confirm, the signUp will keep failing with a usernameExists error.
Instead of signUp you should try to use the signIn api which would provide you with more context and provide the next step too. In this particular scenario the signIn API will respond with a next step in the response which would be (Amplify.AuthSignInStep) nextStep = confirmSignUp.. And if you encounter this, you could proceed with the confirmation step.
An example of how you can inspect the confirmation.
func signIn(username: String, password: String) {
    Amplify.Auth.signIn(username: username, password: password) { result in
        do {
                let signinResult = try result.get()
                switch signinResult.nextStep {
                case .confirmSignUp(let info):
                    print("Confirm signup additional info \(info)")

                    // User was not confirmed during the signup process.
                    // Invoke `confirmSignUp` api to confirm the user if
                    // they have the confirmation code. If they do not have the
                    // confirmation code, invoke `resendSignUpCode` to send the
                    // code again.
                    // After the user is confirmed, invoke the `signIn` api again.
                case .done:

                    // Use has successfully signed in to the app
                    print("Signin complete")
                }
            } catch {
                print ("Sign in failed \(error)")
            }
    }
}

Hope this answers your question.
More documentation on how sign in next steps work.


add timeout parameter to #ping (default 10 seconds)

...and delay parameter (default 0.1) - implemented by setting the socket read timeout immediately before calling #receive_response.
I believe this resolves issue #1

Good additions @joatca, thank you. Do you mind writing some tests to go with it? If you do, no worries.

Sure - I guess I'll need to learn how to write tests... ;-)

I've added some tests, as well as a commit to fix a minor spelling error.
On rare occasions these tests fail - testing timeouts is obviously sensitive to transient network conditions and local system load. I don't think that's fixable.
By the way, your final test "sequence number" always fails on my system. It's not my changes because I tried it on your original code and it failed there too. The first two pings seem to both come back with sequence number 0.


Auto-TensorRT engine compilation, or improved documentation for it

Feature request
For decoder models with cache, it can be painful to manually compile the TensorRT engine as ONNX Runtime does not give options to specify shapes. The engine build could maybe be done automatically.
The current doc is only for use_cache=False, which is not very interesting. It could be improved to show how to pre-build the TRT with use_cache=True.
References:
https://huggingface.co/docs/optimum/main/en/onnxruntime/usage_guides/gpu#tensorrt-engine-build-and-warmup
https://github.com/microsoft/onnxruntime/issues/13559
Motivation
TensorRT is fast
Your contribution
will work on it sometime

https://github.com/microsoft/onnxruntime/issues/13851 will make this much easier

Is there any update on this? I also suffer from the difficulty to use TensorRT as the provider after run optimum-cli optimization.

Hi,
ORT TRT currently supports explicit input shape meaning users can provide the shape range for all the dynamic shape input.
Please see the PR as well as the doc for usage/details.
Let us know if you have further questions or other feedbacks for ORT TRT. We are willing to make ORT TRT easier to use.

@chilo-ms Thanks a lot, that looks great! It is not at the top of our todo for now, but we're welcoming community contribution to interface well TensorrtExecutionProvider with ORTModel classes!


follow from accounts doesn't work

follow from accoubts doesn't work: popup is opened and the list of accounts is shown, but nothing can be selected
browser: Chrome

I would like to bump this up and add a visual representation of the issue.

This is what it looks like. It was fixed temporarily when the new version was uploaded, but when the changes to the package were made 8 hours ago, it seems to have reverted the fix.
Somewhat inconvenient and would appreciate a fix whenever possible!

I didn't change anything in code related to following, also I don't see anything on your screenshot

That is exactly the issue! It should have a button that allows you to follow the account, which, as you can see (or not see) is missing. This is in Opera, Chrome seems to be working just fine

This is what it should look like when the button is actually there