Is there a local backup option?

I can't find(if there is) an option for ONLY local backup. No G, no private.

I wanted to ask the same question. I had a look at the source code and it seems that the only backup option implemented is the IMAP option. Also, adding a local backup option might not be trivial.
@jberkel Is it a feature you are interested in? I can work on a pull request.

This would be great! Once created a local backup in the local storage, the termux software can be used to analysis the message or call log and do anything I want. @Nicop06 do you have any plan implement this?

@chaoqing I completely forgot about that. I currently don't have time to work on it. I will update this issue if I find the time to start implementing this feature unless someone else wants to work on it.

What do you mean by "local backup"? Copying the data to a database/csv file?

On the sd card. So that it can be exported using cloud synchronization applications for instance.

I'd prefer (esp. for privacy reasons) not to store a copy of the user's data locally.

But uploading to Google is an A-OK privacy move?

@AeliusSaionji it's a question of responsibility: if they are stored on Gmail I'm not responsible for privacy issues (Google is), but storing locally is a different thing (cf. the privacy policy)
Aren't there already dedicated apps which just do local backups?

If there's a solution on Fdroid that still works on Oreo and supports import/export, I haven't seen it.

Specifically open source apps I don't know, was thinking of "SMS Backup & Restore" (which uses some XML format I believe)

A common use case would be migrating from an old to a new phone. Especially for users with google free phones.

@jberkel  Privacy concern is exactly why I prefer to store locally.
Also, "SMS Backup & Restore"  seems to have been sold to some company. Before it was a one man show - Ritesh Sahu, now SyncTech Pty Ltd controls the app.
There has been some bad precedents on companies buying popular small apps.
It would be great to have an option to store it locally ( internal sd, as external sd causes headaches for developers due to googles lack of proper code regarding external devices).
Regarding to privacy concerns of local storage, a popup disclaimer, xould warn users about any privacy issues due to saving data locally, i.e. easily accessible to anyone etc.

I just want to add my voice to those requesting a local import/export. A trusted open source application is not in the same class as some shady app from the Play store that does who knows what with our data. A local file can be deleted after use. We would have to run a local IMAP server to be able to achieve that level of control with this app without local import/export, and that seems a bit of an overkill.

as a half workaround there is "slight backup"
https://f-droid.org/en/packages/de.shandschuh.slightbackup/
https://github.com/handschuh/Slight-backup
unfortunately already 5 years old and maintainer seems to be dead.
i was able to export smses. it produces an xml file. but i was not able to import. it just says import successful but no sms imported.

What about MMSes? Neither does SlightBackup nor SMSGate backup them. It would be useful to have sms-backup-plus to save them locally.

I suggest, if this is to be implemented, that it uses MailDir format, rather than combining multiple messages in an archive (eg, XML).  That means you don't need to worry about combining or overwriting such files when you export, since every individual message is its own file.
This would behave similarly to an IMAP service, so it may simplify some of the coding needed.
Taking that one step further, one possible approach would be to install an IMAP server on the phone, as a separate app, with its own privacy policy. Does anyone know of such a thing?

@kurahaupo, Epistolaire has a SMS/MMS-JSON to Maildir converter, but this specific converter is written in Python. You might still be interested: https://github.com/hydrargyrum/epistolaire/tree/master/converters/maildir
Python interpreters exist on Android, and if the need arises, the converter could be ported to Kotlin with some effort.
Some of the design decisions were:

each conversation on SMS/MMS app corresponds to one mail thread
each conversation has a virtual root message which doesn't exist, i.e. it's not present in the maildir
all messages in a conversation include a References header pointing to that root message
this is useful for subsequent backups, the root message id always stays the same for a conversation, even in the case where a backup has no common message with a previous backup

@hydrargyrum thanks for those very useful points.
I've added some improvement suggestions for you.
I'll look into how threading is currently implemented in SMS Backup+ to see whether these ideas can be incorporated.

channelling further discussion into #974


fix: misleading error message in PublicState::read

Apply aayush's fix on head

Benchmark results
Metrics with a significant change:

note_history_trial_decrypting_time_in_ms (5): 389 (+219%)
note_trial_decrypting_time_in_ms (32): 105 (+177%)

Detailed results
All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call  to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.
This benchmark source data is available in JSON format on S3 here.
Values are compared against data from master at commit 7438db31 and shown if the difference exceeds 1%.
L2 block published to L1
Each column represents the number of txs on an L2 block published to L1.

Metric
8 txs
32 txs
128 txs

l1_rollup_calldata_size_in_bytes
45,380
179,492
716,036

l1_rollup_calldata_gas
221,168
865,940
3,446,264

l1_rollup_execution_gas
928,940
3,899,754
24,032,735

l2_block_processing_time_in_ms
1,427 (+2%)
5,425 (-1%)
22,237 (+3%)

note_successful_decrypting_time_in_ms
310 (-1%)
978 (+7%)
3,424 (+3%)

note_trial_decrypting_time_in_ms
19.8 (-9%)
:warning: 105 (+177%)
138 (+1%)

l2_block_building_time_in_ms
16,208 (+1%)
64,137
258,386 (+1%)

l2_block_rollup_simulation_time_in_ms
11,787 (+1%)
46,742 (+1%)
188,411 (+1%)

l2_block_public_tx_process_time_in_ms
4,381 (+1%)
17,293
69,559 (+1%)

L2 chain processing
Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric
5 blocks
10 blocks

node_history_sync_time_in_ms
16,802 (+4%)
31,776 (+5%)

note_history_successful_decrypting_time_in_ms
2,291 (+5%)
4,508 (+5%)

note_history_trial_decrypting_time_in_ms
:warning: 389 (+219%)
226 (+28%)

node_database_size_in_bytes
3,580,403
3,640,980

pxe_database_size_in_bytes
29,923
59,478

Circuits stats
Stats on running time and I/O sizes collected for every circuit run across all benchmarks.

Circuit
circuit_simulation_time_in_ms
circuit_input_size_in_bytes
circuit_output_size_in_bytes

private-kernel-init
213
44,681
25,561

private-kernel-ordering
166 (+1%)
43,753
14,809

base-rollup
1,196
128,338
881

root-rollup
79.3
4,088
677

private-kernel-inner
293
71,208
25,561

public-kernel-private-input
206
31,863
25,561

public-kernel-non-first-iteration
204
31,905
25,561

merge-rollup
7.77 (+2%)
2,608
881

Tree insertion stats
The duration to insert a fixed batch of leaves into each tree type.

Metric
1 leaves
2 leaves
8 leaves
16 leaves
32 leaves
128 leaves
64 leaves
512 leaves
1024 leaves
2048 leaves
8192 leaves

batch_insert_into_append_only_tree_16_depth_ms
13.0 (+5%)
21.0 (-2%)
13.6
21.5 (+6%)
26.3 (+4%)
67.6 (+3%)
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_count
16.9
17.5
23.0
31.6
47.0
143
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_ms
0.635 (+2%)
0.653 (-3%)
0.548 (-1%)
0.543 (-1%)
0.485
0.445 (+1%)
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_32_depth_ms
N/A
N/A
N/A
N/A
N/A
100 (+9%)
52.8 (+3%)
256 (+3%)
484 (+1%)
926
3,590 (+1%)

batch_insert_into_append_only_tree_32_depth_hash_count
N/A
N/A
N/A
N/A
N/A
159
96.0
543
1,055
2,079
8,223

batch_insert_into_append_only_tree_32_depth_hash_ms
N/A
N/A
N/A
N/A
N/A
0.519 (+5%)
0.497 (+1%)
0.446 (+2%)
0.436
0.431 (-1%)
0.428

batch_insert_into_indexed_tree_20_depth_ms
N/A
N/A
N/A
N/A
N/A
127
65.5 (+5%)
385 (+3%)
792 (+4%)
1,470 (+2%)
6,010 (+2%)

batch_insert_into_indexed_tree_20_depth_hash_count
N/A
N/A
N/A
N/A
N/A
197
104
691
1,363
2,707
10,771

batch_insert_into_indexed_tree_20_depth_hash_ms
N/A
N/A
N/A
N/A
N/A
0.519 (+2%)
0.508 (+1%)
0.489 (+2%)
0.485 (+1%)
0.478
0.486

batch_insert_into_indexed_tree_40_depth_ms
N/A
N/A
N/A
70.7 (+4%)
N/A
N/A
N/A
N/A
N/A
N/A
N/A

batch_insert_into_indexed_tree_40_depth_hash_count
N/A
N/A
N/A
94.1
N/A
N/A
N/A
N/A
N/A
N/A
N/A

batch_insert_into_indexed_tree_40_depth_hash_ms
N/A
N/A
N/A
0.599
N/A
N/A
N/A
N/A
N/A
N/A
N/A

Miscellaneous
Transaction sizes based on how many contracts are deployed in the tx.

Metric
0 deployed contracts
1 deployed contracts

tx_size_in_bytes
15,455
32,219

Transaction processing duration by data writes.

Metric
0 new commitments
1 new commitments

tx_pxe_processing_time_ms
378 (-6%)
1,091 (+1%)

Metric
1 public data writes

tx_sequencer_processing_time_ms
540 (+1%)


Solution/ Feature Request for testing API while using Postgres

Having a test_client that can stack requests would be great while using after request and before requests to handle database connection.
I use before request to create a connection to database and after making the request, if any error doesn't occur, I commit the connection, otherwise roll it back.
This setup was great so far until I reached testing. with test_client() I can only make one request and since the before and after request will run while using test_client, my database will get affected with test data which is not my preference.
I tried using test_request_contaxt to have a control on before and after requests funcs, but still cannot stack the requests.
for example I want to test a functionality called "update user". As the name suggest we need to sign up a test user, login in to it, and thet update the feature we want (for example name, email, etc). It would be great if we could stack all three requests and at the end rollback the changes.
Maybe my sample/ sudo codes illuminates my point better:
# create app and register blueprints
app = create_app(DevelopmentConfig)

# Create a Global Connetion Pool:
connections = conn_pool(1, 10) # using psycopg2.ThreadedConnectionPool

@app.before_request
def open_conn():
    global connections
    g.conn = connections.getconn()

@app.after_request
def close_conn(response):
    if g.conn is not None:
        global app
        global connections
        if app.config['TESTING']:
            g.conn.rollback()
            connections.putconn(g.conn)
            g.conn = None
            return response
        elif not app.config['TESTING']:
            g.conn.commit()
            g.conn.cursor().close()
            connections.putconn(g.conn)
            g.conn = None
            return response
    else:
        return response

############# TESTING USING PYTEST :
def test_login2(app_runner):
    with app.test_client() as client:
        rs = client.post('/auth/signup',
                         data={'inputEmail'<EMAIL_ADDRESS>                               'inputPassword': 'A',
                               'client_name': 'ForTest'}
                         )
        data = json.loads(rs.data)['data']
        error = json.loads(rs.data)['error']
        assert error is None
        assert data is not None
        assert rs.status_code == 201
        rs2 = client.post('/auth/login',
                          data={'inputEmail'<EMAIL_ADDRESS>                                'inputPassword': 'A'})
        data = json.loads(rs2.data)['data']
        error = json.loads(rs2.data)['error']
        assert error is None
        assert rs2.status_code == 200

assert rs2.status_code == 200 will fail because we have created a user, and rolled back the connection. So we cannot login in to that.
Is there any feature that I am not aware of to handle this situation without restructuring my app?
Many thanks in advance

This is outside the scope of Flask. There are libraries like Pytest-Flask-SQLAlchemy that help with testing databases.

Thanks for your quick reply.
I managed to hack it somehow... I pushed an app_context, set a connection on g and after yielding test_client, I rolled back the connection. It is working like a charm...
@pytest.fixture(autouse=True, scope='module')
def _app_runner():
    app = creat_app(TestingConfig)
    yield app

@pytest.fixture(autouse=True, scope='module')
def client(_app_runner):
    conns = conn_pool(1,10)
    conn = conns.getconn()
    with _app_runner.app_context():
        g.conn = conn
        g.secret_key = _app_runner.config['SECRET_KEY']
        with _app_runner.test_client() as client:
            yield client
            conn.rollback()
            conns.putconn(conn)


Build time configurable library management

[X] I've read the CONTRIBUTING guidelines

This is a pre-requisite to automate integration tests for alternative implementations of librarymanagement as discussed in the Gitter channel with @eed3si9n and @leonardehrenfried

ping for a review, this should be easy.

Done all requested renamings, squashed and rebased on latest develop.


Fixed RO placement and generates preliminary layout

Work in progress, need to create a blockage to push all divider cells to the right.
Pin locations need to be adjusted.

Great job @tuohangzeng . Thanks!


Strings should be extracted "as is"

// M.fst
module M
let s = "×"

$ fstar.exe --universes --codegen OCaml M.fst
$ cat M.ml

open Prims

let s : Prims.string = "Ã"

In the source file × is 0xC397. In the extracted file, it is 0xC383C297
See #391.

This works as intended now, but the problem still occurs when using \x, as in let s = "\xC3\x97".  Opened #1458 to track it.


addOnTextChangedListener()

Need to be able to add a listener. Kinda defeats the purpose of an EditText without it.

Actually there is a listener in code but i forgot to add a "set" method for it because i didnt need it at that time.
Added now in version 1.2.


add more details when runtime init failed

What type of PR is this?

/kind documentation
What this PR does / why we need it:
add more details when runtime init failed
Which issue(s) this PR fixes:

Fixes #361
Special notes for your reviewer:
Does this PR introduce a user-facing change?

Additional documentation e.g., design docs, usage docs, etc.:

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
4 unchanged lines in 2 files lost coverage.
Overall coverage increased (+0.07%) to 72.55%

Files with Coverage Reduction
New Missed Lines
%

pkg/engine/operation/watch.go
1
85.93%

pkg/engine/operation/parser/parser.go
3
78.21%

Totals

Change from base Build<PHONE_NUMBER>:
0.07%

Covered Lines:
5109

Relevant Lines:
7042

💛  - Coveralls


Initial IP settings support

Add initial support to get and configure IP address. This needs nmstate to run the GLib MainLoop so that the async() methods to activate the connections are actually called. Inspecting /etc/sysconfig/network-script  shows here, that IP settings are changed as expected.

Coverage decreased (-50.7%) to 36.842% when pulling 64b2d7693b2ee64d48110325291c8314e46b399a on tyll:ipsettings into 07d3800bf3922f1d3c10199f2f40af9a29726db8 on nmstate:master.

This is how I made it work:
https://github.com/tyll/nmstate/commit/97cb145d27ed9195fe6e2bb2f6c3735f4f0eb0f7
It still needs an externally generated NM configuration for the interface.
Any idea why tox fails in Travis? it works locally...

#20 suggests a specific structure for the code.
Lets try to agree on the structure first so it will be simpler to extend the functionality (adding types and options).

I split this into #27, #28, #29  and it should address your comments


[v0.14] tests of fluent-plugin-sql fails with fluentd v0.14.0.rc.1

How to reproduce
git clone https://github.com/fluent/fluent-plugin-sql
cd fluent-plugin-sql

Change in Gemspec
gem.add_dependency "fluentd", [">= 0.14.rc.1", "< 2"]

Run tests
bundle exec rake test

Result
/Users/kakoni/dev/fluent-plugin-sql/lib/fluent/plugin/in_sql.rb:22:in `<module:Fluent>': uninitialized constant Fluent::Input (NameError)
        from /Users/kakoni/dev/fluent-plugin-sql/lib/fluent/plugin/in_sql.rb:18:in `<top (required)>'
        from /Users/kakoni/.rubies/ruby-2.3.0/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:120:in `require'
        from /Users/kakoni/.rubies/ruby-2.3.0/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:120:in `require'
        from /Users/kakoni/dev/fluent-plugin-sql/test/helper.rb:6:in `<top (required)>'
        from /Users/kakoni/.rubies/ruby-2.3.0/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:120:in `require'
        from /Users/kakoni/.rubies/ruby-2.3.0/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:120:in `require'
        from /Users/kakoni/dev/fluent-plugin-sql/test/plugin/test_in_sql.rb:1:in `<top (required)>'
        from /Users/kakoni/.rubies/ruby-2.3.0/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:120:in `require'
        from /Users/kakoni/.rubies/ruby-2.3.0/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:120:in `require'
        from /Users/kakoni/.gem/ruby/2.3.0/gems/rake-11.1.2/lib/rake/rake_test_loader.rb:15:in `block in <main>'
        from /Users/kakoni/.gem/ruby/2.3.0/gems/rake-11.1.2/lib/rake/rake_test_loader.rb:4:in `select'
        from /Users/kakoni/.gem/ruby/2.3.0/gems/rake-11.1.2/lib/rake/rake_test_loader.rb:4:in `<main>'

Expected result
Pass tests.
Proposed fix
Include needed requires as per https://github.com/fluent/fluentd/commit/7d4baa06b8dc8c89cb05e7ce7950ed867b5c4a98

Also compat/output failures as in https://github.com/fluent/fluentd/issues/975

duplicated with https://github.com/fluent/fluentd/issues/973


Improve perfs of the font renderer

Some SVG paths are generated from the font and used in the main thread to render the glyphs.

/botio test

The numbers are a bit impressive:

https://share.firefox.dev/3VsOC2g (disableFontFace: false), 137ms in showText
https://share.firefox.dev/3ZIpmYs (disableFontFace: true, no patch), 401ms in showText
https://share.firefox.dev/4gq9eQE (disableFontFace: true, patch), 55ms in showText

I want to do the same kind of stuff for general paths and I hope we could have better performance with large maps.

/botio test

/botio test

/botio makeref


Change itemsPerPage dynamically

Would like a way to change itemsPerPage after the table is loaded. Would then be able to build user controls for changing the number of itemsPerPage.
Another option would be to have this feature built in and having the option of setting intervals like 5,10,25,100,500 itemsPerPage.

Does it not work for passing in as a prop and then changing the props?

I just built a user control for changing the number of itemsPerPage prop to the table and it works just fine.


CI: Use 2.6.3

This PR only updates the CI matrix to the latest generally available Ruby versions.
Source for version numbers: https://github.com/rvm/rvm/blob/master/config/known

Thanks!


validate if timkey is less than equal 0

Signed-off-by: Yuta Iwama<EMAIL_ADDRESS>
Which issue(s) this PR fixes:
Fixes https://github.com/fluent/fluentd/issues/2607
What this PR does / why we need it:
I made a change to raise an error when a user specifies timekey  less than equal 0.
when timekey is 0, fluentd will raise ZeroDivisionError at here.
Docs Changes:
no need
Release Note:
same as the title

Looks good.
The potential problem is fluentd uses non-strict conversion for parameters like https://github.com/fluent/fluentd/blob/07d9eecf0ce1413ab7873da84462b6f8fe9fae04/lib/fluent/config/types.rb#L101
We should improve this point to use Integer() or others.
This change has compatibility probolem so it will be implemented in v2 or via configuration.


Node run.js ends abruptly with no error

Love the demo. I am trying to get this to run on my Ubuntu 16.04 VMWare insstance. It looks like it is going to work and Watson Says "Hi there... I am awake." but then it does not wait for me to reply and goes to the command prompt with no errors.. Any idea of what May be happening?
jdellaria@ubuntu:~/IBM/tjbot-sports-buddy$ node run.js
Retrieved MLB Teams
Retrieved MLB standings
Retrieved schedule for date: 2017-07-28T00:00:00
Retrieved schedule for date: 2017-07-31T00:00:00
Retrieved schedule for date: 2017-07-29T00:00:00
Retrieved schedule for date: 2017-08-02T00:00:00
Retrieved schedule for date: 2017-08-01T00:00:00
Retrieved schedule for date: 2017-07-30T00:00:00
Retrieved schedule for date: 2017-08-03T00:00:00
Retrieved MLB schedules TJBot is listening, you may speak now.
Microphone paused for 2.1126080000000003 seconds.
Microphone resumed.
jdellaria@ubuntu:~/IBM/tjbot-sports-buddy$

Hey @jdellaria silly question, my gut says it's an audio config issue, does the microphone on your Ubuntu VM work at all? Can you record something? Both @rhagarty and I have used OSX for our demos -- we outlined some of the issues we found with audio configuration here: https://github.com/IBM/tjbot-sports-buddy#troubleshooting

Yes. I can record Mp3, ogg and wav files by using audio-recorder. The audio quality seems to be rather good. Is there a particular format or device you are looking for? The devices that work for recording are "ES1371/Creative Labs CT2518/ES1373 (Microphone)", "System's default device" and "user defined audio source"

Hello @jdellaria - I've updated https://github.com/IBM/tjbot-sports-buddy#troubleshooting with some mic tests and fixes. Please let me know if this fixes your issue.

Perfect!
Thanks!
Jon
On Fri, Jul 28, 2017 at 2:12 AM, Rich Hagarty<EMAIL_ADDRESS>wrote:

Hello @jdellaria https://github.com/jdellaria - I've updated
https://github.com/IBM/tjbot-sports-buddy#troubleshooting with some mic
tests and fixes. Please let me know if this fixes your issue.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/IBM/tjbot-sports-buddy/issues/21#issuecomment-318569344,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AVYbCYjX9JzLhQLDLNniQN7r88qLJH7Lks5sSXu8gaJpZM4Olrjr
.

I'll take that as I can close this issue now :)

Sure... :)
On Fri, Jul 28, 2017 at 8:57 AM, Steve Martinelli<EMAIL_ADDRESS>wrote:

Closed #21 https://github.com/IBM/tjbot-sports-buddy/issues/21.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/IBM/tjbot-sports-buddy/issues/21#event-1183466540,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AVYbCTyKtNBHGFwBEuC0uACttuGXiuf2ks5sSdq1gaJpZM4Olrjr
.


There is "-number" for volume working but not "+number" for volume -> would be cool for symfosik remote to just have + and -

Description
I use a symfosik remote which can be clicked once/twice/or three times for play/pause, skip, skip backward.
Also it can be rotated left or right.
For that it would be gread to have also an volume + function in the node - actual volume plus the number given.
Yes I could not go on and build some calculations with actual volume plus 1 - but nicer would be the counterpart to the given "-1" for player.adjust.volume :-)  -> +1 for player.adjust.volume.
What node/command (topic)/state (payload)
node: Universal
command: player.adjust.volume
Versions and Infrastructure
what system: Rancher Kubernetes Cluster - on a Ubuntu LTS
node-red-contrib-sonos-plus version: 5.2.7
Node-RED version: 2.1.4
NodeJS version: v14.18.2
Would be funny if I tested it wrong and it is still working - i tried as number "+1" and als string -> no change at the volume.
Merry Xmas!

Uhhmm wait a second......
Seems only one SONOS does not what it should.... and it seems to be already there - says the wiki :-D

Okay all working :-D
my fault :-D one connection missing in node red :-D
Great Work! Thanks.


Show weapon notes in attack cards

When you roll an attack, it would be great if any text in the Notes field shows up in the attack card.

Covered by #390


Added function to edit data source relationship settings in the dashboard

Is your feature request related to a problem? Please describe.
Added function to edit data source relationship settings in the dashboard
Describe the solution you'd like
-We need to add an edit button on datasources in the dashboard detail page.

-Go to the Data Source Connection Settings and Joins popup.

@AnnieHwang  아래 기획서 중 8번 영역(데이터소스의 연결키 아이콘, 툴팁)은 퍼블리싱 결과물을 어디에서 찾을 수 있나요?

@AnnieHwang 찾았습니다.


Function components

Any plans to move from class component to function component using hooks?

I would love to see this change as well


[FEATURE] Add CONTRIBUTING.md file

Description

Add the contributing.md file explaining how to clone the repo and get started with contributing.
What are the requirements one should have before cloning the repo?
Mention how to propose a feature request?
How to create a pull request?

Screenshots
No response
Additional information
No response

Can I work on this?

Sure @Santosh1176 You can start working on this.

Hey, I'd like to work on this issue.
Please let me know.


[BUG] Client components render on server

It seems that components marked with .client.jsx render on the server as well, causing problems when using useServerState or even just basic window APIs.
For instance, retrieving the pathname from location requires an if statement (if (typeof window != "undefined")) which in turns will return errors in the hydration on the client, since the classes for instance have changed because the pathname is different from the server than the client.
Which leads me to my point, the docs explicitly say that .client.jsx files will only render on the client: https://shopify.dev/custom-storefronts/hydrogen/framework/react-server-components
This seems like a regression. I've only noticed this now that we can't use useLocation from react-router

Do you have a repo or Stackblitz or something we can look at? Thanks!


Setting Width of Canvas for DZI Tile Source Firefox Issue

Hi @iangilman thank you again for your help on my last issue with the overlay plugin. I am now running into an issue with tileSource sizing to circumvent the Firefox SVG issue other's have mentioned.
In a prototype of the application I am trying to create I am able to use this code
        var viewer = OpenSeadragon({             id: 'a6',             prefixUrl: 'https://openseadragon.github.io/openseadragon/images/',             tileSources: {                 tileSource: './7045-A7-NEUN.dzi',                 width: 1000,             }         });
Specifically you can see in the TileSources I am able to set a tileSource property with the url and a width related to the tileSource. This allows me to multiply my svgOverlay width and height by 1000 and it fixes the issue with Firefox.
            var d3Svg = d3.select(overlay.node())             .append("svg:image")             .attr("xlink:href", "./7045-A7-NEUN.svg")             .attr("width", 1000)             .attr("height", ratioComputedHeight * 1000)         })
When I attempt to utilize the same tileSource formatting in my React application I receive this error
No TileSource was able to open undefined [object Object]
Setting the width of the tileSource in my sample application was a huge breakthrough for getting Firefox to work with the SVG but now I'm unable to replicate that in my application.
Any help you can offer would be greatly appreciated, thank you in advance!

Hmm... It should work the same in React. Perhaps there's some subtle difference in how you are expressing it there? Unfortunately, the tileSources option is so flexible it can be hard to work with. Can you share a code snippet of how you're doing it in React?

Yeah definitely and as always thank you again for the help!
Here is the version in React that is currently working in Chrome. I had abandoned trying to use tileSources as a property in the initialization of OpenSeaDragon because of the issues I was running into and instead and using the viewer.open with a string to a .dzi source. Below this code is an example of my prototype where I am able to use the tileSources property on intialization and I can pass in a tileSource width which I haven't been able to do in this React code
` useEffect(() => {
if (tileSource && viewer) {
viewer.open(tileSource);
}
// viewer needs to be opened again only on image change and not on viewer update.
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [tileSource]);
const InitOpenseadragon = () => {
    viewer && viewer.destroy();
    setViewer(
        OpenSeaDragon({
            id,
            ...OSD_CONFIGRATION,
        })
    );
};

useEffect(() => {
    InitOpenseadragon();

    return () => {
        viewer && viewer.destroy();
    };
    // Initializing open seadragon should happen only on mount, and not on viewer update.
    // eslint-disable-next-line react-hooks/exhaustive-deps
}, []);

// On viewer tileSource change the svg overlay needs to be updated
useEffect(() => {
    if (viewer) {
        viewer.addHandler('open', () => {
            // remove any existing svg overlays on image change
            selectAll('.svgOverlay').remove();
            const overlay = OsdSvgOverlay(viewer);
            const ratioComputedHeight = 1 / viewer.source.aspectRatio;
            svgSource &&
                select(overlay.node())
                    .append('svg:image')
                    .attr('xlink:href', svgSource)
                    .attr('class', 'svgOverlay')
                    .attr('width', 1)
                    .attr('height', ratioComputedHeight);
        });
    }
    // eslint-disable-next-line react-hooks/exhaustive-deps
}, [tileSource]);`

Prototype code with working tileSource width Properties
`        var viewer = OpenSeadragon({
id: 'a6',
prefixUrl: 'https://openseadragon.github.io/openseadragon/images/',
tileSources: {
tileSource: './7045-A7-NEUN.dzi',
width: 1000,
}
});
    var overlay = viewer.svgOverlay();

    viewer.addHandler('open', function() {
        var overlay = viewer.svgOverlay();
        var ratioComputedHeight = (1 / viewer.source.aspectRatio)
        console.log(ratioComputedHeight, viewer.viewport)
        var d3Svg = d3.select(overlay.node())
        .append("svg:image")
        .attr("xlink:href", "./7045-A7-NEUN.svg")
        .attr("width", 1000)
        .attr("height", ratioComputedHeight * 1000)
    })`

So you can see in the prototype I'm setting a tileSource within tileSources as well as a width. I tried the same syntax in the React code and it threw an error. I also tried
tileSources : {url: dziSource, width: 1000} which didn't upset the linter but it failed to load with an error along the times of "cannot read undefined of [object][object]."
Any thoughts come to mind?
Your help is much appreciated!

You can use open just the same as tileSources. You would do it like so:
viewer.open({ tileSource: tileSource, width: 1000 });

And yes, you can change the width of images after they are loaded as well. For instance:
viewer.world.getItemAt(0).setWidth(1000, true);

BTW, for your code examples, you wrapped them in a single backtick, but for multi-line examples, you should use three backticks (```). I've gone ahead and fixed yours so they're easier to read :)

Ah thank you so much Ian and roger on the backticks good to know moving forward. Can't thank you enough for all your help, you're effort to respond so promptly is really really appreciated.

Excellent... I'm glad to be of help! If you want, you can try to convince your organization to sponsor OpenSeadragon on https://opencollective.com/openseadragon. 😉

Yeah that's a great idea! I can't make any promises but I'll definitely ask my supervisor if there is something we can do.

Thank you! If they don't go for it, and you have feedback as to why, I'd love to hear that as well. Feel free to contact me directly if you don't want to do it here in public 😄

Yeah that sounds good I'll definitely let you know either way once I'm able to get an answer from my org.

Hey @iangilman I wanted to follow up with you, my manager has put a request in to higher ups to see if there is anything we can do in the form of sponsorship for OSD.
On a separate note I realized that while viewer.open({ tileSource: tileSource, width: 1000 }); worked for getting the overlay to open and be displayed on firefox it broke the ability to zoom in and out of the image. I got so excited that the overlays were showing in Firefox at the time that I forgot to test the zoom doh!

Hey @iangilman I wanted to follow up with you, my manager has put a request in to higher ups to see if there is anything we can do in the form of sponsorship for OSD.

Wonderful, thank you!

On a separate note I realized that while viewer.open({ tileSource: tileSource, width: 1000 }); worked for getting the overlay to open and be displayed on firefox it broke the ability to zoom in and out of the image. I got so excited that the overlays were showing in Firefox at the time that I forgot to test the zoom doh!

What other options are you using when creating the viewer? Are you doing anything to limit how far you can zoom? If not, does it seem like it might be an issue with the overlay stealing events from the viewer (I don't expect that's it, since it's usually the other way around, but I have to ask)?

Thanks Ian as always super grateful for the quick replies. Here are the viewer configurations
                id,
                ...OSD_CONFIGRATION,
            })```

export const OSD_CONFIGRATION = {
animationTime: 0.5,
blendTime: 0.1,
constrainDuringPan: true,
showZoomControl: false,
showHomeControl: false,
showFullPageControl: false,
maxZoomPixelRatio: 2,
minZoomLevel: 1,
visibilityRatio: 1,
zoomPerScroll: 2,
};,```
Please let me know if you see anything in there that might be an issue, per your comment maybe maxZoomPixelRation: 2?

I'm guessing it's actually the minZoomLevel. I believe the zoom value is relative to the default image width of 1, so when you change the width to 1000, you now start out zoomed at 0.001 or something. You can easily do viewer.viewport.getZoom() (after it's loaded and settled) to verify that.
What is your purpose for using minZoomLevel? There's probably a better way to do it, like minZoomImageRatio which is relative, rather than absolute.

Awesome! :)


.env Files Being Uploaded to the Codebase

Problem:
The .env files, which contain sensitive configuration data, are being mistakenly uploaded to the codebase. This is exposing critical information such as API keys, database credentials, and other secrets.
Expected Behavior:
.env files should be excluded from the repository to prevent accidental exposure of sensitive data. This is typically done by adding the .env file to the .gitignore file.

go ahead @OINDIL , work on it ;)


Banyandb integration

Add an OAP storage plugin to support Skywalking-BanyanDB

[x] UTs have been added to cover query build and write entity build (IT and E2E may not be included in this preliminary version)

[x] If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #7412

[ ] Update the CHANGES log.

Request review @hanahmily @wu-sheng @kezhenxu94

Please don't change anything in another storage option, you should copy all from its provider to your provider. Because we will see they are all going to be removed eventually.

I think you get me wrong about this point. I said copy all from its provider to your provider, not copy all codes. It is still OK to depend on the H2 storage in this phrase. My point is, don't change the source codes of H2 storage, because you are doing a POC level testing integration.

e2e tests for your new storage is required, when the BanyanDB trace core id testable.

This is not provided yet.
And your official codes, BanyanDB integration, should have comments.

One important suggestion before I am going deeply in code review.
As a database client(we may need to release it separate one day), we need to organize codes and keep the concept consistent.

Schema is usually referencing a database.
Table/Index/Entity is the object we are going to write and read.
Client is the one controlling the connection. We could have typically SimpleClient or BatchClient to deal with different data model. Or add QueryClient in the future.
All read/write/query should be organized on the top of different entity, which is the normal case. Take a reference from InfluxDB or ElasticSearch client. Such as, TraceWrite should be an object could host the fields and tags. It also could be accepted by BatchClient only.

All above are not mandatory, but I hope these could inspire you that, this is a very serious set of APIs, we need to make it good.

Notice, all protobuf and gRPC definition are network protocol, like MySQL and ElasticSearch JSON format. What we need in SkyWalking are a set of Java BanyanDB Client APIs.
If needed, we could do review on those APIs first, but change all the codes. Feel free to set up a discussion on GitHub issue, mail list or meeting.
We need to take this very seriously. @hanahmily Please set this discussion up.

Notice, all protobuf and gRPC definition are network protocol, like MySQL and ElasticSearch JSON format. What we need in SkyWalking are a set of BanyanDB Java Client APIs.
If needed, we could do review on those APIs first, rather than change all the codes. Feel free to set up a discussion on GitHub issue, mail list or meeting.
We need to take this very seriously. @hanahmily Please set this discussion up.

Sure. We will setup a meeting tomorrow morning (11 Aug.) and have a discussion first.

One important suggestion before I am going deeply in code review.
As a database client(we may need to release it separate one day), we need to organize codes and keep the concept consistent.

Schema is usually referencing a database.
Table/Index/Entity is the object we are going to write and read.
Client is the one controlling the connection. We could have typically SimpleClient or BatchClient to deal with different data model. Or add QueryClient in the future.
All read/write/query should be organized on the top of different entity, which is the normal case. Take a reference from InfluxDB or ElasticSearch client. Such as, TraceWrite should be an object could host the fields and tags. It also could be accepted by BatchClient only.

All above are not mandatory, but I hope these could inspire you that, this is a very serious set of APIs, we need to make it good.

I completely agree with you that we could have provided a well-encapsulated, clearly modularized, hierarchical Java Client for BanyanBD.
Since I am not familiar with the OAP server and its APIs, I temporarily provide a POC to make it work. We may discuss some more details tomorrow.

I completely agree with you that we could have provided a well-encapsulated, clearly modularized, hierarchical Java Client for BanyanBD.
Since I am not familiar with the OAP server and its APIs, I temporarily provide a POC to make it work. We may discuss some more details tomorrow.

The key issue right now, is not familiar with SkyWalking or not. Instead, you are mixing SkyWalking and BanyanDB.

Sure. We will setup a meeting tomorrow morning (11 Aug.) and have a discussion first.

@lujiajing1126 please share the meeting link

https://meeting.tencent.com/dm/9rnGuYvqjTCj


Proposal for CAMARA mandated minimum acceptable JWT token lifetime

Problem description
CAMARA mandate that clients be authenticated using private_key_jwt. The OIDC code specification mandates that the JWT includes an exp claim. Whilst tokens that have expired must be rejected by the authorisation server, creating long lived tokens (to remove the risk of token expiration) itself leads to potential security risks.
This is recognised by RFC 7523, which states:

... the authorization server may reject JWTs with an "exp" claim value that is unreasonably far in the future

However, neither RFC 7523 nor OIDC nor CAMARA give any guidance as to what is considered to be "unreasonably far in the future" , or what should be the minimum token lifetime that must be accepted by all authorisation servers. This could lead to a situation where client implementations generate tokens that are accepted by some authorisation servers, but rejected by others, purely because of the token lifetime.
Possible evolution
A minimum JWT token lifetime that must be accepted by all authorisation servers should be mandated by CAMARA. It is proposed that this minimum lifetime be 300 seconds, which is plenty of time for a token to be generated and then presented to the authorisation server. This requirement could be appended to the existing text on client authentication as follows:

The token expiration time SHOULD be no more than 300 seconds later than the time of issue. Authorization servers MAY reject tokens whose lifetime (measured from the "issued at" time if known, or otherwise from the time of receipt) exceeds this.

Alternative solution
None offered
Additional context
None

If we are considering adding this restriction for the exp claim, or warning that some implementations may have this restriction, it might be convenient to include a similar statement for the jti claim according to RFC 7523:

The authorization server MAY ensure that JWTs are not replayed by maintaining the set of used "jti" values for the length of time for which the JWT would be considered valid based on the applicable "exp" instant.

Please propose some text to https://github.com/camaraproject/IdentityAndConsentManagement/blob/main/documentation/CAMARA-Security-Interoperability.md#client-credentials-flow

Please propose some text to https://github.com/camaraproject/IdentityAndConsentManagement/blob/main/documentation/CAMARA-Security-Interoperability.md#client-credentials-flow

I think you mean https://github.com/camaraproject/IdentityAndConsentManagement/blob/main/documentation/CAMARA-Security-Interoperability.md#client-authentication, right?

Following up from today's meeting:

For now, I propose this change apply only to the JWT used for client authentication (i.e. the client_assertion), and hence will only propose text for client authentication section referenced by @jpengar. If we make progress on the issues discussing signed authentication requests, then restrictions on the lifetimes of these tokens can be agreed in those issues.
If the authorisation server rejects a JWT used for client authentication because its expiration date is "unreasonably far in the future", the appropriate error to return appears to be invalid_client, delivered with a 400 for the  /token endpoint or a 401 for the /bc-authorize endpoint.
@MarkCornall mentioned he would like all authorisation servers to apply the 300 second limit, and reject tokens with a longer lifetime. The issue with trying to standardise this behaviour is that the iat (issued at) field of the JWT is not mandatory, and so the authorisation server can only measure the remaining lifetime from when they receive the token. I see two options here:

Mandate the iat claim for private_key_jwt tokens (this is already mandatory for CIBA signed authentication requests) and thus limit the allowed token lifetime, irrespective of when the token was presented to the authorisation server; or
Limit the allowed remaining lifetime of the token when presented to the authorisation server, irrespective of whether or not the token contains an iat claim

Let me have any further thoughts by end of the week, and I'll propose some text early next week


scripts/west_commands/run_common.py : misleading error message in runners_yaml_path

Describe the bug
west flash returns the following error message when runners.yaml doesn't exist in <application_build_directory>/zephyr/ :

FATAL ERROR: either a pristine build is needed, or board {board} doesn't support west flash/debug (no ZEPHYR_RUNNERS_YAML in CMake cache)

However, the cause of the error is not the absence of ZEPHYR_RUNNERS_YAML in CMake cache, but rather the absence of the runners.yaml file itself.
That is evident from the source code and from the reproducing procedure. Thus, the error message is misleading and should be changed, as it does not accurately describe what causes the error.
To Reproduce
Steps to reproduce the behavior:
Part 1 :  verify that the error isn't actually caused by the absence of  ZEPHYR_RUNNERS_YAML in CMakeCache :

west build -b nucleo_wba55cg -p always samples/basic/blinky (I used nucleo-WBA55CG as an example but this shouldn't be board-specific)
in build/CMakeCache.txt, look for the line starting with ZEPHYR_RUNNERS_YAML (line 580 in my case) and delete it.
west flash --skip-rebuild (use --skip-rebuild to prevent west from regenerating cache. This isn't necessary to reproduce but makes the bug more apparent as ZEPHYR_RUNNERS_YAML will still be unset after flashing).
No error, flashing works as intended

Part 2 : verify that no runners.yaml triggers the error even when ZEPHYR_RUNNERS_YAML is set :

west build -b nucleo_wba55cg -p always samples/basic/blinky (I used nucleo-WBA55CG as an example but this shouldn't be board-specific)
in build/zephyr : rename build/zephyr/runners.yaml to another name. For instance :
cd build/zephyr && mv runners.yaml runners
Verify that ZEPHYR_RUNNERS_YAML is set in build/CMakeCache.txt
in build directory or application directory : west flash

-- west flash: rebuilding
ninja: no work to do.
FATAL ERROR: either a pristine build is needed, or board {board} doesn't support west flash/debug (no ZEPHYR_RUNNERS_YAML in CMake cache)

Expected behavior
When no runners.yaml file is present in the build system, the error message should accurately describe the problem, in line with what actually caused the error to happen in the code (trying to open a file that doesn't exist in this case).
A more accurate error message would be something like :

FATAL ERROR: no runners.yaml found in <build_dir>/zephyr . A pristine build might be needed.

Impact
Annoyance when tweaking with the build system/runners.
Environment :

OS: Ubuntu 22.04.4
Toolchain : Zephyr SDK 0.16.5-1
Zephyr 3.6.99 (commit SHA : b25eb36cb81917d529212c5a5c3d31ed56cd568d)

I don't get it, you are manually altering the cache or removing a file, not really sure what you expect to happen? Don't manually edit the cache or remove the flash runner file?

I'll add some context to clarify the issue : I am tweaking with the build system (with CMake) and adding extra images to be flashed alongside the zephyr image itself.
My problem is not that there is an error and that flashing doesn't work. This is expected.
My problem is that the error message I got points to a missing cache variable when the actual issue is a missing file.
Now, this is my first contribution and maybe the "bug" label is unfit for this issue. If so please let me know. I haven't found a very precise definition outlining what qualifies as a" bug" vs "enhancement" vs "feature" etc. If such a resource exists also please let me know.

Thanks for raising this.
While I do know how annoying it is to be sent off track when debugging, then I have changed into Enhancement as the error message for sure can be improved.
A bug is more when something is broken in the system, which isn't the case here.

Perfect, thank you.


Doesn't work with moz-readability

import Readability from<EMAIL_ADDRESS>import { DOMParser } from "https://deno.land/x/deno_dom/deno-dom-wasm.ts";

    const doc = new DOMParser().parseFromString(body, "text/html");
    const article = new Readability(doc).parse();
    console.log(article);

/// [uncaught application error]: TypeError - Cannot read properties of undefined (reading 'nextElementSibling')

Readbility needs a window.document object. Does deno_dom provide that?

Yes, the document returned by parseFromString is what would be referred to as window.document. Can you share a stack trace please

http://sprunge.us/BnZnYi

$ deno run -A --unstable --no-check ./app.ts
https://intechitec.com/i-underestimated-bitcoin-says-paypal-co-founder/
[uncaught application error]: TypeError - Cannot read properties of undefined (reading 'nextElementSibling')

request: {
  url: "http://localhost:8000/fetch?url=https://intechitec.com/i-underestimated-bitcoin-says-paypal-co-found...",
  method: "GET",
  hasBody: false
}
response: { status: 404, type: undefined, hasBody: false, writable: true }

    at ee._cleanStyles (https://cdn.esm.sh/v57/moz-readability@0.2.1/es2021/moz-readability.js:3:17962)
    at ee._prepArticle (https://cdn.esm.sh/v57/moz-readability@0.2.1/es2021/moz-readability.js:3:6347)
    at ee._grabArticle (https://cdn.esm.sh/v57/moz-readability@0.2.1/es2021/moz-readability.js:3:13706)
    at ee.parse (https://cdn.esm.sh/v57/moz-readability@0.2.1/es2021/moz-readability.js:3:22976)
    at file:///home/ettinger/src/reader/app.ts:17:40
    at async dispatch (https://deno.land/x/oak@v9.0.1/middleware.ts:41:7)
    at async dispatch (https://deno.land/x/oak@v9.0.1/middleware.ts:41:7)
    at async dispatch (https://deno.land/x/oak@v9.0.1/middleware.ts:41:7)
    at async dispatch (https://deno.land/x/oak@v9.0.1/middleware.ts:41:7)
    at async EventTarget.#handleRequest (https://deno.land/x/oak@v9.0.1/application.ts:379:9)

I made a change. http://sprunge.us/6vjTB2

I'm wondering if the html comment on the first line is causing problems with your library.

The bug is caused by the fact that firstElementChild isn't implemented

how do we fix?

@b-fuze Are you going to fix this or can I take it?

Go ahead

I don't see it on deno.land yet

I don't see it on deno.land yet

There hasn't been a new release yet.  You should be able to just pull from the github master like so if you want:
import { DOMParser } from "https://github.com/b-fuze/deno-dom/blob/master/deno-dom-wasm.ts";

Or lock the version of the code to the PR (#69) that fixed your issue to keep the module at the same version forever but not receiving updates:
import { DOMParser } from "https://github.com/b-fuze/deno-dom/blob/a83fc017366bf9e77cdb92640abfb1eea8d263e7/deno-dom-wasm.ts";

z I get this error:
Download https://github.com/b-fuze/deno-dom/blob/master/deno-dom-wasm.ts
error: An unsupported media type was attempted to be imported as a module.
  Specifier: https://github.com/b-fuze/deno-dom/blob/master/deno-dom-wasm.ts
  MediaType: Unknown
    at file:///home/ettinger/src/reader/app.ts:4:27

Looks like still missing something here:
https://www.nytimes.com/live/2021/11/05/business/jobs-report-stock-market-news
[uncaught application error]: TypeError - Cannot read properties of null (reading 'tagName')

@ralyodio please include a stack trace when you encounter an error.

That's it.
ALso I notice your parser never returns with this url:
https://ipinfo.io/ip
and a few others.https://www.bbc.com/news/uk-england-kent-59167648

any luck?

https://github.com/mozilla/readability/issues/716#issuecomment-963530794

@ralyodio I know that some bug in Deno DOM is causing Readability to get stuck in an infinite loop. When I get some time I'll look into it
@chovyprognos thanks

Thanks. Really appreciate it. It’s not something I know where to even begin.
Anthony Ettinger
(408) 656-2473

On Nov 8, 2021, at 5:52 PM, b-fuze @.***> wrote:

@ralyodio I know that some bug in Deno DOM is causing Readability to get stuck in an infinite loop. When I get some time I'll look into it
@chovyprognos thanks
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

@chovyprognos alrighty... 86c7a679a0c05f354acf8b837cf05408e411dedf and 26d3a006a74ef53ae318c0c04868dd7e4ac62965 should fix the problems with moz-readability... They're part of the last Deno DOM release. Let me know if anything else comes up

looks like its working now, thank you!

This url causes moz-readability to hang.
https://www.bloomberg.com/news/articles/2021-11-13/apple-to-pay-30-million-over-security-checks-for-store-workers

This url causes moz-readability to hang.
https://www.bloomberg.com/news/articles/2021-11-13/apple-to-pay-30-million-over-security-checks-for-store-workers

Make sure you have the newest version of deno-dom:
deno cache -r https://deno.land/x/deno_dom/deno-dom-wasm.ts

And then import up-to-date version of Readability:
import { Readability } from "https://esm.sh/@mozilla/readability@0.4.1?no-check";
import { DOMParser } from "https://deno.land/x/deno_dom/deno-dom-wasm.ts";

We weren't able to reproduce the bug you describe.

@ralyodio thanks for reporting the other page with issues... I just pushed a fix for it, please test it with the latest version of Deno DOM.
Use the following command to upgrade Deno DOM to the latest version as 0kku mentioned above:
deno cache -r https://deno.land/x/deno_dom/deno-dom-wasm.ts

Also, please keep all Deno DOM bugs related to Readability in this issue. We've confirmed that Readability itself shouldn't have any issues.

Since I didn't feel confident enough changing deno-dom, I fixed up Readability so that it works with deno-dom:
https://www.npmjs.com/package/readability-denodom
Particularly, the problem with images being lost due to the lacking NamedNodeMap iterator, see #109.

One other issue I just ran into: Readability wants the URL to be part of the Document (via documentURI). However, there's no way set the URI in deno-dom, since it's a read-only property and I can't find a way to set it via a constructor.
@b-fuze is there a way to set Document.documentURI that I'm missing?

@henningko nah, there isn't. You can create an issue for it, it sounds like something that I need to implement a deno-dom only API for.

Extended https://www.npmjs.com/package/readability-denodom to allow for a URL in the constructor as a workaround for now.
Will open an issue to keep it on your radar.


[camera] - Fix bug - app crashes on iOS 9

Enclosed AVCaptureDeviceDiscoverySession in a @availabe check and managed older versions with the older versions with the older api
Fixes flutter/flutter#20708

I signed it!

this won't work as the changes are not handling other iOS 10+ only  like AVCapturePhotoOutput

this won't work as the changes are not handling other iOS 10+ only  like AVCapturePhotoOutput

I'm going through some old PRs to see what we can do with them. First, sorry for dropping this on the floor, we do appreciate your contributions though apparently we don't do a good job of showing it.
For this PR specifically, I'm going to close it because it is lacking tests and all our PRs need to have tests.


added ddm_test_validate.py and logger

DDM validation using simulator data
Description
Modification adds a file ddm_test_validate.py. It addresses the problem of validating the trained ddm using the simulator, and logs two log files: 1) simulator state, action, config logs and 2) ddm state, action , config logs for same actions and configs. The logged states in the two csv files can be directly compared to see the difference and error propagation. No visualization or other changes are added there. The DDM_predictor file is also kept separate.
The goal was to keep test and validation simplified and borrow same sim config as used in the brain training integration using microsoft-bonsai-api. So, functions are written only once and re-used from simulator-bonsai integration template as exists in mba samples. Further work or iteration could include connecting to bonsai-platform. But, test_random_policy function is sufficient for DDM explore experimentation, testing and validation.
Fixes # (issue)
Type of change
Please delete options that are not relevant.

[ ] Added logging facility to log both DDM prediction logs and Simulator logs for same actions and configs.
[ ] Modified episode start to add symetric initial state and config randomization, and a function to accomodate both states/actions/configs with and without prefix.
[ ] Added moab sim folder inside a sim folder to allow using same functions as for platform integration.
[ ] Added policies module from mba as a placeholder to test DDM using benchmark control methods for testing more planned test cases. This can have further development.
[ ] ddm_test_validate.py will need more documentation to be added later.

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[ ] Test A
[ ] Test B

Reviewer Tasks
List here all the tasks the reviewer needs to review:

[ ] Test examples/

Test Configuration:

OS:
Python Version:
Bonsai SDK Version:

Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules

@sdk-bonsai : thanks for the PR! Couple questions:

there are four csv datasets, what are their differences?
this repo is meant to just be the ddm library. Let's remove policies.py and entire sim folder

sim/cartpole - any differences from main branch in microsoft-bonsai-api/Python/samples/cartpole? If not, let's remove and just mention the commit SHA so users can revert if they want to replicate that specific environment.
sim/moab - keep this in your branch in microsoft-bonsai-api

I'm going to commit directly here to move everything from ddm_test_validate to ddm_predictor.

verified with @sdk-bonsai in meeting, will try and document sims in the readme and remove from branch, and remove duplicated csv.

@akzaidi , updated with a new commit.  Can either be a string which is then matched for all columns in the dataset, or a list of strings with exact matches, by default "state"
        augm_cols : Union[str, List[str]], optional
            Exact match of additional columns to use for modeling, such as the actions of the current iteration and any scenario/config parameters, by default ["action_command"]
        output_col : Union[str, List[str]], optional
            output columns of the dynamical system. Can either be a string which is then matched for any columns or a list of exact matches, by default "state"
        iteration_order : int, optional
            in the order of the raw dataset, what is the lag between iteration t and iteration t+1, by default -1
        max_rows : Union[int, None], optional
            max rows to read for a large dataset, by default None

        Returns
        -------
        Tuple[np.array, np.array]
            Features and labels for modeling

        Raises
        ------
        ValueError
            Data not found
        """

        csv_reader = CsvReader()
        if not os.path.exists(dataset_path):
>           raise ValueError(f"No data found at {dataset_path}")
E           ValueError: No data found at csv_data/off_names.csv

base.py:77: ValueError

is there a reason that file was renamed? has that file changed entirely?


Question about baseline comparisons in Tables 1 and 3

I have a quick question regarding the comparison to Moreno-Noguer [27].
If I understand correctly, your Protocol #2, is all actions and viewpoints, (S1, S5, S6, S7, S8) for training, and (S9, S11) for test. You compare to Protocol #3 from [27] in Table 3, but in their paper is this not all actions and viewpoints, (S1, S5, S6, S7, S8 and S9) for training and 1/64 frontal view S11 for test.
Also, I was wondering where your got the GT/CPM result for [27] in your Table 1. I couldn't seem to find it in theirs.
Thanks for your help!

Hi @macaodha

If I understand correctly, your Protocol #2, is all actions and viewpoints, (S1, S5, S6, S7, S8) for training, and (S9, S11) for test. You compare to Protocol #3 from [27] in Table 3, but in their paper is this not all actions and viewpoints, (S1, S5, S6, S7, S8 and S9) for training and 1/64 frontal view S11 for test.

There is large variety in the protocols used in previous work, either in

the number of joints used (accounted for in our paper)
the amount of training data (eg Moreno-Noguer CVPR'17, Chen and Ramanan CVPR'17)
subsampling at test time (eg Moreno-Noguer CVPR'17,  Bogo et al ECCV'16) -- presumably because those approaches take a long long time to run
scale knowledge at test time (arxiv v1 of Pavlakos et al, CVPR'17 -- this was corrected in arxiv v2 / CVPR'17 camera-ready)

As you might imagine, it is a bit hard to set up all these different experiments and compare against the particular tweaks of other researchers. We instead decided to go for the most straightforward protocol, using both 14 and 17 joints, less data than some previous work, not subsampling, and avoiding any scale knowledge under protocol 1. This is a very bare-bones setup that puts us at a disadvantage, but turned out perform quite well nonetheless. I imagine that you re-ran the experiments under different setups our advantage would increase.

Also, I was wondering where your got the GT/CPM result for [27] in your Table 1. I couldn't seem to find it in theirs.

Good catch! We got it from the arxiv version: https://arxiv.org/pdf/1611.09010.pdf. We forgot to update it in the final version.

Yes, it seems like there are a lot of different ways to compare the results. Thanks for confirming.

Can I confirm one more thing, Im trying to understand the SH experiments.
In your Table 1 am I correct in saying that GT/SH means: "train on ground truth 2D, test on SH 2D"?
As opposed to Table 3, where Ours (SH detections) (MA) means: "train on SH 2D and test on SH 2D".

In your Table 1 am I correct in saying that GT/SH means: "train on ground truth 2D, test on SH 2D"

Yes, the caption of Table 1 says: "(Bottom) Training on ground truth and testing on the output of a 2d detector." What is the source of confusion here?

As opposed to Table 3, where Ours (SH detections) (MA) means: "train on SH 2D and test on SH 2D".

Correct. This line can be directly reproduced from the repo. I hope that is clear as well.

Makes sense, thanks!


Test data: dm_metabolic

Background Information
We need to create an {admiralmetabolic} version of pharmaversesdtm::dm which contains a small subset (5?) of the patients in the full dm dataset, so that we can use the same 5 patients across all of our new test data.
This dataset will eventually be moved to {pharmaversesdtm}, but for the moment we will create it within {admiralmetabolic}.
Check out pharmaversesdtm repo for inspiration.
Definition of Done

[ ] top-level data_raw folder created to house dm_metabolic, i.e. the script that generates our metabolic version of dm
[ ] dm_metabolic program set up. program should read in dm, keep only a few (5?) patients.
[ ] Review all dm_metabolic variables to see if any changes are required.
[ ] New dm_metabolic dataset documented in data.R

I am wondering if it would make sense to change the pre-defined treatment / ARM? The DM example from pharmaversesdtm have three treatment arms (Xanomeline Low, Xanomeline High, and Placebo). If we change, I would argue that it should be something non-descriptive (i.e. insulin, or weight loss drug A).
Furthermore, I am contemplating on weather to include a screen failure participant in the dataset. I notice that most Admiral extension don't have any screen failures, but would it make sense for the vignettes to have something to sort away? Or do we consider this to be covered in admiral base?

I am wondering if it would make sense to change the pre-defined treatment / ARM? The DM example from pharmaversesdtm have three treatment arms (Xanomeline Low, Xanomeline High, and Placebo). If we change, I would argue that it should be something non-descriptive (i.e. insulin, or weight loss drug A).
Furthermore, I am contemplating on weather to include a screen failure participant in the dataset. I notice that most Admiral extension don't have any screen failures, but would it make sense for the vignettes to have something to sort away? Or do we consider this to be covered in admiral base?

@AndersAskeland Agree that changing the arm makes sense, happy to go with one of your suggestions.
In terms of screen failure, generally we don't include them, so I would continue down that path unless there was a reason specific to this TA as to why we might want to include that type of record.


BUG: Seaborn.objects Plot.add() incompatible with Modin

Modin version checks

[X] I have checked that this issue has not already been reported.

[X] I have confirmed this bug exists on the latest released version of Modin.

[X] I have confirmed this bug exists on the main branch of Modin. (In order to do this you can follow this guide.)

Reproducible Example
import modin.pandas as pd
import seaborn as sns
import seaborn.objects as so

ex_tips_df = pd.DataFrame({'total_bill': [16.99, 10.34, 21.01], 'tip': [1.01, 1.66, 3.50], 'sex': ['Female', 'Male', 'Male'], 'smoker': ['No', 'No', 'Yes'], 'day': ['Sun', 'Sun', 'Sun'], 'time': ['Dinner', 'Dinner', 'Dinner'], 'size': [2, 3, 3]})

so.Plot(ex_tips_df, "total_bill", "tip").add(so.Dot())

Issue Description
The add() function results in KeyError: 'x'and works in Pandas. However, so.Plot(ex_tips_df["total_bill"], ex_tips_df["tip"]).add(so.Dot()) runs successfully.
Expected Behavior

Error Logs

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File ~/Desktop/ponder-integ/lib/python3.9/site-packages/pandas/core/indexes/base.py:3803, in Index.get_loc(self, key, method, tolerance)
   3802 try:
-> 3803     return self._engine.get_loc(casted_key)
   3804 except KeyError as err:

File ~/Desktop/ponder-integ/lib/python3.9/site-packages/pandas/_libs/index.pyx:138, in pandas._libs.index.IndexEngine.get_loc()

File ~/Desktop/ponder-integ/lib/python3.9/site-packages/pandas/_libs/index.pyx:165, in pandas._libs.index.IndexEngine.get_loc()

File pandas/_libs/hashtable_class_helper.pxi:5745, in pandas._libs.hashtable.PyObjectHashTable.get_item()

File pandas/_libs/hashtable_class_helper.pxi:5753, in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'x'

The above exception was the direct cause of the following exception:

KeyError                                  Traceback (most recent call last)
File ~/Desktop/ponder-integ/lib/python3.9/site-packages/IPython/core/formatters.py:342, in BaseFormatter.__call__(self, obj)
    340     method = get_real_method(obj, self.print_method)
    341     if method is not None:
--> 342         return method()
    343     return None
    344 else:

File ~/Desktop/ponder-integ/lib/python3.9/site-packages/seaborn/_core/plot.py:278, in Plot._repr_png_(self)
    276 def _repr_png_(self) -> tuple[bytes, dict[str, float]]:
--> 278     return self.plot()._repr_png_()

File ~/Desktop/ponder-integ/lib/python3.9/site-packages/seaborn/_core/plot.py:820, in Plot.plot(self, pyplot)
    816 """
    817 Compile the plot spec and return the Plotter object.
    818 """
    819 with theme_context(self._theme_with_defaults()):
--> 820     return self._plot(pyplot)

File ~/Desktop/ponder-integ/lib/python3.9/site-packages/seaborn/_core/plot.py:850, in Plot._plot(self, pyplot)
    848 # Process the data for each layer and add matplotlib artists
    849 for layer in layers:
--> 850     plotter._plot_layer(self, layer)
    852 # Add various figure decorations
    853 plotter._make_legend(self)

File ~/Desktop/ponder-integ/lib/python3.9/site-packages/seaborn/_core/plot.py:1350, in Plotter._plot_layer(self, p, layer)
   1347     grouping_vars = mark._grouping_props + default_grouping_vars
   1348     split_generator = self._setup_split_generator(grouping_vars, df, subplots)
-> 1350     mark._plot(split_generator, scales, orient)
   1352 # TODO is this the right place for this?
   1353 for view in self._subplots:

File ~/Desktop/ponder-integ/lib/python3.9/site-packages/seaborn/_marks/dot.py:70, in DotBase._plot(self, split_gen, scales, orient)
     62 def _plot(self, split_gen, scales, orient):
     63
     64     # TODO Not backcompat with allowed (but nonfunctional) univariate plots
     65     # (That should be solved upstream by defaulting to "" for unset x/y?)
     66     # (Be mindful of xmin/xmax, etc!)
     68     for _, data, ax in split_gen():
---> 70         offsets = np.column_stack([data["x"], data["y"]])
     71         data = self._resolve_properties(data, scales)
     73         points = mpl.collections.PathCollection(
     74             offsets=offsets,
     75             paths=data["path"],
   (...)
     83             **self.artist_kws,
     84         )

File ~/Desktop/ponder-integ/lib/python3.9/site-packages/pandas/core/frame.py:3805, in DataFrame.__getitem__(self, key)
   3803 if self.columns.nlevels > 1:
   3804     return self._getitem_multilevel(key)
-> 3805 indexer = self.columns.get_loc(key)
   3806 if is_integer(indexer):
   3807     indexer = [indexer]

File ~/Desktop/ponder-integ/lib/python3.9/site-packages/pandas/core/indexes/base.py:3805, in Index.get_loc(self, key, method, tolerance)
   3803     return self._engine.get_loc(casted_key)
   3804 except KeyError as err:
-> 3805     raise KeyError(key) from err
   3806 except TypeError:
   3807     # If we have a listlike key, _check_indexing_error will raise
   3808     #  InvalidIndexError.  Is lux suitable to already work with Modin?

@YarShev we'd like to support compatibility, but there's no such effort planned in the near term. @labanyamukhopadhyay @dorisjlee do you know if lux is already compatible?

This specific issue is resolved when installing seaborn from its master branch due to recent support of dataframe interchange protocol
A more common example so.Plot(ex_tips_df, x= 'total_bill', y= 'tip', color='smoker').add(so.Dot()) also now works thanks to #6523

This specific issue is resolved when installing seaborn from its master branch due to recent support of https://github.com/mwaskom/seaborn/pull/3369

@labanyamukhopadhyay that's great! thank you for checking.


Add sync batchnorm

add sync batchnorm(implementation from this repo) for the multi-gpu training, also apply linear scale rule from https://arxiv.org/abs/1706.02677

Please do some performance tests on reid datasets, such as market1501 rank1 etc. and speed test. We want to get the same result for rank1 and mAP for syncBN when batch_size=64.
PS: Please put the sync_bn in layers folder to make our code base structured.

Sorry for not replying for so long, I have some other thing to work on..
Here is the mAP and CMC result for syncbn with batchsize 64, I was using two GTX 1080Ti
2019-05-09 16:12:22,187 reid_baseline.train INFO: Epoch 120 done. Time per batch: 83.640[s] Speed: 140.8[samples/s]
2019-05-09 16:12:22,187 reid_baseline.train INFO: ----------
The test feature is normalized
2019-05-09 16:14:43,836 reid_baseline.train INFO: Validation Results - Epoch: 120
2019-05-09 16:14:43,836 reid_baseline.train INFO: mAP: 85.9%
2019-05-09 16:14:43,836 reid_baseline.train INFO: CMC curve, Rank-1  :94.1%
2019-05-09 16:14:43,836 reid_baseline.train INFO: CMC curve, Rank-5  :98.2%
2019-05-09 16:14:43,837 reid_baseline.train INFO: CMC curve, Rank-10 :99.0%

I was using the config file here


Improve RemoveDependenciesFromEntryIfMissing() when there is a lot of input files.

Steps to reproduce
Create a C++ Project with 300+ files.  For simplicity, each file is a clones, named fromo file1.cpp to file file300.cpp.  MSBuild project can use
#include <windows.h>
#include <__msvc_all_public_headers.hpp>
int somefunction() { return 0; }

msbuild.exe /t:ClCompile /p:UseMultiToolTask=true
I added some telemetry:
PostExecuteTool Part 1: 414ms
PostExecuteTool Part 2: 493ms
PostExecuteTool Part 2.1: 531ms
PostExecuteTool Part 2.2: 4800ms<!-- Jump here is only calling RemoveDependenciesFromEntryIfMissing()
PostExecuteTool Part 3: 4967ms
PostExecuteTool Total: 5019ms
Expected behavior
RemoveDependenciesFromEntryIfMissing() should take less time.  100ms or less.
Actual behavior
RemoveDependenciesFromEntryIfMissing() is took 4 seconds.
Notes:
CanonicalTrackedInputFiles.RemoveDependenciesFromEntryIfMissing() has a N^2 loop that is calling FileExists().  Adding a simple cache would speed up this process timing a lot.
OS info:
Windows 10, running under VS developer command prompt.

So current status is that it definitely won't be making it into P2, so apologies for that. But this is now the #1 item on my plate. After getting a more detailed explanation on this area of code  and being able to debug it more (thanks Rainer), you'll hopefully be seeing more activity on my end.

Notes as I go: A lot of the dependency files are checked whether or not they exist hundreds of times. Using a simple HashSet to store all of the files that were detected in the file check, I was able to bring down the time in the worst case from 860msec to 28msec (using a stopwatch capturing the outer for-loop and logging that to a file). But as you can see, it's a significant improvement.
Question: Let's say a file was not found in first iteration of this function during a build, is it worth continually checking for that file further along in the build? My current implementation is a straightforward "have we seen that the file exists already? if so, never look for it again", do I need to consider scenarios where perhaps the file no longer exists, or the file didn't exist originally, but shows up at a later point?
//cc: @rainersigwald

It is probably safe to the assume that HashSet is good for the lifetime of the class.  A smarter system could share with a global system and only need to check the "write" logs as the write is the only difference.
Could also take a look at the Constructor?  I stopwatched it taking 400ms (while not as much as RemoveDependenciesFromEntryIfMissing()), I expected it to take only 40-50ms.  I refactored disk access from the timer and it still took the same time, thus issue is somewhere else.
Thanks.


Fixed #700 - Shorten configuration prefixes to simply 'fluo'

+1
The changes you made look good.  I ran the following command to look for anything that may have been missed, but didn't see anything.
$ find . -type f | grep -v '/.git' | grep -v '.png' | xargs grep org.apache.fluo | grep -v 'package org.apache.fluo' | egrep -v 'import.*org.apache.fluo' | grep -v '<groupId>'


Conversion as LinearOperator from scipy

The LinearOperator interface ^1 is proposed by Scipy for describing linearOperator which are too big to be represented as matrices. MRI-NUFFT Operators are linear operators, so we could support quite easily.
LinearOperator are also used in PyLops ^2 (and PyProximal ^3).
In term of implementation I think a wrapper class, and a custom method (FourierBaseOperator.make_linear_operator) should do the trick.

hi @paquiteau, what up ?
i want try finish this issue, ok ?


Fix Clear Formatting (Fixes #4188)

Clear formatting respects the selection, also converts Headings and Quotes down to Paragraphs, but doesn't convert Code Blocks or any list types, matching the behaviour of other popular text editors.
Before:
https://user-images.githubusercontent.com/7893468/227743734-7101ae98-10a0-4af2-baca-ccfaf612f58d.mp4
After:
https://user-images.githubusercontent.com/7893468/227743740-8ab0ae0b-2d51-47a1-888b-d30c9f1c8d97.mp4

@fantactuka could you help me out how to replace a HeadingNode with a TextNode. I tried
node.replace(new TextNode('abc'), false);

but I get an error that I can only add an Element or a Decorator node to the Root node. What would be the right syntax here?

@ivailop7 you need to wrap the TextNode in a ParagraphNode (or some other element). They can’t exist as direct children of the root.

@ivailop7 you need to wrap the TextNode in a ParagraphNode (or some other element). They can’t exist as direct children of the root.

Understood. Thank you @acywatson .

Updated. The CI failing post the Excalidraw update I believe is because of the parallelization of the tests, thus the sporadic success/failure. I believe it has to be moved from package.json inside the playground to the root one, but needs proper investigation, this is just a theory. Might dig into that one soon.

@zurfyx are we good with this one?

@zurfyx are we good with this one?

Can you check on the test failures please?

Ops, my fault. What's the right order of npm commands to repro these tests locally? When I directly run them, all tests fail, while Unit and Integration pass.

You probably want to run debug-test-e2e-legacy-chromium and the other similar variants listed in the package.json. The CI variants work with a different port if that's what you're referring to

@zurfyx All set here.

@acywatson @zurfyx I think, this one is ready to merge.

Thank you Ivaylo, sorry for the delay, I completely missed the updated version


WIP chore: update to keycloak v23.0.0

Needs testing. In theory v1.7.1 should still be compatible to Keycloak v23 without any changes.

Obsolete by #64


libxc: 5.2.2 -> 5.2.3

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/libxc/versions.
meta.description for libxc is: "Library of exchange-correlation functionals for density-functional theory"
meta.homepage for libxc is: "https://www.tddft.org/programs/Libxc/"
meta.changelog for libxc is: ""
Updates performed

Version update

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 5.2.3 with grep in /nix/store/h8qvc3qjaxgpkxyyii1k6v8cvbjqhiba-libxc-5.2.3
found 5.2.3 in filename of file in /nix/store/h8qvc3qjaxgpkxyyii1k6v8cvbjqhiba-libxc-5.2.3

Rebuild report (if merged into master) (click to expand)

6 total rebuild path(s)

6 package rebuild(s)

First fifty rebuilds by attrpath
cp2k
libxc
octopus
openmolcas
python310Packages.pyscf
python39Packages.pyscf

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/h8qvc3qjaxgpkxyyii1k6v8cvbjqhiba-libxc-5.2.3 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A libxc https://github.com/r-ryantm/nixpkgs/archive/725b8e8f3c6fff8337dd9c66d08ee512c9ca19b9.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/h8qvc3qjaxgpkxyyii1k6v8cvbjqhiba-libxc-5.2.3
ls -la /nix/store/h8qvc3qjaxgpkxyyii1k6v8cvbjqhiba-libxc-5.2.3/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
nixpkgs-review took longer than 45m and timed out

Maintainer pings
cc @markuskowa for testing.

@GrahamcOfBorg build octopus cp2k


http://httparchive.org/ not working

It shows 502 Bad Gateway.

Not sure what broke around 2am ET last night but alarms started firing that it was unreachable.  I just restarted apache and it looks like things are back in order.


added redux and redux toolkit. integrated redux with a simple store. a…

added state managment #4

The changes all look good to me! If you can just fix the conflicts in package-lock.json then this should be ready to merge.

just redid it in another branch rather than deal with package lock merge conflict


CMake changes to generate code

Description:
The projects, which use Service Interface document file (files with extension .siml) contain generated codes, which are manually integrated in the projects by CMakeLists.txt files. Since these files are generated and the sources are kind of fixed, it makes sense to modify the CMakeLists.txt file to generate and compile files during build.
Required:

Modify the CMake to generate and build static libraries from Service Interface document files. Something like addServiceInterface(myproject, myproject/HelloWordl.siml), where the myproject is the name of the project and static library and myproject/HelloWordl.siml is the relative path to the Service Interface document.
There should be possibility to link the static library with the target binary, something like linkProject(target, myproject).
Modify the CMake build to add an option that will only generate files and create MSVC project files.

Note:
The description might be not really clear. Please start the discussion here and ask questions to @aregtech before starting modifying.

@biwiki , can you please help? Of course if you have time. I'm a little stuck making changes in Makefile.
The situation is following: In the branch 175-cmake-changes-to-generate-code I've made changes that code generator generates files in the ./product/generate directory and automatically includes in the build. To do this, I've defined function addServiceInterfaceEx for cmake build. It triggers code generator, creates list of sources and either creates new static library (if does not exist) or adds the sources to the existing target (static library). The examples 10 - 24 contain call of addServiceInterface (simplified version of addServiceInterfaceEx). For example, this call. Or here, where the first call of addServcieInterface generates codes of PowerManager.siml and sets static library 16_generated and the second call of addServiceInterface generates codes of TrafficController.siml and adds generated sources to the existing target 16_generated static library.
Now the similar thing I'd like to have in Makefile, where it would run code generator and create static library. I tried to implement, but my knowledge and experience is not enough. I would appreciate very much help. If you feel that it might be too complicated, please let me know. Maybe I temporary disable make build.
Many thanks in advance and best regards.

@aregtech sure, I will look into this when I can have time

@biwiki , sounds great!
Just for your information. Right now I have disabled (removed) Makefile", but created [a tag](https://github.com/aregtech/areg-sdk/tags) on master branch.  This is the latest version to compile with make`. Let me know if you need any help from me.

Ohh it can be a right thing, maintaining 2 different build systems within the same sourcebase can make things complicated

Great to hear that you like the Makefile, but I would say that it is time to deprecate that old build system
Code generation with Java can also be a problem. I am not sure if this is documented somewhere but I can understand that users need to install correct Java version a long with other dependencies in order to use the tool
I am myself hitting this error with default Java on Ubuntu 22..04
==============================================================================
Generating class defined in HelloService.siml to generated/src ....
==============================================================================
Error: LinkageError occurred while loading main class com.aregtech.CMFMain
        java.lang.UnsupportedClassVersionError: com/aregtech/CMFMain has been compiled by a more recent version of the Java Runtime (class file version 58.0), this version of the Java Runtime only recognizes class file versions up to 55.0

IMO, If this would not take too much efforts, then the code generation tool should be converted (and open) to Python. In embedded domain, there should be more chances that users have knowledge on Python and would be able to resolve dependency issues if there is any with it.

@biwiki , thank you for your feedback. I appreciate that.

make build: OK. I don't mind to deprecate build with make. Indeed it is not easy to support.
Java: good idea. What do you think, is it better to create new repository for code generator or include sources in areg-sdk/tools and build JAR if the codegen.jar does not exist? BTW, which version of Java do you have? In principle, I don't use modern features of Java classes and could compile with older versions.
Python: well, in principle, it is a good idea, but my knowledge of Python are not so good to write a code generator :)

This is Java version on my machine
$ java --version
openjdk 11.0.21 2023-10-17
OpenJDK Runtime Environment (build 11.0.21+9-post-Ubuntu-0ubuntu122.04)
OpenJDK 64-Bit Server VM (build 11.0.21+9-post-Ubuntu-0ubuntu122.04, mixed mode, sharing)

Since we are embedding code generator into the build flow, this can stop users from compiling the library.
I am not against using Java :-) But due to this I just dont know how to proceed
I personally have seen quite a lot of code generators written in Python, and we didn't hit the same issue as this.

I compiled jar with jdk21.
I think, it is not a problem if compile with the lower version.
What is your suggestion, to switch to Python? It would be additional effort

yeah it can take huge effort to convert to Python, sorry but it is upto you to decide

@biwiki,
I've made new build of code generator, compiled with jre1.8 (jre8)
openjdk version "1.8.0_392"
OpenJDK Runtime Environment (build 1.8.0_392-8u392-ga-1~22.04-b08)
OpenJDK 64-Bit Server VM (build 25.392-b08, mixed mode)

The codes are in the branch feature/175-cmake-changes-to-generate-code
Tested with jre8, jre11 and jre21 (jre8 and jre11 under Ubuntu). It worked for me.
Can you please check and confirm whether it works for you as well?
Please pay attention that the Makefile does not exist anymore.

There are not submodules anymore (fetching googletest is compile with tests)
The codes are generated during configuration (created if file does not exist or updated).

Hi @biwiki
I also let the action workflow to run with JRE 11, it generates and compiles fine. Have a look the logs here.  Check the runs ending with -test like this one and here is the JRE setup. Theoretically, you also should not have a problem.

thanks, yes I can run the code generator on my Linux box now

Great!
Thank you for your feedback.
Then I'll merge changes to master.


Fix nightly

The nightly build is failing because librespot overwrites the "core" crate. This PR fixes this by removing this alias and just using the full name "librespot_core".
It also includes the changes in https://github.com/librespot-org/librespot/pull/361 and should probably not be merged before that one.

Can you just do the nightly fix here and then rebase #361 on develop? It'll be a bit simpler to review it then.

Nighly build should be fixed with #371. Closing this for now!


Kubernetes Workload Scaler: ignore terminated pods

The kubernetes-workload scaler is great, but I was surprised to discover that it counts terminated pods. While it's useful to have Shutdown or Completed pods stay around to aid for example troubleshooting I fail to come up with a use case where scaling should count them together with running. This PR excludes the two obvious terminated states from the metric value.
Checklist

[x] Commits are signed with Developer Certificate of Origin (DCO - learn more)
[x] Tests have been added
[x] A PR is opened to update our Helm chart (repo) (if applicable, ie. when deployment manifests are modified)
[x] A PR is opened to update the documentation: https://github.com/kedacore/keda-docs/pull/605
[x] Changelog has been updated

/run-e2e kubernetes-workload.test*


Header UI changes and Social Sharing of the App

Issue Number
fixes #178  , #258
Describe the changes you've made
I added a design for social sharing of the app and also changed the UI of the header for all pages.
Checklist

[x] My code follows the code style of this project.
[] My change requires a change to the documentation.
[] I have updated the documentation accordingly.
[x] All new and existing tests passed.
[x] The title of my pull request is a short description of the requested changes.

No description provided.

I think the social sharing has to work as well @poojadas07 i.e add functionality as well, you can refer capacitor docs for it.

I think the social sharing has to work as well @poojadas07 i.e add functionality as well, you can refer capacitor docs for it.

Yeahh It is working.

I think the social sharing has to work as well @poojadas07 i.e add functionality as well, you can refer capacitor docs for it.

Yeahh It is working.

Okay, I'm not sure but I tried it on my phone and it didn't redirect me to whatsapp or linkedin. Is this compatible with ios as well?

@Abhishek-kumar09 Please check it once.

Yeah the sharing option is working perfectly in android devices.
@sakshi-choudhary Do you have iOS, is it not opening whatsapp, and others?

Yeah the sharing option is working perfectly in android devices.
@sakshi-choudhary Do you have iOS, is it not opening whatsapp, and others?

yes, I have ios, so for me whatsapp and Linkedin wasn't working. I'll try again and let know

Yeah the sharing option is working perfectly in android devices.
@sakshi-choudhary Do you have iOS, is it not opening whatsapp, and others?

yes, I have ios, so for me whatsapp and Linkedin wasn't working. I'll try again and let know

this seems to be working fine now.


Bug: Fix Allowance Holdings API order by

Actual Results
The allowance holdings api is ordered incorrectly
Currently the Allowance holding API is ordered by ACCOUNT_NUMER > PRG_CODE > VINTAGE_YEAR > START_BLOCK

The allowance holdings API should be ordered by PRG_CODE > ACCOUNT_NUMER > VINTAGE_YEAR > START_BLOCK:
ACCOUNT_NUMBER
ACCOUNT_NAME
ORISPL_CODE
PRG_CODE
VINTAGE_YEAR
TOTAL_BLOCK
START_BLOCK
END_BLOCK
STATE
EPA_REGION (2 digit number)
OWN_DISPLAY
ACCOUNT_TYPE

https://epacamd.testquality.com defect D65.


fix bugs in get started and API documents

Signed-off-by: jie.hou<EMAIL_ADDRESS>

Codecov Report

Merging #904 (5473f5f) into main (cfed234) will decrease coverage by 0.00%.
The diff coverage is 66.66%.

:exclamation: Current head 5473f5f differs from pull request most recent head df07800. Consider uploading reports for the commit df07800 to get more accurate results

@@            Coverage Diff             @@
##             main     #904      +/-   ##
==========================================
- Coverage   69.95%   69.94%   -0.01%
==========================================
  Files         200      200
  Lines        8729     8728       -1
  Branches     1431     1431
==========================================
- Hits         6106     6105       -1
  Misses       2205     2205
  Partials      418      418

Impacted Files
Coverage Δ

towhee/functional/mixins/dispatcher.py
66.66% <ø> (ø)

towhee/functional/mixins/entity_mixin.py
86.66% <ø> (ø)

towhee/functional/mixins/parallel.py
96.19% <ø> (ø)

towhee/functional/mixins/data_source.py
50.58% <25.00%> (ø)

towhee/hub/builtin/operators/computer_vision.py
72.46% <60.00%> (+0.82%)
:arrow_up:

towhee/__init__.py
82.35% <100.00%> (+0.21%)
:arrow_up:

towhee/functional/data_collection.py
92.81% <100.00%> (-0.16%)
:arrow_down:

towhee/operator/__init__.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 07cebee...df07800. Read the comment docs.

/lgtm
/approve


Update documentation.

This ticket is a place for me to keep track of all the things that need to be documented.
new API
new granular notifications
representations

I reviewed all of the documentation and opened a couple of pull requests (PR #25 and PR #26) with minor fixes. I noticed, though, that all objects have only very superficial listing of methods and attributes, and nothing more descriptive of their usage and functionalities. The concept pages, though, are already looking good :-)


how to use server timestamps

as in the firstore docs : https://firebase.google.com/docs/firestore/manage-data/add-data
how can we use server timestamps
`created_at: firebase.firestore.FieldValue.serverTimestamp()`

or
regions: firebase.firestore.FieldValue.arrayRemove("east_coast")

Hi @jawainc
You can add timestamps or regions when you add the data to firestore.
for example:
addData () {
   this.$firestore.myData.add({
        name: 'sample',
        created_at: firebase.firestore.FieldValue.serverTimestamp()
   })
}

have to import import { firebase } from '@firebase/app'; in every component where I want to use server timestamps.
following code example here: https://alligator.io/vuejs/vue-cloud-firestore/

it's not recommended to import the firebase and firestore in every single component, that will make your app bundle bigger.
you can use vuex instead and import one state of firebase and firestore, then you can call them through this.$store.state in your components.

Thanks for the idea, I did it from different approach
my firebase.js
import config from '@/resource/config';
import { firebase } from '@firebase/app';
import '@firebase/firestore';

const firebaseApp = firebase.initializeApp(config.fireBaseConfig);
let db = firebaseApp.firestore();
const settings = {timestampsInSnapshots: true};
db.settings(settings);

export {db, firebase};

exported firebase and db
now in vue component
import { db, firebase } from './firebase';
and it works


latency of unirest.get()

Hallo Rabun,
I chked the latency of this function for making http request. It took like 1.5s in average to get 28 candles, I feel this took too long time. You got any better ideas?
My network was fast enough(1+ Mb/s downstream), please do not take that into consideration.
Cheers,

Yes, there is such a problem. I haven't tested the performance of the services.You can try C# api and compare the performance https://github.com/rabun/oanda-rest-cs
Did you check how long execute primary oanda rest service?

can you do me a favor to test C#? I guess you had the test code somehow, won't take big effort.....thx

I am not sure that it would be correct to compare a C# implementation to a java implementation.  With a couple of google searches it appears that unirest itself is just slow.  I am sure if one was to swap out unirest for another API client the speed would improve.
This is a really awesome project. I would close this issue since it is not specific to this project, but relates to unirest itself.


Add link to Colloquy client for macOS

Add Colloquy macOS client

@unixorn can you change it to http://colloquy.info? They seem to be having problems with their cert at the moment.

That was quick!


Template error: resource BookingTable does not support attribute type StreamArn in Fn::GetAtt

Describe the bug
When running amplify push api I get this error which is utterly un useful. On production version, i always get this error but will then run again and it always works, but on our dev version it fails with this every-time.
Amplify CLI Version
4.40.1
To Reproduce
Run - amplify push ap
Expected behavior
It should update the amplify API
Desktop (please complete the following information):

OS: Mac Big Sur
Node Version: v14.0.0

My Model:
type BookingTable @model(subscriptions: null)
    @key(name: "entityBookingTables", fields: ["entityId"])
    @key(name: "entityBookingTablesByMaxPeople", fields: ["entityId", "maxPeople"], queryField: "listEntityBookingTablesByMaxPeople")
    @key(name: "entityBookingTablesByCapacity", fields: ["entityId", "minPeople"], queryField: "listEntityBookingTablesByCapacity")
    @auth(rules: [
    { allow: owner },
    { allow: groups, groups: ["admin"] },
    { allow: groups, groups: ["users"] },
    { allow: public, provider: iam, operations: [read] }
]) {
    id: ID!
    tableNumber: Int
    capacity: Int
    area: String
    entityId: ID!
    ttl: Float
    maxPeople: Int
    minPeople: Int
    bookings: [Booking] @connection(keyName: "bookingTableKey", fields: ["id"])
    availability: [TableAvailability]
    createdAt: Float
    updatedAt: Float
}

Hey @oliverandersencox :wave: thanks for raising this! For more context, what is the error you are experiencing?

Hey @oliverandersencox I'm unable to recreate this issue with the latest version of the CLI. Was this issue appearing before the auth resource was fully created and pushed? If possible, are you still experiencing this issue, and may you try with the latest version of the CLI?

Closing this , Is the issue still presists , Feel free to comment and we will reopen this again

I had the same problem.  In my full schema, it had both type @model definitions for RateSheet and RateSheetTable.  When I changed the second table to RatesSheetTable2, it worked.  Maybe there is some code gen happening that creates RateSheetTable from the first RateSheet type?


Test src/osvr/PluginHost/BinaryLocation.cpp Android implementation

Currently just using the Linux implementation.

Appears to work.


git pull,first 401,and then 200

run git pull in client:
zdh@CSZX:/doc/xx> git pull
Already up-to-date.
zdh@CSZX:/doc/xx>

gitea server log will show two, first is 401, and ok, why ??????
[Macaron] 2019-07-10 17:01:38: Started GET /dev/XData/info/refs?service=git-upload-pack for <IP_ADDRESS>
[Macaron] 2019-07-10 17:01:38: Completed GET /dev/XData/info/refs?service=git-upload-pack 401 Unauthorized in 629.013µs
[Macaron] 2019-07-10 17:01:38: Started GET /dev/XData/info/refs?service=git-upload-pack for <IP_ADDRESS>
[Macaron] 2019-07-10 17:01:38: Completed GET /dev/XData/info/refs?service=git-upload-pack 200 OK in **88.473014ms**

can set to show one ?

If I'm not mistaken this is part of the HTTP authentication definition (https://en.wikipedia.org/wiki/Basic_access_authentication).
The server first responds with a 401 to notify the client that it needs to send its credentials. The client then sends a second request with its credentials and receives a 200.

tks


Update tailwind.py

Was returning one door less than actually present. Fixed

Oops - thanks for that.  I must not have checked what my device was telling me properly; I'd assumed that the door_num value provided by the API was the number of doors that you've configured, but it actually just seems to always be 3, which is the maximum number of doors supported by the device.  I don't write a lot of Python, so I guess I thought the slice operator was taking a length rather than an (exclusive) end index when I saw that I was getting one more door than expected (my garage has two doors).