Openverse deployments log

Start Date
ETA
Project Lead
Actual Ship Date

YYYY-MM-DD
YYYY-MM-DD
TBD
TBD

Description

Openverse deployments are handled via GitHub workflows in the WordPress/openverse-infrastructure repository. While these may not stay in a private repository forever, relying on the actions log as the "deployment log" leaves us without features that would be helpful for maintainers and for Openverse users. Specifically:

A clear distinction between manual deployments, release deployments, rollbacks, and re-deployments of the same version
Links to changelogs for release deployments
Deployment timing metrics

This project should explore the best way to record this information for each deployment and present it for maintainers and users alike. Suggested ideas to explore are:

A service that deduces the above information based on the workflow logs
A service that receives a ping each time the deployment workflows finish (whether successfully or failed) and stores the metadata described

Other ideas should also be explored if they are thought of.
 Folks with access to the infrastructure itself do have a usable deployment log in at least two places and the changelogs on the documentation site give a picture of the changes to production.


fix(pipeline-modal): fix table row width and content for pipeline modal

Fixes

https://issues.redhat.com/browse/KFLUXBUGS-1127

Description

Fix the way content inside the table behaved by adding proper modifier.
Added a truncate option for image url.

Type of change

[x] Bugfix

Screen shots / Gifs for design review

Browser conformance:

[x] Chrome
[ ] Firefox
[ ] Safari
[ ] Edge

/retest

/retest

/retest

/retest


[BUG] LIVE stream

Describe the bug
Show error in LIVE stream video
To Reproduce
`late YoutubePlayerController _controller;
@override
void initState() {
super.initState();
_controller = YoutubePlayerController(
initialVideoId: '21X5lGlDOfg',
flags: const YoutubePlayerFlags(
isLive: true,
),
);
}
YoutubePlayer(
controller: _controller,
liveUIColor: Colors.amber,
),
`
Expected behavior
Show controls
Screenshots
Infinity bucle

Technical Details:

Device: iPhone X
OS: [e.g. iOS8.1]
Version15.0

The video doesn't play here either. I can't play any Live video and there is nothing in the Log.

Any luck with the live videos?

This PR fixes the issue https://github.com/sarbagyastha/youtube_player_flutter/pull/868#issue-1889119772

use the changes like this :youtube_player_flutter:
git:
url: https://github.com/Add00w/youtube_player_flutter.git
ref: master
path: packages/youtube_player_flutter
for temporary


[lint] Demote lint error for default branches in full case statements

Our current lint rule deck treats default: branches in full case statement as lint errors.
For example, the below snippet would throw an error:
https://github.com/lowRISC/opentitan/blob/92e4298f8c2de268b2420a2c16939cd0784f1bf8/hw/ip/hmac/rtl/sha2_pad.sv#L82-L88
@eunchan and I discussed this and think the clean way in terms of coding case statements is to always specify a default: statement, even if the case is complete. Hence, we suggest to demote that lint error to an info. WDYT?

Agreed, @vogelpi. The lint rule is already in effect. I moved the case-inside out into a separate issue and close this one. See also #901.


update to calcium imaging demo

This pertains to Issue #151, mostly minor update to the calcium imaging demo. I did my best to address many of the things I mentioned in that issue, but not all. For instance:

I said it was GCaMP6f I'm not sure if this is right can we find out? :smile:
The Pynapple object has lots of data we don't discuss it might be helpful to very briefly mention. E.g., are x,y,z animal location? are rx and rz head direction like ry, but on a different plane?
Is there an advantage of the soft-plus nonlinearity when using calcium imaging data?
In general at the end it sort of just sputters out -- we show how to fit the GLM to the data, and compare to linear regression, but don't really discuss or compare and contrast. Also, what might be added to improve the fit, reduce the bias (the bias looks fairly high)? Tweaking the regularization? Am I right in assuming we only have activity history filter, but not inter-neuron filter? As it stands the results are pretty far off, so I think this needs to be addressed. I would probably low-pass filter the signals with a Gaussian (on top of the subsampling already done) and then re-run it to see how it does, it just seems to not be handling the high frequency stuff very well (but this is conjecture on my part).

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 97.30%. Comparing base (87d3c29) to head (cc6e3e0).

Additional details and impacted files
@@             Coverage Diff              @@
##           development     #159   +/-   ##
============================================
  Coverage        97.30%   97.30%
============================================
  Files               15       15
  Lines             1485     1485
============================================
  Hits              1445     1445
  Misses              40       40

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

This pertains to Issue #151, mostly minor update to the calcium imaging demo. I did my best to address many of the things I mentioned in that issue, but not all. For instance:
1. I said it was `GCaMP6f` I'm not sure if this is right can we find out? 😄

2. The Pynapple object has lots of data we don't discuss it might be helpful to very briefly mention. E.g., are `x,y,z` animal location? are `rx` and `rz` head direction like `ry`, but on a different plane?

3. Is there an advantage of the soft-plus nonlinearity when using calcium imaging data?

4. In general at the end it sort of just sputters out -- we show how to fit the GLM to the data, and compare to linear regression, but don't really discuss or compare and contrast. Also, what might be added to improve the fit, reduce the bias (the bias looks fairly high)? Tweaking the regularization? Am I right in assuming we only have activity history filter, but not inter-neuron filter? As it stands the results are pretty far off, so I think this needs to be addressed. I would probably low-pass filter the signals with a Gaussian (on top of the subsampling already done) and then re-run it to see how it does, it just seems to not be handling the high frequency stuff very well (but this is conjecture on my part).

Hi Eric, for the bias stand point, we can cross-validate the regularizer.
from sklearn.model_selection import GridSearchCV
# assume you have the glm model
grid = dict(regularizer__regularizer_strength=[0.001, 0.01, 0.1])
clf = GridSearchCV(model, grid)
clf.fit(X, Y)

This should work, modulo "regularizer__regularizer_strength" being the right parameter name. You can check the key to use,
print(model.get_params())

For what it concern what we are fitting, we are fitting only the coupling filters from other neuron, not the autocorr filter. If you add an autocorr filter, there is a long autocorr, that will result in overfitting

Yes it's gcamp6f

Thanks a lot Edoardo for this  reply. I'm adjusting things accordingly. I tried running on some Gaussian smoothed data -- it did not go well, things looked ... not better :smile: I'm adding a regularizer so the tuning curves seem to better match the actual, and also this might give students a chance to tweak it (I did a manual grid search :upside_down_face: ).

I added a little thing at the end about how this is a more experimental tool than the stuff for spiking models. Feel free to alter/cut I often err on the side of verbose or overly flowery language. :rose:
I could do a lot more with this, but I'm about out of time I have to get ready for a trip soon so have at it; what you did looks really good, I just made some minor adjustments! :fire:


Script to extract all links from a given webpage[GSSOC23]

Aim

Extract all links from a given webpage

Details

Python code to get all links

Hi @avinashkranjan I open PR related to this issue please review.
Script added for show List files in a directory #1609


Fix 'The name of the outer type and the file do not match.' issue in Array\Rearrange_array\Rearrange_array.java

CodeFactor found an issue: The name of the outer type and the file do not match.
It's currently on:
Array\Rearrange_array\Rearrange_array.java:5

Hey , Can I take this up?


[TIR] Implement API for padded layout transformations

Implementation of API in tvm.tir.schedule for layout transformations with padding, as part of https://github.com/apache/tvm/issues/12261, item "Insert pad value into generated TIR, using tir::if_then_else, builtin::assume, and builtin::undef".
Following the RFC discussion here and here, this commit preferentially rewrites the loops that surround a padded transformation where possible, in order to express padding in terms of tir::if_then_else.
cc @Hzfengsy @junrushao1994

cc @wrongtest-intellif @vinx13

Looks like the final failing unit test is due to an incorrect mapping in tir.tensor_intrin.cuda.shared_32x16_to_ldmatrix_32x16_layout.  It currently returns [(i % 4) + 4 * (j % 8), 8 * (j // 8) + (i // 16) * 4 + i % 4], which doesn't fill all 32x16 indices, and fails when attempted to use NonSurjectiveInverse on it.

@Hzfengsy Can you review/verify that the requested changes (use non-opaque blocks in unit tests) are made?  I think that's the only item remaining on the PR.


Why prepending output with prompt?

I'd rather created issue for discussion, but this repo doesn't have issues enabled.  First of all such prepending seems redundant especially for long RAG prompts. Then, it's an actual problem since I notice that triton gRPC crops the long response. Curiously, REST doesn't crop payload and full concatenation of prompt and output arrives to client.
Here I put more details of the issue https://github.com/langchain-ai/langchain/issues/12474#issuecomment-1925914483

@oandreeva-nv hello, may I ask you a question? 👆

Hi @mkhludnev, our ReadMe mentions that you can report all issues on issues page, which is for server. Apologies for confusion, we use server repo as the main entry-point for questions & issues.
Regarding your question, I believe the main goal is to report the full text prompt + generated portion. Do you happen to have a reproducer for gRPC cropping the part of the output? If it does, I am concerned that simply removing prompt from the response may not work

Considering https://github.com/triton-inference-server/server/discussions/6867 it's worth to implement echo=False option for vlm and trt backends.


Odd entry in the project Add Item context menu

@MikhailArkhipov to open a bug in Web Essentials Github and link to it from here.

https://github.com/madskristensen/WebEssentials2013/issues/1992


Unable to access databases through functions

Hello everyone, I'm currently using OrientDB 1.7-rc2 and I've created the following function with the studio:
return orient.getDatabase().query('select from ouser');

When I try running it, I get the following message:

Error on parsing script at position #0: Error on execution of the script Script: test ------^ sun.org.mozilla.javascript.internal.WrappedException: Wrapped com.orientechnologies.orient.core.exception.OConfigurationException: No database instance found in context (#2) in at line number 2 Wrapped com.orientechnologies.orient.core.exception.OConfigurationException: No database instance found in context (#2) No database instance found in context

I have the following (default) handler set on the server configuration file:
<handler class="com.orientechnologies.orient.graph.handler.OGraphServerHandler">
    <parameters>
        <parameter value="true" name="enabled"/>
        <parameter value="50" name="graph.pool.max"/>
    </parameters>
</handler>

Is there anything I can do to get this working?

Any workaround till it is fixed?

Please don't use a so old version, also unstable (rc2). Please try with 2.0.5

It's already fixed in stable release.
Closing


Duck support?

Just wondering how it goes with Duck ( https://github.com/erikras/ducks-modular-redux )?
As far as i can see, the Store it's inside ORM, and Ducks use a decoupled Store.

Uh... no, no, no.
Ducks are merely an approach for file organization for your actions and reducers - putting related actions and reducers into one file rather than potentially three separate files (actions file, constants file, reducers file).  There's no "decoupled store" or anything.  If you look at the example in that Ducks README, you see that since a duck should export default function theReducer(), you just import your duck reducers and hook them up into your app.
You can use Redux-ORM in two main ways: either define reducers per-model class and hook the generated combined reducer up to your main reducer, or use Redux-ORM as a helper/abstraction library inside of your own reducer logic.
So, overall, your reducer setup might look like:
import {combineReducers, createStore} from "redux";

import duckReducer1 from "./duck1";
import duckReducer2 from "./duck2";
import schema from './schema';
import myCustomReducerThatUsesORM from "./myCustomReducer";

const mainReducer = combineReducers({
    duck1 : duckReducer1,
    duck2 : duckReducer2,
    entities : schema.reducer(),
    otherReducer : myCustomReducerThatUsesORM
});

const store = createStore(mainReducer);

No interference with ducks at all.


Targeted Release freeze for 20H2

If we wanted to freeze the OS to stay at 20h2 for 30 months, using MDM,  What XML settings can be used, previously we have been using
https://github.com/vmware-samples/euc-samples/blob/master/Windows-Samples/Custom Settings/Windows Update for Business/TargetVersion-2004.xml
In the XML we are entering 20h2 and that does not appear to be supported.
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6a31db82-81be-16aa-9e39-62c6810517d8
Version Independent ID: 30fe7813-ac6e-8167-02d6-9fa5e530a3f6
Content: What's new in Windows 10, version 20H2 - What's new in Windows
Content Source: windows/whats-new/whats-new-windows-10-version-20H2.md
Product: w10
Technology: windows
GitHub Login: @greg-lindsay
Microsoft Alias: greglin

I am researching now. Stay tuned.

@greg-lindsay Thank you  I will look forward to your findings.

I've confirmed that this is a bug and it is going to be resolved soon. However at this time it isn't possible to use the XML file because the CSP rejects the "H" letter.  A workaround for now is to use deferrals. I am sorry I don't have an ETA on fixing this but I personally think it won't be too long, on the order of weeks rather than months.


Is it possible to display some kind of help text on launching a console?

Just wondering if there's a way to be able to display a helpful message when the user opens the PyConsole. For example I'd like to explain that some data was pushed to the namespace and is available to the user.

I had the same problem. I ended up commenting out  line 99: self._show_ps(), and then in my app, I used:
pyqt_console_obj._insert_output_text(helpful_message)
pyqt_console_obj._show_ps()


fix: Documentation

Hey!
While working with this great implementation I stumbled across some missing words in the documentation and fixed them. I understand if you don't want to merge PRs concerning docs so feel free to add these missing words by yourself :smile:.
Keep up the amazing work!
Jakob

Thanks!


Rancher recreating all RKE2 machines provided via Digitalocean

Rancher Server Setup

Rancher version: 2.6.4
Installation option (Docker install/Helm Chart):

Helm 1.29.6 - RKE2

Proxy/Cert Details:
Let`s encrypt

Information about the Cluster

Kubernetes version: 1.21.9
Cluster Type (Local/Downstream):

Downstream, Digitalocean, created via rancher RKE2

Infrastructure Provider = Rancher provisioning the nodes using different node drivers (e.g. AWS, Digital Ocean, etc)
-->

User Information

What is the role of the user logged in? (Admin/Cluster Owner/Cluster Member/Project Owner/Project Member/Custom)

Admin

Describe the bug
After Upgrading to rancher 2.6.4 a complete new rollout of our digitalocean machines was triggered, new machines have been recreated and the older ones have been deleted.
To Reproduce
Install rancher v2.6.3-patch2, Create cluster via Digitalocean cloud provider, upgrade to 2.6.4
Result
complete new rollout of our digitalocean machines was triggered, new machines have been recreated and the older ones have been deleted.
Expected Result
No Changes to the downstream cluster.

I dug a bit through the rancher logs and found this entry occuring for all nodes, which seems to lead to the recreation, after the recreation the older nodes get destroyed to meet the machine pool config-
2022/04/05 09:58:54 [ERROR] error getting machine from node (NODENAME-882cd3fb-44255) for snapshot (fleet-default/): unable to find node NODENAME-882cd3fb-44255 in machines


Somehow this map stoped filtering for me

https://team.cartodb.com/u/saleiva/editor/cea0f564-19e4-11e6-95b4-0e674067d321/embed
But it works perfectly in the editor view.
It was working and suddenly stopped filtering. I guess it might be something related to the cache.

It looks like it is working again! ;-)


LocalisationUtil.cs

Value = "The Google Local Search API is no longer available.Migration to the Google Places API (https://developers.google.com/places)"

Thanks. Will remove it.

Fixed in v1.7


Controlling Android Phone on your Computer

Controlling Android Phone Using SRCPY
Introduction paragraph (2-3 paragraphs):
SRCPY is a free and open-source screen mirroring tool that lets you operate an Android device from a desktop computer running Windows, Mac OS X, or Linux. Genymobile, which also created Genymotion, an Android emulator, is now working on the software.
The following are the steps of installing and running SRCPY  to mirror the Android phone on the computer By mirroring the Android phone on your computer ,you will be  able to control it from your computer.
Key takeaways:

Mirror and control android device on Kali Linux via USB cable.
Mirror and control android device on  windows using TCP/IP.
Installing srcpy on Kali Linux.
Remote controlling of an android phone

References:
N/A

@nancymumbimaina
Good afternoon and thank you for submitting your topic suggestion.
Your topic form has been entered into our queue and should be reviewed (for approval) as soon as a content moderator is finished reviewing the ones in the queue before it.

@nancymumbimaina
Sounds like a helpful topic - lets please be sure it add value beyond what is in the official docs and what is covered in other blog sites.
Please be sure to double check that it does not overlap with any existing EngEd articles or incoming topic suggestions (if you haven't already) to avoid any potential article closure, please reference any relevant EngEd articles in yours. - Approved

closed #3033 via #3566


COMP-3 fields after Variable occurs clause is not getting allocated

I am trying to read Mainframe data using following copybook structure. It has variable occurs clause and Comp-3 fields after variable occurs.

Code below:
    val df = spark.read.format("cobol").option("copybook","Mdet5.txt").option("debug_ignore_file_size", "true").option("file_start_offset", "4").option("record_length_field", "RECORD-LEN").option("schema_retention_policy", "collapse_root").load("mcy_mdet.dat")

I tried option("variable_size_occurs", "true"), but values are getting shifted to the left. I think some bytes were allocated to Header in the beginning. But I don't RDW header to be allocated since i have RECORD-LEN field in my copybook.
When i process with above code, COMP-3 fields after Variable occurs clause are not getting allocated. I am not seeing these fields (RDT-CENTURY-JULIAN-POST-DATE, RDT-LS-IVA-AM, RDT-1098-PREV-INTR-THRD-AM, RDT-1098-PREV-INTR-TOTL-AM, RDT-DCLN-AVLB-CRDT-AM, RDT-CNCY-CNVR-FEE-AM, RDT-CRSS-BRDR-FEE-AM)  in the output df.  Please help!
Input file opened in Hex editor below:

What about fields with PIC X., PIC X(3) and PIC X(16)? Are they correct?

Yes. The PIC X, PIC X(3), and PIC X(16) fields are getting allocated in the output df. Only Comp-3 fields are missing.

It is strange. COMP-3 values have such an encoding that not every character sequence corresponds to a value. So incorrect bytes cause COMP-3 fields to become null.
The fields that have PIC of X, X(3), etc are always non-null. So the presence of values in these fields doesn't mean they are correct.
I think in your case there is a shift of data due to mismatch offset or a not complete correspondence between the copybook and the data file. This leads to the situation when COMP-3 fields are not decoded correctly. This needs to be carefully investigated to determine when this shift happens and how many bytes it spans.

COMP-3 fields are not present in the output df itself. Only PIC values are displayed.  Comp-3 fields are missing. I will paste the output screenshot in few mins.

The COMP-3 fields are missing because the sequence of bytes cannot be decoded into a number. I wish I could help, but it requires knowledge of the copybook and the data. You need to identify when data starts to shift against the copybook and why.
There is an option .option("rdw_adjustment", 0). Although it says RDW, it adjusts length returned by RECORD-LEN as well. If the usage of RECORD-LEN shifts the data, you can try to adjust the length to the actual length by trying various values of the adjustments.
Another suggestion. Could you attach a JSON file of the first parsed record, I'll take a look, maybe I can find something obvious. But no guarantees.

I have fixed the problem.  I removed the first 4 bytes in the file manually and removed option(file_start_offset, 4) and added option(variable_size_occurs, true) in the code.  I think file_start_offset and variable_size_occurs option does not work well together. But have to do more testing.  Thanks a lot for your help!


intiial testing materials

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@4818a90). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master       #3   +/-   ##
=========================================
  Coverage          ?   76.92%
=========================================
  Files             ?        4
  Lines             ?       13
  Branches          ?        0
=========================================
  Hits              ?       10
  Misses            ?        3
  Partials          ?        0

Impacted Files
Coverage Δ

setup.py
0% <ø> (ø)

rhf/__init__.py
100% <100%> (ø)

rhf/rhf.py
100% <100%> (ø)

tests/test_rhf.py
100% <100%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4818a90...28abbd5. Read the comment docs.


[Bug] Cannot gain write permission in sdcard directory

Device: Android 6.0
Already enable Document, but still can set custom download directory to sdcard, it said, The Directory is not writeable.

Please click the "DOCUMENT" button instead of the "CONTINUE" button.

I did, but it bring me to empty activity, with side menu,  that i think is Document App, there is only "Recent" button that does nothing.

Is there a three-dot button at the top-right corner?

Yes its there,  it open menu, "show internal storage", and toggle to "Show external storage" vice versa, yet it too doesn't t do anything


chore: update @fugle/trade and apply getVolumePerUnit

@fugle/trade 升到最新板
getOrders 加上正確的張數股數資訊

Codecov Report

Merging #4 (18e1dac) into master (0ef9964) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##            master        #4   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           18        18
  Lines          257       274   +17
  Branches        34        36    +2
=========================================
+ Hits           257       274   +17

Impacted Files
Coverage Δ

src/placed-order.ts
100.00% <ø> (ø)

src/client.ts
100.00% <100.00%> (ø)

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more


Gets snapshot storages before calling verify_accounts_hash()

Problem
On pop256, one of the nodes kept failing during accounts hash verification. The root cause was due to the incremental accounts hash calculation getting wrong snapshot storages to use for the actual calculation.
Accounts hash verification happens in the background, and getting the snapshot storages is concurrent with normal node activity. Verification also checks the full accounts hash first; so by the time incremental accounts hash is checked, it has been many minutes later on pop256, which has over 5 billion accounts.
Getting the snapshot storages happens lazily—right before calculating each accounts hash. Thus, calculating the incremental accounts hash doesn't get the snapshot storages it needs until the node has been running for a while, and clean has run many times. In this case, the storages were marked as dead, and thus the IAH did not get all the storages it needed for the calculation.
We need to get the snapshot storages before calling verify/before the foreground processing begins, to ensure we have all the correct storages.
Summary of Changes
Pass in the snapshot storages as a function parameter to verify_accounts_hash_and_lamports() as the fix for startup verification accounts hash mismatch issues due to storages getting cleaned away prematurely.
This change fixed the issue seen on pop256.

Codecov Report
Attention: Patch coverage is 97.29730% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 82.1%. Comparing base (aa2f078) to head (a1d7153).
Report is 7 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff            @@
##           master    #1202     +/-   ##
=========================================
- Coverage    82.1%    82.1%   -0.1%
=========================================
  Files         886      886
  Lines      236417   236442     +25
=========================================
- Hits       194266   194256     -10
- Misses      42151    42186     +35


Change Normal & Pressed Color Programatically

Is there a way to change the color of the Floating Menu programmatically using setNormalColor or setPressedColor methods. It is very important for me as I want to change the colors at run-time.

Duplicate of #49

I'm also need to change the "fab_addButtonColorPressed" color programatically.

I also need to change the "fab_addButtonColorPressed" pragmatically.


Oneshot latch acts over more than one keypress

Okay, I'm not sure if this is a bug or intended behavior:
When a oneshot modifier is tapped and the following keypress is not released before a second key is pressed the oneshot actually works over both keys. Like this:
~Press~ Tap oneshot shift; press a, press b, release a, relese b
produces: AB.
I believe this behavior is helpful for pressing combos on a oneshot layer without holding the layer activation button, but it does disrupt typing when using oneshot shift or control: If I hit shift for a oneshot capital letter I expect to get only a single capital letter, not two of them. In my experience this does disrupt typing, especially for passwords as we have no feedback of what was actually typed.
I have 'fixed' this for myself by clearing the oneshot at every keydown event that emits a keycode:
@@ -424,8 +424,9 @@ static long process_descriptor(struct keyboard *kbd, uint8_t code, struct descri
                } else {
                        kbd_send_key(kbd, d->args[0].code, 0);
                        send_mods(kbd, descriptor_layer_mods, 1);
-                       clear_oneshot = 1;
+                       //clear_oneshot = 1;
                }
+               clear_oneshot = 1;

                oneshot_latch = 0;
                break;

Again, I'm not sure if this is intended or if it is there for any other specific reason, I'm just sharing my rationale based on my own experience. I can surely provide a pull request if that is desirable.

The current behaviour is exactly the reason for me to love oneshot keys, so I'm pretty sure it's intended behaviour. However, I suppose that I could get it back with overload(shift, oneshot(shift))? It would evidently cover both our use cases then... but it doesn't look as nice :P

I'm sorry, I don't think I was very clear on my wording. The behavior you describe is desirable and it is also maintained with the change I proposed. I have edited my text to be more clear.

I believe the original reason for this was to produce sequences like:
shift down
j down
j up
shift up

as opposed to
shift down
j down
shift up
j up

Aside from offending my aesthetic sensibilities, the former intuitively seemed less likely to produce conflicts, though I suspect in practice most software will probably just inspect the modifier mask on key down.
I am not opposed to changing the default behaviour to counteract the problem you describe, but I will have to think about potential side effects.
I am currently in the process of performing some major surgery on the input logic to address a few other problems, and will probably integrate this into those changes if no objections are raised.

I suppose once instance in which the current behaviour might be desirable is if the user does something like <control down> <control up> <c down> with a oneshot control. The present behaviour will cause holding c to produce successive C-c events, while the proposed change would subsequently produce an unmodified c.

I believe the original reason for this was to produce sequences like:
shift down
j down
j up
shift up

as opposed to
shift down
j down
shift up
j up

Aside from offending my aesthetic sensibilities, the former intuitively seemed less likely to produce conflicts, though I suspect in practice most software will probably just inspect the modifier mask on key down.

Well, that does make sense, but I also think most software will just use the keydown immediately and just ignore the order in which the keyups happen. I think the reasoning lemnos provided at #34 relates to this: when typing naturally key releases are not really accounted by our brains, so we might really do:
shift down, a down, shift up, a up
when we mean A instead of the intuitive way of thinking of the keypresses:
shift down, a down, a up, shift up
so any software relying on shift being still down on keyup would probably show itself quite weird to
About this:

The present behaviour will cause holding c to produce successive C-c events, while the proposed change would subsequently produce an unmodified c.

Well, that does make sense, but I also think most software will just use the keydown immediately and just ignore the order in which the keyups happen. I think the reasoning lemnos provided at #34 relates to this: when typing naturally key releases are not really accounted by our brains, so we might really do:

I should probably note that I am lemnos :P.

so any software relying on shift being still down on keyup would probably show itself quite weird to use even without keyd.

This is likely true, though as noted it breaks down in the case in the case of repetitions since the display server generates additional key down events after the modifier is released.

From what I read of the source code it seems that 'pressed' almost always takes the value from evdev event value, which makes me believe it would have value 2 if an auto-repeat is being triggered and the described behavior could be maintained by not setting clear_oneshot if pressed == 2.

Your patch clears the oneshot modifier immediately on key down, so changing behaviour on repeat wouldn't make a difference. If you clear on key up, then the intervening key down inevitably becomes collateral, but if you clear on key down you lose the ability to repeat the modified key. Having said that, I think the first option is probably more desirable than the first. One could also generate a synthetic key up event after an intervening key down, but that potentially has other implications.

I believe this should be fixed in the latest commit, which facilitates both behaviours by updating mod state on either the next key up or key down.

a1edfab5d124aa70493dd988973f17bf4a9a4c30 definetely solves the issue for me. It also preserves the behavior of repeating modifiers. Thank you very much!


Publish ExchangeType missing

Hi,
In latest version 5.1.5 when i follow example from docs
Bus.Factory.CreateUsingRabbitMQ(..., cfg =>
{
    cfg.Send<SubmitOrder>(x =>
    {
        // use customerType for the routing key
        x.UseRoutingKeyFormatter(context => context.Message.CustomerType);

        // multiple conventions can be set, in this case also CorrelationId
        x.UseCorrelationId(context => context.Message.TransactionId);
    });
    //Keeping in mind that the default exchange config for your published type will be the full typename of your message
    //we explicitly specify which exchange the message will be published to. So it lines up with the exchange we are binding our
    //consumers too.
    cfg.Message<SubmitOrder>(x => x.SetEntityName("submitorder"));
    //Also if your publishing your message: because publishing a message will, by default, send it to a fanout queue.
    //We specify that we are sending it to a direct queue instead. In order for the routingkeys to take effect.
    cfg.Publish<SubmitOrder>(x => x.ExchangeType = ExchangeType.Direct);
});

i can't set cfg.Publish ExchangeType like in example, there is no property ExchangeType, did something change, if yes can you provider correct way to set up ExchangeType ?
Thanks

sorry, wrong interface used, i used base IBusFactoryConfigurator not IRabbitMqBusFactoryConfigurator


adding unit tests

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@782c8d2). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master      #39   +/-   ##
=========================================
  Coverage          ?   92.32%
=========================================
  Files             ?      117
  Lines             ?     1720
  Branches          ?       97
=========================================
  Hits              ?     1588
  Misses            ?      132
  Partials          ?        0

Impacted Files
Coverage Δ

core/src/main/scala/hydra/core/akka/KryoInit.scala
100% <100%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 782c8d2...96b5332. Read the comment docs.


Create first draft of OpenAPI specification for website

I've wanted to have an OpenAPI specification for the website for a while. Here's an early cut, drafted in part by OpenAPI.

Seems good, I'm a fan of auto-generating these

To be fair, the API spec is not auto-generated, this one is hand-generated by OpenAPI. We could do that instead, but that'd require some real work.

Nowadays maybe there's some kind of LLM or something that could spit out a fully made openapi spec from the code I bet?


Add Footer Tests

Add footer tests for the following links:

My Account
Orders and Returns, via the checkout suite
Search Terms
Contact

Minor refactoring of some selectors, based on the current test site at https://hyva-demo.elgentos.io/

Hello friends! My first pull request to this project 🤝

@lefte awesome, thanks! Could you also update the readme?

@lefte awesome, thanks! Could you also update the readme?

I was just thinking about that, one minute!

Thanks!


Rollback to java 1.6 version

This change is

PR is added to the queue for testing as 1. task. (b9e18a9)

Test 107004850 (b9e18a9) build started.

Test (Fail) 107004850 build is completed (b9e18a9)
See https://github.com/ThaliTester/TestResults/tree/107004850b9e18a9_Rollback_to_java_1_6_version_evabishchevich/ for the logs

Comments from Reviewable


Reversing of array

->REVERSING OF ARRAY
create a file reversing of array.cpp in  which I code 2 methods of reversing any array -->

reverse1--->reversing any array with the help of auxilliary array in this method first we will copy the elements of first array in auxilliary array in reverse order and then again copying the elements of auxilliary array to main array this will reverse our main array.
reverse2-->with the help of two pointers high and low, low will iterate from left to right while high will iterate form right to left until low<high and swapping the high index element with low index element this will reverse our main array.
I code above 2 methods with proper comments and mention  time complexity which is O(n) and space complexity which is O(n) in reverse1 and O(1) in reverse2.

Related Issue
close #498
Proposed Changes

Create a reversing of array.cpp file
update README.md  accordingly and mention about my file in README.md with some test cases and discuss about the algorithm of  both methods of reversing any array i.e. reverse1 and reverse2.

Checklist

[x] ✅ My code follows the code style of this project.
[x] 📝 My change requires a change to the README.
[x] 🎀 I have updated the README accordingly.
[x] 👀 I have read the CONTRIBUTING document.
[x] ✨ I have added tests to cover my changes.
[x] 🚩 All new and existing tests passed.
[x] 🌟 ed the repo

Output Screenshots
code image 1

code image 2

input:

output:

You need to follow particular guidelines for your README , please follow
these guidelines click here
also you can see how your README should look like see here
for more please go through these here

Update readme👍

Update readme👍

Where is your topic

Changes Done👍

Add as a link Format

Readme Updated

Readme Updated

Again 4 files changed...Sorry but closing Your PR for having 9 Commits...make fresh PR with less commits 2-4.
If you have an doubt why 4 files changed dm me on discord .


Add favicon to web server

Generated via:
convert ./docs/images/logo.png -resize 16x16 web/favicon.ico

Generated via:
convert ./docs/images/logo.png -resize 16x16 web/favicon.ico

I recommend using a larger size. Browsers will resize it so something smaller depending on usage. Maybe 128x128?

Generated via:
convert ./docs/images/logo.png -resize 16x16 web/favicon.ico

I recommend using a larger size. Browsers will resize it so something smaller depending on usage. Maybe 128x128?

Based on this SO's post I used a generator on realfavicongenerator.net. The file is now in the docs/images folder, and symlinked into the web folder


Added capability to Makefile to create release candidates

As it says above.  The idea is just to set the version, RC, and run make like normal.
To create a final release, as it says in the Makefile, set the RC to 0.

Forgot to add - I tested this creating both an "RC" and a "Final" release, on both SL7 and AL9.


Xenko does not work without the Internet.

Xenko does not work without the Internet.
https://pastebin.com/ZW5kmx5b

Xenko not starting.
I deleted Xenko.

hi @Degaron, the installer needs to download packages from the xenko server in order to install it. its an online installer.
i think the issue with internet connection on startup is already fixed for <IP_ADDRESS>. see https://github.com/xenko3d/xenko/commit/d17e74d6619835219b06776258089af02a1a22fb

I downloaded the packages.

how did you download them? via the installer?

@Degaron The installer from the download page is a light-weight installer that will just install the Xenko launcher. Similarly to what Microsoft does with Visual Studio 2017. Then it needs to get the newest package from the server. So indeed you do need a working Internet connection to install it and also to get updates.
Note that the initial installer also needs an Internet connection to get the prerequisites packages (.Net SDK, MSBuild, etc.).
@xen2 Maybe we should make it more obvious in the download page that a connection is required. I was also wondering whether we could provide an offline package (i.e. launcher+latest version) but I don't think we can include the third-party dependencies (such as MSBuild) without license issues.

You probably did not understand.
The first post is exactly what you are talking about.
But after that, I installed Xenko and got a lot of errors when trying to run Xenko.

Sorry can't read pastebin links. Write relevant information here (maybe not the full log but at least the relevant error message).
Secondly, the title of the issue is "Xenko does not work without the Internet" so I am answering that.

#399 covers this issue, closing this one.


Shorter, more focused referencing guide

This reworks the less-controversial parts of a prevoius attempt (#69) at a referencing guide that was over-ambitious, among other things.  It's amazing how being in better health makes my writing less cranky and confrontational! 😅

Thanks @handrews! I need more time to look at this properly, there are some really great nuggets in here of what doesn't work and why, but I think we might also add some "do it this way" advice and some examples to try to lead people out of the dark place in which they find themselves. Would it be okay for me to add changes directly to this pull request to restructure some of the content?

I forgot to add that I put the rendered changes up here:

https://handrews.github.io/renderings/learn-referencing/referencing/faq.html
https://handrews.github.io/renderings/learn-referencing/referencing/what.html

Thanks for being open to some content restructure @handrews - I shifted things out of FAQ format (not considered current practice in documentation, and not an established pattern in these docs) mostly because those articles can get long and difficult to navigate. The content is split into topics, with each page added to the sidebar.
In general, I think paragraphs might be easier to read than bullet points, but I suggest we don't block the pull request for that (I've done enough holding this one up already!).
I would have loved to have seen at least one or two examples in the text, I think that's something that the learn site can excel at - but we can add them later, maybe after hearing user feedback.

@lornajane thanks!  I will look at this in detail soon.  Just to respond to a few comments first:

shifted things out of FAQ format (not considered current practice in documentation

Great!  I'm not up on current best practices and was not attached to a FAQ format at all.

In general, I think paragraphs might be easier to read than bullet points

I've been told "bullet points, not paragraphs!" over and over by people who outright refuse to read anything I write in prose because I'm too wordy.  I'm happy for anyone else to restructure it, I'm just hyper-sensitive at this point.

I would have loved to have seen at least one or two examples in the text

I am horrible at examples. I learn theory-first, unless the theory just isn't articulated and I have to construct it from examples.  Most people learn examples-first.  I've never understood how to connect with that, so hopefully others will fill them in.  I'm happy to provide a topic list or feedback, though.

Let's leave the bullet points and not worry too much about the examples. They can be fleshed out over time - I do think they're useful, but let's add them as we see them in the wild or something!

@handrews I think this is an improvement and we should merge if you're happy enough to let this version go as-is.


Release for zig 0.13

Hi,
just tried mecha 0.8 with zig 0.13 via zig package manager - fails to build due to incompatibility. Works with head, so wondering if we can release that is compatible with 0.13. Thanks!
Regards

Aah right. Zig had a new release and I forgot. Give me a moment and I'll tag a new release :)

Yea, just made sure everything works. https://github.com/Hejsil/mecha/releases/tag/0.9.0


Getting error while running the LINTUL3 model

I am using the sample data quickstart_part3 and code to run the LINTUL model.
All the code is working fine except the following line-
lintul3 = LINTUL3(parameterprovider, weatherdataprovider, agromanagement).
The above line is giving the following error (attached)

Please help me to resolve this issue.
Thank you

I think this is an outdated agromanagement file (can you provide the download link so I can update it?). Can you try with the input data in the data folder within the pcse_notebooks repository?
Allard

Thank you for your reply:
I used the agromanagement file from the data folder within the pcse_notebooks repository. The error has been resolved.
Regards
Rohit


Support git branches containing slashes

After upgrading the puppet-r10k module, I had to migrate from the built-in webhook to webhook-go.  After doing so, r10k stopped working for some branches.
If the branch name contains one or more slashes, webhook-go only handles the part after the final slash.  Ex. when the branch is named feature/TICKET-1234, the webhook-go systemd log shows this:
Jun 18 13:44:35 puppet-server-01 webhook-go[3046986]: time="2024-06-18T13:44:35-07:00" level=error msg="failed to execute local command `[/opt/puppetlabs/puppet/bin/r10k deploy environment TICKET-1234 --config=/etc/puppetlabs/r10k/r10k.yaml --verbose --generate-types --modules]` with error: `exit status 1`
<snipped>
ERROR\t -> Environment(s) 'TICKET_1234' cannot be found in any source and will not be deployed.\n

It is common to use slashes in branch names.  This post has several good examples.  r10k itself handles slashes by replacing them with underscores like it does with hyphens, ex. the feature/TICKET-1234 branch becomes known to Puppet agents as the feature_TICKET_1234 environment.
Looking through the source code, it appears the behavior is caused by this: https://github.com/voxpupuli/webhook-go/blob/fef55c882e825f130d63b68308604e1dbd53ae5e/lib/parsers/github.go#L26
From the Go documentation:

Base returns the last element of path.

This is a major regression from the webhook daemon that used to be included in the puppet-r10k module and is effectively a blocker for me.  Looking at the logs from the old Ruby webhook daemon, it runs r10k like this, retaining the full branch name:
umask 0022; r10k deploy environment feature/TICKET-1234 -pv branch: feature/TICKET-1234

This is how the old webhook determined the branch name:
https://github.com/voxpupuli/puppet-r10k/blob/b565bd104ee9826e443b5461dad1075404774b75/templates/webhook.bin.erb#L168-L176
    branch = (
        data['ref']                                               ||  # github & gitlab
        data['refChanges'][0]['refId']                 rescue nil ||  # stash
        data['push']['changes'][0]['new']['name']      rescue nil ||  # bitbucket
        data['changes'][0]['ref']['displayId']         rescue nil ||  # bitbucket server repository event
        data['pullRequest']['fromRef']['displayId']    rescue nil ||  # bitbucket server pull request event
        data['resource']['refUpdates'][0]['name']      rescue nil ||  # TFS/VisualStudio-Git
        data['repository']['default_branch']                          # github tagged release; no ref.
      ).sub('refs/heads/', '') rescue nil

It stripped 'refs/heads/' from the string.
Please parse the branch name with something other than path.Base(), or at least provide a configuration option to do so.

Should be merged. I'll get a release out later

@dhollinger Do we have any idea when a release with this change might be coming out?

@fffialho Sorry, for some reason I thought I had already triggered a release. I'll get that done soon


cube: Demonstrate how to select other VkPresentModeKHR's

This comes from the Khronos-internal discussion of
https://github.com/LunarG/VulkanSamples/issues/98.  Cube will, by default,
choose VK_PRESENT_FIFO_KHR mode.  However, when that change was made, the old
(example) code was lost that showed how to search for the other present modes.
It was also discussed that there isn't good guidance on why an application
might want to use a mode other than FIFO.  This change adds both of those, but
only selects FIFO by default.  #ifdef's can be used to select other modes.
With that code is a reason why an application might wish to use that present
mode.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

LGTM too, without the printf

Cherry-picked and pushed to master.


Support Gcloud too

At present, this essentially supports Gcloud, as long as indexes are created locally and then uploaded.
It doesn't include any functionality for accessing a location on Gcloud storage and retrieving its contents.
This would be very useful.
Alternatively, use some third-party module to cloud-agnostically retrieve a file list.

See https://pypi.org/project/cloudstorage/
and also this related version https://github.com/monarch-initiative/monarch-file-server/blob/main/scripts/directory_indexer.py


Emotes parse wrong somehow

Describe the bug
Emotes parse wrong somehow
To reproduce
Try to send this in chatterino: PowerUpL 🤓 PowerUpR hehe, you will be banned ❗
Screenshots
Original message:

In chatterino:

Add more spaces and it will look like this in chatterino:

Chatterino version
Chatterino 2.2.3-beta2 (commit 6499b7c12)
Operating system
Windows 10 64Bit

Cannot reproduce, Chatterino 2.2.3-beta2 (commit 1f5b62e6e)

Duplicate of #2479
Was fixed in #2490. Bug is no longer present in nightly.


refactoring log format way

When I'am refactoring the base code. I got boring about the code, so I want to be lazy, and keep the code concise.
    def province(self) -> Optional[str]:
        """
        get the province of the account
        """
        log.info('province() <%s>', self)
        if self.payload is None:
            return None
        return self.payload.province

there are some problems in the code.

the logging format is fixed, why can't we movie it to the decorator to implement it.
we log the input of the function, but can't log theoutput of the function. eg : contact.city(), ccontact.province(), contact.gender(). I think the result of the funciton is important that we should to log. But we can't log the result in one line code.

In order to resolve the above problems, I create function decorator.
class wechaty_logger(object):
    def __init__(self, module_name: str = None):
        self.module_name = "Wechaty" if module_name is None else module_name

    def __call__(self, func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            log = logging.getLogger(self.module_name)
            log.debug('%s: <%s> <%s>', func.__name__, *args)
            result = func(*args, **kwargs)
            self.can_do_other_things()
            log.debug('%s return value: <%s>', result)
            return result
        return wrapper

    def can_do_other_things(self):
        pass

@wechaty_logger('Contact')
def province(self) -> Optional[str]:
    """
    get the province of the account
    """
    if self.payload is None:
        return None
    return self.payload.province

So, with this code we can keep the code more concise, and more functional. we can  create different decorator for different func to implement different functions.
@huan  What do you think ?

It's a very good suggestion, and it will help us save lots of typings when we are coding.
However, I believe we need to think about the following cases before we use this decorator for logging:

Some methods need to log at level info and other methods to log at level verbose or silly, we need to be able to deal with it
Some methods need to log in the middle of the function, for example, we always use log.verbose(...) at the beginning of the method, and we will use log.silly(...) in the middle of the method if we need any debug information to be logged.
In some cases, we should not log all the input arguments, because it will be too verbose. E.g. we might use log.verbose('Puppet', 'contactParseRawPayload({id:%s)', payload.id) so that we only log the payload id instead of the whole payload
In most of the cases, we need not log the return value of the methods. And there are many cases that the return value will be too verbose, like Puppet.roomMemberPayload(id) will return a JSON payload which includes hundreds of the room members.

Let's think about how to manage the above cases in a good way, and then we will be able to continue discussing this great decorator for better logging.

LogDecorator can only replace the common log pattern and have some tricks.

log in the front of the function.
unify log format.
log full args of the input and result of the funcs when log level is debug.

Below is my thinking:

info, verbose, silly log type can be passed to the logger.

@wechaty_logger('Contact', logging.INFO)
def func(self):
    pass

This belongs to special functions, no designs can resolve it. so , handing code.

Different class instance can have different log format. eg: ContactPayload often log id attribute, so we should return id attribute in __str__ func. if ContactPayload often log all attributes, so we should return joined full attributes in __str__ func. I think log format depends on the frequency of the log format.
If a class have different log format, it should provide different log funcs depends on log frequency.

About why to log the result of the funcs, I think this is important when user set debug log-level. Developer can see input and output of the functions. So they have more information about how to debug and fix the debug.
There is a problem about many list of the result, or a inner attribute have a list attribute that contains many data. We should log all of them? No, we will detect the list attribute and log one or two of them.

Thanks for sharing your thoughts about the log decorator in this detail.
I believe that we can build a POC of this and try to use it in the new code so that we can get to know how well it works.
There's one more thing I want to suggest, is that I believe we can simplify the decorator more by adding a new factory function to it:
contact_logger = wechaty_logger('Contact', logging.INFO)

@contact_logger()  # with the default 'Contact' and logging.INFO level
@contact_logger(logging.DEBUG)   # to override the logging.INFO

Please go-ahead to implement it, and I'm looking forward to seeing how it works with out code.

I have created WechatyLogger, which support different logger method.
log = WechatyLogger("ModuleName")

@log
def create(self):
    # do something

@log(logging.ERROR)
def create(self):
    # do something

But I think I was wrong with one idea about the logger. I thank wechaty_logger could simplify the code when developer wanna to get more detail about debug information. But they can also get all the input and output of the method in wechaty.

Hi @wj-Mcat , thanks for the update!
I'd like to suggest that we should focus on the basic Python Wechaty API implementation at this stage, the logger I believe it's not a big deal because we will only write them for once, mostly.
Please feel free to let me know when your PR is ready for merging and let's finish it and move forward.


Old version in bower

I install admin lte with bower why the version still 2.3.0 (currently 2.3.1)
code note : "admin-lte": "*" or "admin-lte": "~2.0"

Hello @adiramardiani
Version 2.3.1 hasn't been released yet. We working on a couple of bugs before we release it. Once it is ready, it will be available on Bower. Thanks!


Multithreading

Support more threads during lexing and parsing.

Basic support is completed using pthread and windows thrading.


Missing permute in (A::ArNet)(x::Matrix{T})?

In the following function
function (A::ArNet)(x::Matrix{T}) where T <: Integer
    @extract A : J H p0 idxperm
    backorder = sortperm(idxperm)
    N, M = size(x)
    length(H) == N - 1 || throw(DimensionMismatch("incompatible size between input and fields"))
    q = length(p0)
    output = Array{eltype(p0),2}(undef,N,M)
    #Threads.@threads
    for i in 1:M
        output[:,i] .= _outputarnet(view(x,:,i), J, H, p0, N, q)
    end
    permuterow!(output,backorder)
end

the input matrix x is backpermuted ( permuterow!(output,backorder)) after calculating output, but never permuted before calculating output.
Is that a bug or wanted behavior? If the latter it is slightly confusing, since the vector version do permute the input before calculating output. If it's a bug I can make a PR.

Well if you want to evaluate a trained model on a new dataset, then
arnet, arvar = ardca(some_fasta)
Z = read_fasta_alignment(some_other_fasta, 1.0)
arnet(Z)

gives wrong results, since the sequences are not permuted if given as part of a matrix Z.
The code
arnet, arvar = ardca(some_fasta)
Z = read_fasta_alignment(some_other_fasta, 1.0)
hcat([arnet(Z[:,m]) for m in 1:size(Z,2)]...)

gives correct results, since the vector version of arnet permutes the sequence Z[:,m].
If that is wanted behavior, we can close this issue.

Then I have to think about it ... but if you are willing to do a PR that
would be great!
A
On Wed, Mar 9, 2022 at 5:06 PM Christoph Feinauer @.***>
wrote:

Well if you want to evaluate a trained model on a new dataset, then
arnet, arvar = ardca(some_fasta)
Z = read_fasta_alignment(some_other_fasta, 1.0)
arnet(Z)
gives wrong results, since the sequences are not permuted if given as part
of a matrix Z.
The code
arnet, arvar = ardca(some_fasta)
Z = read_fasta_alignment(some_other_fasta, 1.0)
hcat([arnet(Z[:,m]) for m in 1:size(Z,2)]...)
gives correct results, since the vector version of arnet permutes the
sequence Z[:,m].
If that is wanted behavior, we can close this issue.
—
Reply to this email directly, view it on GitHub
https://github.com/pagnani/ArDCA.jl/issues/18#issuecomment-1063088737,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJYTHVXCMLA2GVUGLUMOIDU7DD7VANCNFSM5QJYHCJQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID: <pagnani/ArDCA.
@.***>

--
Politecnico Torino
Corso Duca degli Abruzzi 24,
I-10129 Torino - Italy
+39<PHONE_NUMBER>
web: http://www.polito.it/cmp/home/pagnani
Italian Institute for Genomic Medicine (IIGM)
IRCCS di Candiolo, SP142 km 3,95
10060 Candiolo (TO)

I finally understood the problem. Of course you are right there was an inconsistency
corrected by commit: 9112dc6

I wonder if  I should tag a new version


Improve Actions hygiene

👋 hello there! I'm a fellow Googler who works on projects that leverage GitHub Actions for CI/CD. Recently I noticed a large increase in our queue time, and I've tracked it down to the limit of 180 concurrent jobs for an organization. To help be better citizens, I'm proposing changes across a few repositories that will reduce GitHub Actions hours and consumption. I hope these changes are reasonable and I'm happy to talk through them in more detail.

(you were already doing this, thank you!) Only run GitHub Actions for pushes and PRs against the main branch of the repository. If your team uses a forking model, this change will not affect you. If your team pushes branches to the repository directly, this changes actions to only run against the primary branches or if you open a Pull Request against a primary branch.

For long-running jobs (especially tests), I added the "Cancel previous" workflow. This is very helpful to prevent a large queue backlog when you are doing rapid development and pushing multiple commits. Without this, GitHub Actions' default behavior is to run all actions on all commits.

There are other changes you could make, depending on your project (but I'm not an expert):

If you have tests that should only run when a subset of code changes, consider gating your workflow to particular file paths. For example, we have some jobs that do Terraform linting, but they only run when Terraform files are changed.

Hopefully these changes are not too controversial and also hopefully you can see how this would reduce actions consumption to be good citizens to fellow Googlers. If you have any questions, feel free to respond here or ping me on chat. Thank you!

Looks reasonable to me, thanks for making the change!


Add support for directory descent

Allows the user to configure the max depth of descent into sub-directories desired.
Default depth is 0 to maintain current behavior.
Setting the directory descent depth to > 0 will enable recursively descending into sub-directories until either there are no more sub directories, or the directory descent depth is met.

Will solve #9


Stats calc tool

This PR adds a generic stats-calculation tool utilizing the PixelStatisticsCalculator.
Related #2542

Currently fails to read prod5 files (which have no EFFECTIVE focal length information).

prod 5 files should have effective focal length

In the output, how can I tell what column was aggregated? It is always named "statistics" and there is no metadata in the group or tables that contain that information. Wouldn't it be better to name the group like monitoring/statistics/{input_column_name}?

The former (Unexpected exception) should e caught and raises as a ToolConfigurationError, so the user gets a nice message. And please explain in the message what parameters controls this,  i.e. say Change --StatisticsAggregator.chunk_size to decrease this.

This is solved now. I added also a test for it.
I do not know why it's raised this UnclosedFileWarning. Is this related to my changes or a more general warning?

In the output, how can I tell what column was aggregated? It is always named "statistics" and there is no metadata in the group or tables that contain that information. Wouldn't it be better to name the group like monitoring/statistics/{input_column_name}? (i.e. maybe set the default of output_column_name to be the value of input_column_name? And also add the input_column namein the output table's metadata (table.meta['input_column_name']=input_column_name)

This is a good point. I think we need to metadata here. However, in the current base functionality the input_column_name is either image, peak_time, or variance. So the input_url  is also needed in the metadata to understand which quantities were actually aggregated. I'm adding this information to the metadata. The output_column_name is needed to configurable and organize the different output tables. (Maybe we should rename this config parameter to output_table_name to be more correct).
So in the end the user should put in the config e.g.:
TableLoader.input_url: "path/to/pedestal.dl1.h5"
input_column_name: "image"
output_table_name: "pedestal"
Would this sounds good to you?
--
Another point you raised about the tree schema. I think it should be monitoring/telescope/... since it is telescope-wise monitoring data.


PGP key expired

Hi,
when I try to install on Arch using makepkg, it fails the signature check using git log.
It looks like your PGP key expired 2019-01-13. If you still own this key could you update the expiration date and re-publish?

It looks like this issue has been resolved on the git master branch a few days ago. Would you mind to recheck?

Looks like makepkg finishes now, cheers!


Bug: High contrast icon is used when high contrast isn't enabled in Windows

Description
I received an upgrade notification, so I upgraded and the icon seemed to have changed, but it was not colored.
The previous v2.5.21 had a blue icon similar to Windows 11 Explorer.

Steps To Reproduce
Upgrading from v2.5.21
Requirements
To be an orange-based icon.
Files Version
<IP_ADDRESS>
Windows Version
10.0.25987.1000
Log File
debug.log

Thanks for the report, I've added this to the project board.

This seems to be an issue in light mode, I'll take a look.

I identified the issue and a fix will be released soon.


Explain extra parameter added for brief publishing

It was raised in another analytics story on the supplier app that adding a query parameter to requests just for analytics leaves no explanation for those reading the code in future why it's there.
This adds a comment to explain the reasoning behind the changes made in #331

:+1:


Move ViewModel code to its own file

Some ViewModels can get pretty large. Can we move the viewmodel code to it's own file when generating a new modlet?

I don't think we need to add this to the generated modlet. If your viewmodel gets too big you can always create a separate file for it.


Possible fix for HEAD requests

This is in reference to #54. I am not sure if this is the right place to do so, however.

I think the best place for this code would be somewhere both the cache and non-cache paths could call it, see the response created in Cache::response_with_cache.

@reem I updated the code to use the response_with_cache method. If it looks good, I will squash.

@reem I can close this PR if it's gone stale. The issue it references is still open.

Rebased and squashed old commits.

@gsquire Thank you for being persistent with this. I'm going to help you merge this.  :)

@Hoverbear Thanks for the comments. I was first learning Rust when I made this PR so that showed :)
I can squash when you give me a thumbs up.

@gsquire LGTM. :) Thanks so much.


Ethernet Data Logger (DLS-L) Support

Hi
When trying the register_scan.py I get following error (verbose=1):
RECD: a5 17 00 10 45 03 00 XX XX
frame_len does not match payload_len.
I played a bit with the code line:
modbus = PySolarmanV5("192.168.xxx.xxx", 1XXXXXXXXX, port=8899, mb_slave_id=1, verbose=1)
Changing IP address and port number gives me other errors, so I presume these are OK.
Modifying serial number (added one) only changes the second last byte in the response
Modifying mb_slave_id doesn't change anything
Kind regards
LDos

Hi Jon
Thx for the response. I added the error correction, but this gives me an error
I added some debug print statements to notice that line 219
frame_len = frame_len_without_payload_len + payload_len
causes the index out of range. The response I get seems to short?
Scanning input registers
SENT: a5 17 00 10 45 00 00 c5 2d 61 72 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 04 75 46 00 01 ca 13 d1 15
RECD: a5 17 00 10 45 03 00 c5 02
frame: b'\xa5\x17\x00\x10E\x03\x00\xc5\x02'
framelen: 9
payloadlen: 23
frame_len does not match payload_len.
corrected framelen: 36
Traceback (most recent call last):
  File "D:\_prj\solis\solismon3\register_scan.py", line 27, in <module>
    main()
  File "D:\_prj\solis\solismon3\register_scan.py", line 11, in main
    val = modbus.read_input_registers(register_addr=x, quantity=1)[0]
  File "D:\_prj\solis\solismon3\pysolarmanv5.py", line 310, in read_input_registers
    modbus_values = self._get_modbus_response(mb_request_frame)
  File "D:\_prj\solis\solismon3\pysolarmanv5.py", line 264, in _get_modbus_response
    mb_response_frame = self._send_receive_modbus_frame(mb_request_frame)
  File "D:\_prj\solis\solismon3\pysolarmanv5.py", line 259, in _send_receive_modbus_frame
    mb_response_frame = self._v5_frame_decoder(v5_response_frame)
  File "D:\_prj\solis\solismon3\pysolarmanv5.py", line 224, in _v5_frame_decoder
    v5_frame[frame_len - 1] != int.from_bytes(self.v5_end, byteorder="big")
IndexError: index out of range

What is strange: a random serial number for the logger gives me the same output. Nevertheless I checked the number several times with the one on the soliscloud site and on the stick itself.
the stick is a wired one:
model: Solis-DLS-L
FCC ID: 2AWEB-LAN-STICK
Kind regards
Lieven

I've just discovered this :)
I got an error with struct.error: unpack requires a buffer of 2 bytes
SENT: a5 17 00 10 45 00 00 71 19 8e f3 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 04 77 2c 00 01 eb b7 c4 15
RECD: a5 db 00 10 15 00 84 71 19 8e f3 02 01 53 c7 0b 00 19 22 00 00 c1 36 96 62 01 04 c8 00 00 00 00 00 00 00 00 2a f8 03 e8 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 07 01 00 00 00 00 00 00 00 00 00 27 34 e0 09 7c 03 37 00 00 06 fa 00 21 01 fd 00 02 00 01 00 6b 09 75 00 5a 00 64 00 63 13 b3 00 00 00 00 00 fa 00 00 00 00 01 58 00 28 00 00 00 0a 00 00 08 52 00 00 00 00 00 00 00 00 00 dd 00 00 00 00 00 01 00 00 01 6e 00 18 00 1b 00 00 01 d1 00 07 00 23 00 00 06 00 00 18 00 40 00 00 04 08 00 5d 00 10 00 00 0b 13 00 3e 00 8a 00 00 00 00 08 35 00 00 00 46 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 1d 7d e6 15
Traceback (most recent call last):
  File "register_scan.py", line 27, in <module>
    main()
  File "register_scan.py", line 11, in main
    val = modbus.read_input_registers(register_addr=x, quantity=1)[0]
  File "/home/barney/.local/lib/python3.8/site-packages/pysolarmanv5/pysolarmanv5.py", line 224, in read_input_registers
    modbus_values = self._get_modbus_response(mb_request_frame)
  File "/home/barney/.local/lib/python3.8/site-packages/pysolarmanv5/pysolarmanv5.py", line 179, in _get_modbus_response
    modbus_values = rtu.parse_response_adu(mb_response_frame, mb_request_frame)
  File "/home/barney/.local/lib/python3.8/site-packages/umodbus/client/serial/rtu.py", line 190, in parse_response_adu
    function = create_function_from_response_pdu(resp_pdu, req_pdu)
  File "/home/barney/.local/lib/python3.8/site-packages/umodbus/functions.py", line 138, in create_function_from_response_pdu
    return function.create_from_response_pdu(resp_pdu, req_pdu)
  File "/home/barney/.local/lib/python3.8/site-packages/umodbus/functions.py", line 911, in create_from_response_pdu
    read_input_registers.data = list(struct.unpack(fmt, resp_pdu[2:]))
struct.error: unpack requires a buffer of 2 bytes

I'm going to tack on to this issue, because I have the same error as the OP:
The serial number of my logger starts with 19xxxxxxxx
The settings in /config_hide.html exactly match the expected state mentioned in other issues.
The scan.py correctly discovers my logger when ran from the same subnet:
root@wdullaer-XPS-13-9380:/# python scan.py
{'ipaddress': '192.168.xxx.xxx', 'mac': '34xxxxxxxxxx', 'serial': '19xxxxxxxxxx'}

The firmware reported by the html status page is: ME_0C_0501_5.08
When I run the basic client I get the following error:
root@fe5ea366462e:/# python basic.py
DEBUG:pysolarmanv5.pysolarmanv5:SENT: a5 17 00 10 45 00 00 5e a2 58 72 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 04 80 fe 00 06 38 38 31 15
DEBUG:pysolarmanv5.pysolarmanv5:RECD: a5 17 00 10 45 03 00 de 02
DEBUG:pysolarmanv5.pysolarmanv5:frame_len does not match payload_len.
Traceback (most recent call last):
...

Essentially the first 5 bytes are an echo of the input. The 03 00 feels like an error code and the de 02 bytes like a frame footer.
When I intercept the data that gets sent to the solarman platform, the data does seem to follow the format from other solarmanv5 loggers.
If I replay some of the data the solarman server sends to the logger, it will echo more bytes back before it starts with the error code.
My intuition tells me some of the 'signing' or 'preamble' of the messages the 8899 server expects are slightly different. Because it does respond to the discovery messages, I'm guessing the changes will be rather small.
Maybe that means support for this logger is outside of the scope of this library, but if you have any resources to share I'd be very grateful. For example: how did you find the discovery UDP protocol?

Hi @wdullaer,
I have documented the protocol as best as I understand it here. The response frame frame should start with A5 and end with 15. The second and third bytes denote the payload length (0x0017 in this case, little endian).
I am really not sure what is going on with the response you have received because it makes no sense. Even if the frame length matched the reported payload length, the control code field is also wrong. I suspect the issue is with the data logger version. My data logger is running firmware MW_08_512_0501_1.82 (S/N starting with 405xxxxxxx), and it works. Perhaps contact Solis or Solarman to see if there is a firmware update available for your data logging stick?
As for the scan utility, this is a feature of the HF-A11 SOC used by Solarman data loggers. I first came across it here.
Regards,
Jon

Thanks for the links.
After refreshing my memory on modbus again, the 03 in that frame is likely the error code and the two bytes after the crc16.
However, it does indeed feel like the response payloads are weirdly truncated. It might be possible the server code on this firmware is bugged. I'll see if an updated version exists.
If it's all the same, I'll report my progress here: more people are likely to see it here than if I were to put it up on my blog.

@wdullaer Are you using a DLS-W (WiFi) or DLS-L (ethernet)?

I have a DLS-L (ethernet).
The firmware has all the wifi options as well, I never tried them out.

That's the same firmware version my stick is currently running.
I am going to try to flash the latest firmware found on the solis support portal when I get some free time later this week.

Is there any chance that the DLS-L doesn't actually require Solarman V5 encapsulation at all? Have you tried querying the device on port 8899 using Modbus TCP?
@drsmarsden Seemed to get pysolarmanv5 working on his DLS-L. See issue https://github.com/jmccrohan/pysolarmanv5/issues/7

Hi
I found my DLS-L would not accept TCP connections from HA , so I botched the code to stop the server initiating connections , just to wait for the DLS-L to initiate the connection.
I monitored the traffic using Wireshark and found that the DLS-L would send sever unsolicited messages (some short) on a regular basis. I think one is a keep alive and one a status message
I think what you are seeing is one of those witch will not decode. I just ignored short messages ,  you then get the reply to the request which does decode. The problem I then had was deciphering the data, some values where correct other garbage. Ginlong will not release the packet definitions, so I gave up.
I switched to using https://github.com/hultenvp/solis-sensor which talks to soliscloud.com
it worked 1st time , and populates the energy panel ok.
I have fixed a couple of issues with that code (in my copy) , but they have not been integrated back yet.
The HA guys are now working on a version of Solis support for /core.
I have just looked, I did not keep the DLS-L frig that I did
Stuart

The HA guys are now working on a version of Solis support for /core.

Do you have any more info on this?

Is there any chance that the DLS-L doesn't actually require Solarman V5 encapsulation at all? Have you tried querying the device on port 8899 using Modbus TCP?
@drsmarsden Seemed to get pysolarmanv5 working on his DLS-L. See issue #7

This did it, thx!
import socket
from umodbus import conf
from umodbus.client import tcp

# configuration
CFG_IP   = 'xxx.xxx.xxx.xxx'
CFG_PORT = 8899

# address + length
MSG_ADDR = 33022
MSG_LEN  = 10        # max recomended data frame 100 bytes (50 registers)

# Enable values to be signed (default is False).
conf.SIGNED_VALUES = True
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((CFG_IP, CFG_PORT))
# Return a message or Application Data Unit (ADU) specific for doing Modbus TCP/IP.
message = tcp.read_input_registers(slave_id=1, starting_address=MSG_ADDR, quantity=MSG_LEN)
s = list(message)
adu = {"msglen": len(message), "transaction ID": s[:2], "protocol ID": s[2:4], "length": s[4:6], "unit ID": s[6:7], "PDU": s[7:] }
#print(adu)
## prints: {'msglen': 12, 'transaction ID': [237, 3], 'protocol ID': [0, 0], 'length': [0, 6], 'unit ID': [1], 'PDU': [4, 130, 220, 0, 100]}
## In words: Request with transaction ID xxx for slave 1. The request uses Protocol ID 0, which is the Modbus protocol
## The length of the bytes after the Length field is 6 bytes. These 6 bytes are Unit Identifier (1 byte) + PDU (5 bytes)

response = tcp.send_message(message, sock)
print( response);
# prints [22, 9, 8, 14, 11, 31, 0, 0, 2491, 0]
# year, month, day, hour, minute, secons, ...
sock.close()

So I can get the timestamp from the inverter. This does not match an address table I have, but now I can go further...

Found an address list: https://www.scss.tcd.ie/Brian.Coghlan/Elios4you/RS485_MODBUS-Hybrid-BACoghlan-201811228-1854.pdf
My readings seem to be OK
Thanks everyone for the help - next step: implement this on ESP32 (ESPhome) :)

@LDossche @wdullaer Thanks for this. I'll update the documentation referencing this issue.

Thanks for all the hard work done here
I was wondering if there is a way to set/send the settings to the inverter?
Funny how HA works well with the Ethernet connector and has trouble with the WiFi version.
I have HA to read all settings from my Solis hybrid inverter & I can also SET thing like battery start & end charge times but I would like to do all this from python.
Manage to get python to read from the inverter thanks to the above code, not tested too much as really need to set the values.
CFG_IP   = '<IP_ADDRESS>'
CFG_PORT = 8899
#43143 to 43146   start hour & mins & end hour & mins
address + length
MSG_ADDR = 43143
MSG_LEN  = 10        # max recomended data frame 100 bytes (50 registers)
Enable values to be signed (default is False).
conf.SIGNED_VALUES = True
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((CFG_IP, CFG_PORT))
Return a message or Application Data Unit (ADU) specific for doing Modbus TCP/IP.
message = tcp.read_holding_registers(slave_id=1, starting_address=MSG_ADDR, quantity=MSG_LEN)
s = list(message)
adu = {"msglen": len(message), "transaction ID": s[:2], "protocol ID": s[2:4], "length": s[4:6], "unit ID": s[6:7], "PDU": s[7:] }
response = tcp.send_message(message, sock)
print( response);
[3, 1, 5, 58, 0, 0, 0, 0, 0, 0]
start 3:10 & end 5:58   - get some odd results if I shorten the msg-len but investigate later
sock.close()`


Refactor Neato botvac components as a vacuum

Description:
Refactor Neato botvac components as a vacuum
A switch is still use to enable/disable the schedule
Pull request in home-assistant.github.io with documentation (if applicable): home-assistant/home-assistant.github.io#3674
Checklist:
If user exposed functionality or configuration variables are added/changed:

[x] Documentation added/updated in home-assistant.github.io

If the code does not interact with devices:

[x] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[x] Tests have been added to verify that the new code works.

As future improvment, you can remove the switch and make platform service calls for that function or maybe it make sense to add it to base vacumm

This PR should have included a section for the breaking changes section of the release notes.


Bump to moc 0.7.5

Update: this is blocked by a regression that is patched by https://github.com/dfinity/motoko-base/pull/487.


Python GUI editor?

Hi there,
Is it possible to edit a bim file using this library through a GUI (e.g. you have file.view(), maybe there is something like file.edit())?
Just thought I'd ask if this is currently possible (or even theoretically possible - I'm happy to have a go at adding this feature if it is possible).
Thanks for any help, and for this amazing lib! :)

Hello :)
There is no such option currently. Is it possible: it depends on what you want to edit inside a file and how :)

Thanks for the response and info @paireks.
I'm thinking it would be great if we could do something like edit a 3D model in e.g. the Plotly view (e.g. move objects, stretch/shrink/drag corners etc.), and the export this to a new .bim file. Not sure if this is possible/how this might work?
Thanks again!

I don't know if it would be possible specifically in Plotly view.
However, such functionality anywhere would be great. The tricky part is it's mostly the UI part which is a bit of a challange to work natively with .bim file. Cause even if someone tries to do it e.g. with Three.js, he can move objects on a Three.js scene, but doing it natively with a file means all the actions (such as moving) should not only move element on scene, but also change e.g. "vector" of a moved object in a file. It can be a bit tricky. You can check https://plainview.fi/ from Slava, he achieved something like this, but with landXML :)
With .bim there was only prototype of properties change from Ilya: https://www.linkedin.com/posts/iliya-muzychuk-122b0a58_rhino3d-sketchup-digitaltwins-activity-7064728795601137666-wLHu?utm_source=share&utm_medium=member_desktop

Thanks for all the information.
Another idea: I wonder instead of being able to edit through a traditional CAD GUI, we could instead create a tool to generate a required dotbim file/3D model through "prompts" (i.e. like ChatGPT). To edit, you edit your prompt and generate a new dotbim/3D model, and then use the existing Plotly viewer to analyse it. I'd have to think a bit more about this (e.g. what the training/fine-tuning data might look like 🤔), but I welcome any thoughts you have on this.

It should be possible, as I've seen multiple times mesh generations through prompts, like this for instance:

https://github.com/openai/shap-e
https://github.com/threedle/text2mesh

They provide geometries only though, without a property set probably.


Adding the -iree-native-bindings-support flag to iree-translate.

Adding the -iree-native-bindings-support flag to iree-translate.
This wraps entry points in functions that take/return !hal.buffer_view.
To allow these at the frontend it introduces a hal.tensor.cast that
has been missing for awhile (it'll make it easier for custom dialects
to interop with HAL types) and tries to plumb it through.
Mostly seems to work but there's no runtime code that can call into it yet.
To emit code supporting this pass -iree-sip-bindings-support=false and
-iree-native-bindings-support=true (the two cannot coexist right now).
The intent is that we switch to these native bindings from SIP, remove the
iree.module.export attribute (all public functions will get exported), and
remove the PublicAbiGeneration.cpp from the compiler entirely. There's
some related code to that for mapping ABI types and doing some type
coercion that can also go away once it's gone.
Went with the "native ABI" naming as we can support many ABIs - tflite
is one of them, and if we shim into any other frameworks we can match
their ABI as well. By having the ABI choice be dynamic we can also version
the ABI and emit wrappers for multiple ABIs/versions into the same
module during transitionary periods.

(I want to land this but also want a kokoro run first... and kokoro is being it's kokoro self... will keep trying and hope it triggers eventually)


UnauthorizedException is back

The UnauthorizedException is back like in the previous issue. Unclear is yet why, but apparently not everyone is suffering from this case, as noted in https://community.home-assistant.io/t/lovelace-postnl/112433/240
I tried to debug, but having a hard time to track my requests on an android device.
The earlier mentioned user agents here don't solve the problem. So it's unclear if it's related to the header.
cc @IcyPalm

Yeah... PostNL made it a LOT harder. I appears that they are using: Akamai Bot Manager
Based on the X-acf-sensor-data header

I'm not sure if/how we are able to bypass this.

Damn... That sounds like they are actively blocking us....

USPS had the same case: https://github.com/home-assistant/home-assistant/issues/13688

Damn... That sounds like they are actively blocking us....
It's a possibility, if any PostNL devs are reading this: Let's have a beer and talk this over 🥂

I wonder if we set the update time to once a day, if we are being "unblocked" after like a week or so. Since it took some time for some people to get blocked.

I got around it.
Make the following changes in the code:
Change:
AUTHENTICATE_URL = BASE_URL + "/mobile/token"

to
AUTHENTICATE_URL = BASE_URL + "/web/token"

and change:
"client_id": "pwAndroidApp",
```

to

"client_id": "pwWebApp",

You are now using their web login, which still gives you an access key which also works with mobile end points.

We have to check whether that is in the design specs of Homeassistant though, since it's borderline webscraping.
But I like the idea, I was just fiddling with the web api myself

Well, it's still an API. So not really scraping an HTML page. On the other hand, PostNL didn't build a "bot protection" without reasons I suppose.

Well I still have trouble to login. I changed the lines as you said, but still received an 403 http error :-(

I am working on a fix, currently trying out new ways to interface with the multiple endpoints

Did anyone have contact with PostNL regarding the API implementation? Possibly we could start the discussion and also discuss items like refresh time with them.

Tried through their webcare team, but failed. Was thinking about reaching out to them through Linkedin, so let me see what I can do.

I got around it.
Make the following changes in the code:
Change:
AUTHENTICATE_URL = BASE_URL + "/mobile/token"

To:
AUTHENTICATE_URL = BASE_URL + "/web/token"

Change:
"client_id": "pwAndroidApp",

To:
"client_id": "pwWebApp",

You are now using their web login, which still gives you an access key which also works with the mobile end points.
Potentially all end points could be rebuild to the web interface even....

Works for me!

I got around it.
Make the following changes in the code:
Change:
AUTHENTICATE_URL = BASE_URL + "/mobile/token"

To:
AUTHENTICATE_URL = BASE_URL + "/web/token"

Change:
"client_id": "pwAndroidApp",

To:
"client_id": "pwWebApp",

You are now using their web login, which still gives you an access key which also works with the mobile end points.
Potentially all end points could be rebuild to the web interface even....

Works for me!

Not for me, still got 403 error.

I got around it.
Make the following changes in the code:
Change:
AUTHENTICATE_URL = BASE_URL + "/mobile/token"

To:
AUTHENTICATE_URL = BASE_URL + "/web/token"

Change:
"client_id": "pwAndroidApp",

To:
"client_id": "pwWebApp",

You are now using their web login, which still gives you an access key which also works with the mobile end points.
Potentially all end points could be rebuild to the web interface even....

Works for me!

Also does not work for me.

Life got in the way, I had less time this week than expected.
I probably won't have any time until sunday/monday to work on this with the attention it needs

This works:
curl -i -s -k -X $'POST' \
    -H $'Host: jouw.postnl.nl' -H $'Accept: */*' -H $'Accept-Encoding: gzip, deflate' -H $'Accept-Language: nl' -H $'Content-Length: 100' -H $'Content-Type: application/x-www-form-urlencoded' \
    --data-binary $'\x0d\x0a\x0d\x0aclient_id=pwWebApp&grant_type=password&password=yoursecretpassword&username=some%40mailaccount.com' \
    $'https://jouw.postnl.nl/web/token'

So some headers are not necessary...
Also saw that the iOS app has the X-acf-sensor-data header, so they're using the Akamai Bot Protection

I got around it.
Make the following changes in the code:
Change:
AUTHENTICATE_URL = BASE_URL + "/mobile/token"

To:
AUTHENTICATE_URL = BASE_URL + "/web/token"

Change:
"client_id": "pwAndroidApp",

To:
"client_id": "pwWebApp",

You are now using their web login, which still gives you an access key which also works with the mobile end points.
Potentially all end points could be rebuild to the web interface even....

Where (which file / code) do I need to change this?

I tried the suggestions from above but with no luck.. it's still not working for me.
I have opened a webcare ticket with postnl about 1.5 years ago with a request for a decent API or some form of IFTTT integration.. but they never got back to me. Everytime I ask them the ticket is still pending.

I've also set the scan_interval to max once a day for now. Not sure if that solves the issue, together with the above mentioned work-around.

Still working like a charm here...
Setting the scan_interval most likely helpes keeping the relationship with postnl in good order.
But once a day, guessing we might miss out some alerts. Do we know for sure at what time postnl updates the data?

Okay it is working. But how can I get the latest python-postNL-api to work with LoveLace from https://community.home-assistant.io/u/ptnijssen ? Is there a recently updated howto?

@0x00-0xFF, @peternijssen someone who would like to do a PR with the discussed changes? I could publish it asap and let's get it to HA core.

Done

Thanks @0x00-0xFF, it has been published on PyPi as version 1.2.3.
@peternijssen, @IcyPalm, would you like to try it out? Let's have it thoroughly tested and get it back in HA. For me it works with those changes.

Good work guys!
Anyone been in touch with PostNL yet?
Any expectations when this could be fixed in HA-core or in a HA-Custom- Component?
I really miss my PostNL info in HA

You could load it as a custom component, the fix is working for me. Clone this repo and add it to your your custom components and let us know if it's working for you as well.

@0x00-0xFF Do you mean to load this new version of the api as a custom component? How should we do that? Or copy the postnl component and adjust it to use the new api?

As you can see @iMicknl it isn't working for me. 😢

I have also tried to install the lib through pypi and that isn't working either. Same error message.

Having the same issues with the postnl-api. Saw that the PostNL website uses HTTP/2.0, so I gave that a go with the hyper Python package. To no avail...

It definitely is working for me though, a few days ago I had to fix the code again -> https://github.com/iMicknl/python-postnl-api/issues/20

I tried again:
$ pip3 install postnl-api
$ python3 -m postnl_api.test_postnl_api<EMAIL_ADDRESS>PaSsW0rdwithlettersandnumbersonly

Error again. Tried with my own internet, a VPN server in NL and one abroad.
If it is working for some, I can only assume blocking takes place at the user level.
I created a completely new account and tried again. Same error.

My latest fix in PR20 is not merged yet, and also not on pypi.
https://github.com/iMicknl/python-postnl-api/pull/22

https://github.com/iMicknl/python-postnl-api/issues/17#issuecomment-562931289:

Clone this repo and add it to your your custom components and let us know if it's working for you as well.

I'm willing to give it a try, but how should I clone the API itself (not the postnl HA component) as a custom component?


Sign Form is not responsive

Feature idea.
The sign form is not responsive the form looks very weird on a big device
Describe the solution you'd like
I want to make the sign form responsive, to improve the user experience as well as the UI of the website

Hi! I would like to contribute to this issue under the GSSoC'23. It seems interesting and aligned with my skills. I would be happy to take it up and work on this project.
Kindly provide me the opportunity to work on the issue.

please assign me this issue.


[Feature Request] sudo rtv used public key for ssh authentication

Hey I don't even know if this is possible, but would it be possible for users to setup a public key and use ssh authentication for login. If this works you wouldn't need to use a web browser.

Unfortunately, part of OAuth is that you need to make an HTTPS connection directly to the reddit server in order for them to send you an auth token. AFAIK they don't allow anything other than typing in your username / password into their web form. The good thing is that once you have done this once, the auth token will be valid forever as long as you don't logout.


'marisa_trie.Trie' object does not support item assignment

How to deal with this problem?

Please share a minimal reproduction code and the full error.

The following example：
 if k not in self.trie_ or self.trie_[k][0] < F:                     self.trie_[self.key_(line[0])] = (F, line[2])                 self.trie_[self.rkey_(line[0])] = 1
I wrote this, but he got an error

I want to use marisa_trie instead of datrie, but I found that some methods do not support it


Error in demo

I copy and paste the demo (editing the token) but the map is blank, i can see the flutter logo
the console print this "Another exception was thrown: Couldn't download or retreive file."

I get the same exception :(

The map is black and I have a make a simular case with mapbox access token.

I copy and paste the demo (editing the token) but the map is blank, i can see the flutter logo
the console print this "Another exception was thrown: Couldn't download or retreive file."

I find a solution in this video.
https://www.youtube.com/watch?v=oc1mktx0Zn8

I can't reproduce this. can you provide a small example that reproduces the issue?

What is the solution in the video?  Or at least where in the video can I find it?
I have seen the same issue in production and found the same symptoms on Stackoverflow: https://stackoverflow.com/questions/56098042/any-clue-for-crash-report-string-couldnt-download-or-retreive-file-got-fro/57122115#57122115
This has only affected two users over the course of 5 months (I have a couple of thousand users a day, but not everyone uses the maps), so really rare issue.

I suspect this might not be an issue any more.  I just saw this happen to two more users in the past week, but this is on a piece of code which was using version 0.5.0 and it was replaced back in May already.  That is what all four users who have experienced this have in common - v0.5.0 (and three of them were using Amazon devices and one was using an iPhone).

Hmm... That sounds like there could be something causing the network request to be failing. There could be more information in the logs, but it sounds like this is happening an a flutter app that has been built in release mode?

I copy and paste the demo (editing the token) but the map is blank, i can see the flutter logo
the console print this "Another exception was thrown: Couldn't download or retreive file."

met with the same error just now..

subdomains will fix this error
new TileLayerOptions(           urlTemplate: "https://{s}.google.com/vt/lyrs=m&x={x}&y={y}&z={z}",           subdomains:['mt0','mt1','mt2','mt3'], )

I've commented on another thread I opened. One thing that happened to me was that there were 2 AndroidManifest.xml files, one for debug and one main (I assume for release).
The none debug one didn't have internet enabled, so all the tiles were downloaded from cache only (debugging the cache provider failed with no dns warning).
Adding internet to the release AndroidManifest.xml file ie https://stackoverflow.com/questions/2169294/how-to-add-manifest-permission-to-an-application
I note in the example app it has internet enabled ok.