HEX / Text Source Code Editing

Description of the enhancement requested
Environment
Version: 1.62.0 (user setup)
Commit: b3318bc0524af3d74034b8bb8a64df0ccf35549a
Date: 2021-11-03T15:23:01.379Z
Electron: 13.5.1
Chrome: 91.0.4472.164
Node.js: 14.16.0
V8: 9.1.269.39-electron.0
OS: Windows_NT x64 10.0.17763
Text / Hex Editing View
I would like to see the source that I am editing with the hex equivalent below each character like the mainframe has in it's editor.  An example is seen below.  I understand there are dump style extensions but I would prefer a hex editor like the mainframe has. There are times when coding MFS or assembler or even COBOL programs that may have embedded hex values, that are non-displayable in the source.
Hex Editing MFS Source example that has embedded hex values

MFS Source code that contains Hex values

Ich bin derzeit nicht im Büro. Ich kehre zurück am  06.12.2021.
Während meiner Abwesenheit bitte bei Postkorb SCM-Admin melden!
Die Rückmeldung bezieht sich auf ein Mail mit folgendem Thema:
[IBM/zopeneditor-about] HEX  (Issue #183)

Gesendet (c) GRZ/RSG Linz 2021 Agent 'Abwesenheit'

Raiffeisen Software GmbH
Firmenbuchnummer: 86804d, Landesgericht Linz
UID: ATU 36961105
Der Austausch von Nachrichten mit dem oben angeführten Absender via E-Mail
dient ausschließlich Informationszwecken. Rechtsgeschäftliche Erklärungen
dürfen über dieses Medium nicht ausgetauscht werden.
Correspondence with above mentioned sender via e-mail is only for
information purposes. This medium is not to be used for the exchange of
legally-binding communications.

Hi,
Personally, I think the use of hex characters directly in the sources should be avoided.
This already poses problems on the mainframe if the codepages are incorrectly declared. But things are worse with the EBCIDIC / UTF8 (or ASCII) conversions that occur for work in VS Code.
After conversion, the characters lose all meaning.
VS Code, and Z Open Editor, is a source code editor (source programs editor). It is not a data file editor. The files handled must only use displayable characters.
You must use the X'nn 'syntax in COBOL, or equivalent in other languages, to introduce hex values in the source codes.
Otherwise to see the values of hexa or non-displayable characters in VS Code, see the extension Gremlins tracker for Visual Studio Code.

Ich bin derzeit nicht im Büro. Ich kehre zurück am  06.12.2021.
Während meiner Abwesenheit bitte bei Postkorb SCM-Admin melden!
Die Rückmeldung bezieht sich auf ein Mail mit folgendem Thema:
Re: [IBM/zopeneditor-about] HEX / Text Source Code Editing (Issue #183)

Gesendet (c) GRZ/RSG Linz 2021 Agent 'Abwesenheit'

Raiffeisen Software GmbH
Firmenbuchnummer: 86804d, Landesgericht Linz
UID: ATU 36961105
Der Austausch von Nachrichten mit dem oben angeführten Absender via E-Mail
dient ausschließlich Informationszwecken. Rechtsgeschäftliche Erklärungen
dürfen über dieses Medium nicht ausgetauscht werden.
Correspondence with above mentioned sender via e-mail is only for
information purposes. This medium is not to be used for the exchange of
legally-binding communications.

Yes, on the VS Code side we are working with UTF-8 only. Conversions are handled for us by z/OSMF or RSE API. Microsoft has a Hex extension as well, but this will not help much here I guess.
I would not think that we would try this any time soon for Z Open Editor. As you might know we are also partnering with other companies for data editing for our other editors, but unfortunately VS Code support is behind.

Thank you both so much for your quick answers!  I see your point that it is a source editor.   I am still checking out why the original developers put non-displayable bytes of data in the MFS.  I will document it here if I come to understand why.  I am suspecting MFS 3270 attribute bytes but am still checking.
I also noticed when I brought my REXX source code into VS Code from the mainframe that the '|' (vertical bar), used as the logical OR operator, is not translating properly so it shows as a non-displayable character. I am also investigating to see if I should be using a different code-page other than  English 037 to ASCII 437 when downloading from the mainframe.

Hi,
For EBCDIC / UTF8 character conversions, see
The ebcdic x'CA 'character (IBM-1147) is missing after transfer with ZOWE Explorer (with encoding 1147) / soft hyphen x'AD' not rendered by Chromium

For RSE API users there is documentation on conversion options here: https://ibm.github.io/zopeneditor-about/Docs/mvs_encoding.html  If you find any conversion problems with RSE let us know here as well.

Thank you!  I just realized you had questions above that I didn't answer.
Questions:
There seems to be another problem with your configuration.
Is the encoding conversion correctly set at Zowe CLI? We are in the process of configuring zOSMF and getting ZOWE connectivity. I am currently downloading each source member with my 3270 Terminal emulator Aviva.  I will address this when we get zOSMF / Zowe configured
Is the correct encoding declared in VS Code?  Same as above
Auto detection problem of the encoding in VS Code? Same as above
Try "reopen with encoding" in VS Code. # Same as above

Hi @davidkcoblentz
This is probably the origin of the problem.
When you download files, be sure to convert encoding from the ccsid used for your mainframe (the ccsid defined in your 3270 emulator) to UTF8 (without bom).
If you use "ind$file" look at parameters of your emulator, (ftp should be a better solution).
I think you can use Zowe CLI "zftp" without z/osmf. Otherwise use client ftp of your workstation.

Great! Thanks, I will try the CLI to see.
Thank you,
@.***
From: Denis FALLAI @.>
Sent: Wednesday, November 17, 2021 12:19 PM
To: IBM/zopeneditor-about @.>
Cc: Coblentz, David @.>; Mention @.>
Subject: Re: [IBM/zopeneditor-about] HEX / Text Source Code Editing (Issue #183)
Hi @davidkcoblentzhttps://urldefense.com/v3/__https:/github.com/davidkcoblentz__;!!OgnrLJOUc50AAzM!eO7dW3UjHFRmWdayjcAlwfxP8z5TO3yfRW70m7HWON2-JVsblxlx2qEU3VngQxxUhg$
This is probably the origin of the problem.
When you download files, be sure to convert encoding from the ccsid used for your mainframe (the ccsid defined in your 3270 emulator) to UTF8 (without bom).
If you use "ind$file" look at parameters of your emulator, (ftp should be a better solution).
I think you can use Zowe CLI "zftp" without z/osmf. Otherwise use client ftp of your workstation.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/IBM/zopeneditor-about/issues/183*issuecomment-971896783__;Iw!!OgnrLJOUc50AAzM!eO7dW3UjHFRmWdayjcAlwfxP8z5TO3yfRW70m7HWON2-JVsblxlx2qEU3Vnx4k5NwQ$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/ATC2YG7H6EY7WOCIBM4LE6TUMP53LANCNFSM5H3HLS6Q__;!!OgnrLJOUc50AAzM!eO7dW3UjHFRmWdayjcAlwfxP8z5TO3yfRW70m7HWON2-JVsblxlx2qEU3VnmLjODpA$.
Triage notifications on the go with GitHub Mobile for iOShttps://urldefense.com/v3/__https:/apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675__;!!OgnrLJOUc50AAzM!eO7dW3UjHFRmWdayjcAlwfxP8z5TO3yfRW70m7HWON2-JVsblxlx2qEU3Vl-7wXefw$ or Androidhttps://urldefense.com/v3/__https:/play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign*3Dnotification-email*26utm_medium*3Demail*26utm_source*3Dgithub__;JSUlJSU!!OgnrLJOUc50AAzM!eO7dW3UjHFRmWdayjcAlwfxP8z5TO3yfRW70m7HWON2-JVsblxlx2qEU3VkGZfEsLA$.

PS :
If you want to do an FTP transfer directly with z/OS, you may encounter problems because the z/OS FTP server does not accept to switch from single-byte encoding (IBM-037) to multibyte encoding (UTF-8 / IBM-1208).
One solution would be to transfer via FTP from IBM-037 to IBM-1252, then apply a local transformation to UTF8, for example by opening the transferred file in VS Code in Windows-1252 and then saving it in UTF-8.

Hi @davidkcoblentz, @phaumer
I remain convinced that we cannot transfer a file containing non-displayable characters (binary data or packaged decimal) and apply an encoding translation to it, (go from EBCDIC to UTF8 and vice versa).
Nevertheless, the editing of (small) data files remains a common need, (for large data files it is better to turn to a specific tool which is no longer direct file editing).
For this, I made two proposals for the development of the Zowe Explorer application, which you could consider and support by voting:

Not all mvs data sets necessarily use the same ccsid #1589
Editor in hexadecimal dump mode with EBCDIC interpretation #1590

Thanks.

Ich bin derzeit nicht im Büro.
Während meiner Abwesenheit bitte bei Postkorb SCM-Admin melden!
Die Rückmeldung bezieht sich auf ein Mail mit folgendem Thema:
Re: [IBM/zopeneditor-about] HEX / Text Source Code Editing (Issue #183)

Gesendet (c) GRZ/RSG Linz 2021 Agent 'Abwesenheit'

Raiffeisen Software GmbH
Firmenbuchnummer: 86804d, Landesgericht Linz
UID: ATU 36961105
Der Austausch von Nachrichten mit dem oben angeführten Absender via E-Mail
dient ausschließlich Informationszwecken. Rechtsgeschäftliche Erklärungen
dürfen über dieses Medium nicht ausgetauscht werden.
Correspondence with above mentioned sender via e-mail is only for
information purposes. This medium is not to be used for the exchange of
legally-binding communications.

Thank you! Great ideas! I up-voted both of these.
Thank you,
@.***
From: Denis FALLAI @.>
Sent: Tuesday, November 30, 2021 11:59 PM
To: IBM/zopeneditor-about @.>
Cc: Coblentz, David @.>; Mention @.>
Subject: Re: [IBM/zopeneditor-about] HEX / Text Source Code Editing (Issue #183)
Hi @davidkcoblentzhttps://urldefense.com/v3/__https:/github.com/davidkcoblentz__;!!OgnrLJOUc50AAzM!bMroF6QitA_b3UQx_7fNhEfZHmri84vZweakpk-6_AK9iQLJJH6UzC1kCUVJtllG9w$, @phaumerhttps://urldefense.com/v3/__https:/github.com/phaumer__;!!OgnrLJOUc50AAzM!bMroF6QitA_b3UQx_7fNhEfZHmri84vZweakpk-6_AK9iQLJJH6UzC1kCUXgWSf4XA$
I remain convinced that we cannot transfer a file containing non-displayable characters (binary data or packaged decimal) and apply an encoding translation to it, (go from EBCDIC to UTF8 and vice versa).
Nevertheless, the editing of (small) data files remains a common need, (for large data files it is better to turn to a specific tool which is no longer direct file editing).
For this, I made two proposals for the development of the Zowe Explorer application, which you could consider and support by voting:

Not all mvs data sets necessarily use the same ccsid #1589https://urldefense.com/v3/__https:/github.com/zowe/vscode-extension-for-zowe/issues/1589__;!!OgnrLJOUc50AAzM!bMroF6QitA_b3UQx_7fNhEfZHmri84vZweakpk-6_AK9iQLJJH6UzC1kCUX01cOLFQ$
Editor in hexadecimal dump mode with EBCDIC interpretation #1590https://urldefense.com/v3/__https:/github.com/zowe/vscode-extension-for-zowe/issues/1590__;!!OgnrLJOUc50AAzM!bMroF6QitA_b3UQx_7fNhEfZHmri84vZweakpk-6_AK9iQLJJH6UzC1kCUVoshIBhw$

Thanks.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https:/github.com/IBM/zopeneditor-about/issues/183*issuecomment-983345264__;Iw!!OgnrLJOUc50AAzM!bMroF6QitA_b3UQx_7fNhEfZHmri84vZweakpk-6_AK9iQLJJH6UzC1kCUVPiYbskA$, or unsubscribehttps://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/ATC2YG3LZBEPNJZPXIMC5BDUOXBT5ANCNFSM5H3HLS6Q__;!!OgnrLJOUc50AAzM!bMroF6QitA_b3UQx_7fNhEfZHmri84vZweakpk-6_AK9iQLJJH6UzC1kCUXR4c7Gmg$.
Triage notifications on the go with GitHub Mobile for iOShttps://urldefense.com/v3/__https:/apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675__;!!OgnrLJOUc50AAzM!bMroF6QitA_b3UQx_7fNhEfZHmri84vZweakpk-6_AK9iQLJJH6UzC1kCUVs6fevOA$ or Androidhttps://urldefense.com/v3/__https:/play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign*3Dnotification-email*26utm_medium*3Demail*26utm_source*3Dgithub__;JSUlJSU!!OgnrLJOUc50AAzM!bMroF6QitA_b3UQx_7fNhEfZHmri84vZweakpk-6_AK9iQLJJH6UzC1kCUUaIZdE4g$.


Add khronos to gitignore in order to avoid "modified" git status

I use submodule to include DPF to my project.
DPF clones khronos in its directory, but it causes "modified" status on git.
This PR adds khronos to gitignore to avoid that status.

I'm recreating the branch to make it go to develop branch


Update package.json to solve husky bug

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

This change is included at #11


[Event Request] New IntegrationEvent in Table 38 "Purchase Header", function SendRecords

Hello,
please extends IntegrationEvent in Table 38 "Purchase Header", function SendRecords.
Both Parameters should have the VAR Property.
   procedure SendRecords()
    var
        DocumentSendingProfile: Record "Document Sending Profile";
        ReportSelections: Record "Report Selections";
        DocTxt: Text[150];
    begin
        CheckMixedDropShipment;
        OnSendRecordsOnAfterCheckMixedDropShipment(Rec);

        // Start kvs.dde
        IsHandled := false;
        OnBeforeSendRecords(Rec, IsHandled);
        if IsHandled then
            exit;
        // Stop kvs.dde

        GetReportSelectionsUsageFromDocumentType(ReportSelections.Usage, DocTxt);

        DocumentSendingProfile.SendVendorRecords(
          ReportSelections.Usage.AsInteger(), Rec, DocTxt, "Buy-from Vendor No.", "No.",
          FieldNo("Buy-from Vendor No."), FieldNo("No."));
    end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


Remove snapshot repos

Now that we have milestones for everything, snapshots are no longer required. Kept as comments for the next major changes.

It is possible, the difficulty being that it's defined in several places. The issue with the DSL is when the repo is added, though.


AzureStorageExplorer keeps crashing randomly

My azure storage explorer keeps crashing randomly. I have observed that this happens when I have copied some text from say like xls file or chrome. I am using the latest version

@Jigar191089 Which OS are you on. Also what do you mean by crashing? Is Storage Explorer prompting an error dialog and suggest you to close the app? If so, what does the error message say?

I am on WIndows 10 Enterprise 64 bit version. I observed that it doesn't crashes but get stuck and then after sometime it turns normal.

Also when this happens at that time my copy-paste stops working and it starts only after the storage explorer has returned to normal state

@Jigar191089 can you give us an example of what you are copy and pasting? Please:

Take a screenshot of the selection in whatever app (excel or chrome) you are copying from and share that as a reply to this issue
Paste the selection as well as a reply to this issue


ticdc/pipeline: cyclic node actor refactor

What problem does this PR solve?
this is a part of table pipeline actor model refactor #3881
What is changed and how it works?

add cyclicActorContext to implement NodeContext interface, so we can resue the original code
refactor the UT case to the testing framework.

Check List
Tests

Unit test
Integration test

Release note
 `None`.

/run-all-tests

/run-all-tests

Codecov Report

Merging #3886 (c1b8f95) into master (3873d39) will decrease coverage by 1.7974%.
The diff coverage is 67.3218%.

Flag
Coverage Δ

cdc
58.8812% <67.3218%> (+0.6446%)
:arrow_up:

dm
52.2240% <ø> (-3.8106%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.
@@               Coverage Diff                @@
##             master      #3886        +/-   ##
================================================
- Coverage   57.0741%   55.2766%   -1.7975%
================================================
  Files           478        486         +8
  Lines         56551      60038      +3487
================================================
+ Hits          32276      33187       +911
- Misses        20978      23486      +2508
- Partials       3297       3365        +68

[2021-12-28T07:13:24.025Z] FAIL	github.com/pingcap/tiflow/cdc/processor/pipeline [build failed]

/merge

/run-kafka-integration-test


PDBClustering loading issue

@staszekdh
In
lpdb = PDB(db_path=db_path, plugins=['PDBClustering', 'ECOD'])
Out
/temp/sdunin/miniconda3/envs/jup/lib/python3.10/site-packages/localpdb/utils/prot.py:60 - Passing a set as an indexer is deprecated and will raise in a future version. Use a list instead.
/temp/sdunin/miniconda3/envs/jup/lib/python3.10/site-packages/localpdb/plugins/Plugin.py:40 - Loaded version of the plugin 'ECOD' (20220613) does not match the localpdb version (20220916). Data may not be fully accurate.

In
lpdb.load_clustering_data(redundancy=30)
Out

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Input In [5], in <cell line: 1>()
----> 1 lpdb.load_clustering_data(redundancy=30)

File /temp/sdunin/miniconda3/envs/jup/lib/python3.10/site-packages/localpdb/plugins/PDBClustering.py:65, in PDBClustering.load_clustering_data(self, redundancy)
     63 _, pdb_idxs = self.lpdb._get_current_indexes()
     64 if os.path.isfile(fn_mapping):
---> 65     initial_clusters = parse_cluster_data(fn)
     66     with open(fn_mapping) as f:
     67         mapping = json.loads(f.read())

File /temp/sdunin/miniconda3/envs/jup/lib/python3.10/site-packages/localpdb/plugins/PDBClustering.py:96, in parse_cluster_data(fn)
     94 for c in range(1, len(data)+1):
     95     for entry in data[c-1].split(' '):
---> 96         pdb, chain = entry.split('_')
     97         cluster_data['{}_{}'.format(pdb.lower(), chain)] = str(c)
     98 return cluster_data

ValueError: too many values to unpack (expected 2)

This is related to the addition of AF2 models to the RCSB clustering data.


Verifying Checksum Example

This PR adds a simple example of how to verify checksums that were provided via #156.
@krallin Let me know if you want to change the example in anyway. I tried to model the example after your other ones.

There's actually a template that controls this. I updated your PR with it


continuously display the TOTP setup error message

Describe the issue:
Below error message which get after enter invalid  TOTP, keep getting display.

https://user-images.githubusercontent.com/39077751/182576252-c90b53cb-5622-4f90-8a36-b3f0ada6a43e.mov
How to reproduce:

Login to myaccount and navigate to  security page
Click Authenticator app and add invalid TOTP
Close the window and click again the Authenticator App

Expected behavior:
Error message should not be there when opening window again
Environment information (Please complete the following information; remove  any unnecessary fields) :

Product Version: IS 6.0.0-beta
OS: Mac
Database: MSSQL
Userstore: JDBC

Optional Fields
Related issues:

Suggested labels:

This is expected behaviour according to the current implementation. We don't have a way to remove the error msgs coming as query param. this can be observed in other UIs as well


refactor: configuration, settings and options

Summary
A preliminary attempt to address #3297, https://github.com/biomejs/biome/pull/3310#discussion_r1659810396 etc.

[X] Use Option<T> in JsParserSettings
[ ] Use a consistent naming style, and improve the doc comments for sustainable maintenance.
[ ] Retain a single source of truth of defaults.
[ ] Make sure that caching works as expected.
[x] Try removing the Partial macro.

Test Plan
CI passes. No performance degradation.

Try removing the Partial macro.

Could you explain the reasons for removing this macro and what you are using instead?

Try removing the Partial macro.

Could you explain the reasons for removing this macro and what you are using instead?

The reason is that we almost always want to wrap options as Option<T> in our internal structure because when 1. merging configurations 2. applying language-specific options to their language-agnositc counterparts 3. applying overrides to base options, we need the None state to indicate one option will not overwrite the other.
Currently, we have some bugs when converting between concrete structs and partial structs, which may accidentally convert a None to a Some and overwrite things unexpectedly. I also find this a source of confusion.
What I'm using instead is forcing Option<T> for every option, and I plan to postpone the unwrapping/resolution to the phase when a concrete value is actually needed, e.g. when resolving the ParseOptions / FormatOptions, when printing the biome rage messages, etc.
I'm also experimenting with a model where each option carries its default value in its type directly, so that whenever we need to get a concrete value of an option, we just need .unwrap_or_default().

Thanks for the rationales.
I agree with your points. Overall, I think this will reduce the complexity of the configuration.

I have some questions regarding how we handle ignored and included. There might be some problems in our current code.
Based on my understanding of this feature, I summarized the logic as the table below. Is it ok that I refactor the code to use 👇 to determain whether a path should be included or ignored?

Ignored list \ Included list
Some(Patterns) & Matched
Some(Patterns) & Unmatched
None

Some(Patterns) & Matched
Path is ignored
Path is ignored
Path is ignored

Some(Patterns) & Unmatched
Path is included
Path is ignored
Path is included

None
Path is included
Path is ignored
Path is included

@Sec-ant yeah that's about right.
Empty lists should not run any matching.
The golden rule is that ignored has precedence over include, always.
If include isn't empty, any path that doesn't match is automatically ignored.

Based on my understanding of this feature, I summarized the logic as the table below. Is it ok that I refactor the code to use 👇 to determine whether a path should be included or ignored?

Everything looks good to me. I changed this in January / February to have a more consistent behavior. As Ema said ignore always wins on include. If include is None, then all files are included. If it is not None, then only matched files are considered. Then ignore is applied.

Everything looks good to me. I changed this in January / February to have a more consistent behavior. As Ema said ignore always wins on include. If include is None, then all files are included. If it is not None, then only matched files are considered. Then ignore is applied.
What looks wrong to you?

Oh, this is exactly how I see it, so nothing looks wrong to me :).
The only thing I'd like to keep you informed before doing it is that, now that we decide to always use Option<T> in this PR to refactor the options, I'd like to keep a consistent mental model across the codebase. Currently, it seems that we use Vector::empty to indicate the list is unset, I'd like to suggest that we use None for this purpose, and regard any Some values as being explicitly set, even if it's an empty vector. I'll also make sure this will not change the default behavior.

And FWIW, I'll keep working on this PR in the following days but I cannot gaurantee a specific timeframe, so this shouldn't be a blocker for any foreseeable release plans.


Update the product logo image in the UI masthead

One of the issues UXD identified was the length of the product name displayed in the masthead of the UI. Brand has approved the updated logo.

Updated.


Hacking is pretty much over now.

Assumptions

Since newest update of the game, hacks are broken due to unknown changes.

Since newest update of the game, devs decided to forbid hacking and implemented a detection more performant than the last one resulting in closing websocket upon suspicious data transfert or so with the local user (causing the game to still run but ur visual stop responding)

Visual glitch occuring due to code not being deobfuscate (see PowerOfUniverse claim to have fixed it but only selling it not giving it)

obviously you have not studied the surviv code and seen its structure, or you are just saying random shit at this point. Hacks are not over

About selling the hack, why buing something that will not work just in next update. (a few days i think)
powerofunverse did not put any prof that his hack is working, one screen with score that you can get before game stops,
second thing and im sorry to say that as i im form Poland myslef, that what he is doing is tipical for some part of Polish people who are not grow enough to share the good ionsteed of only seling it.


Missing refresh of tool setting source on mount

Changes

When mounting toolSettings in Docked or Widget form, will refresh the settings from the tool.
Added same forceRefresh fix on node changes to Widget form than always existed in Docked form
Testing

Validated cover and e2e.
Added relevant e2e test to validate that tool settings gets correctly updating when changing docked/widget.
Issue
Resolves #378

Needs a next version.md

Needs a next version.md

Thanks, I was convinced I had one !!!

@Mergifyio backport release/4.6.x


waffle.io Badge

Merge this to receive a badge indicating the number of issues in the ready column on your waffle.io board at https://waffle.io/OpenDataSTL/opendatastl.github.io
This was requested by a real person (user dacresni) on waffle.io, we're not trying to spam you.

This just modifies the readme with additional info. The webpage itself has to be modified to further use this.


Revert "ci: update release workflow ubuntu version"

Reverts fermyon/spin#871
This caused a glibc compatibility regression in Ubuntu 18.04 (among others, probably).

I can't reopen https://github.com/fermyon/spin/issues/708. Could you help to reopen it? Thank you.

Can't use automated revert because of DCO check. Replaced by #876


Create GraphQLResultNormalizer to create normalized records from GraphQLResponse

At a high level a GraphQLResultNormalizer is a component that takes a GraphQLResponse -> Set<ApolloRecords>. (ApolloRecords to be formalized in separate issue).
High level goals:

Share the same normalization process that is used on Apollo JS and Apollo iOS (motivated by a unified Apollo ecosystem / need to enable cache sharing between ReactNative an Native)

Proposed Implementation:
A BufferedResultReader can be configured with a interface I am leaning towards calling a 'ReaderShadow'. The BufferedResultReader will call the GraphQLResultReaderShadow during various parsing events (willParseObject, didParseObject). This will closely follow the methods in the iOS GraphQLResultReaderDelegate.
A GraphQLResultNormalizer is a GraphQLResultReaderShadow is a per-response object, that upon a succesful read, will contain a set of normalized records, and a set of keys for the record, or dependentKeys which can be used for queryWatching.
The GraphQLResultNormalizer should be configured with a CacheKeyResolver, that will return a unique id for a json blob, or null, if one cannot be determined.
This will only work on BufferedResultReader because so I am also proposing switching all parsing to use BufferedResultReader.
Video of iOS normalized caching impl

Closed by https://github.com/apollographql/apollo-android/pull/260


 Thing

I'm not really sure it works so I have allowed edits by maintainers

Kade Engine is the best example. Kade Dev just updates the changelog with every update. There isn't a bot to make it, you just, update it

Also I saw some stuff that is incorrect in your code, but I'll change it when the update


proposal: os: add (*File).SetReadDeadline and (*File).SetWriteDeadline

Now that the os package uses the poller for some cases, we should add three new methods to the os.File type:
SetDeadline(t time.Time) error
SetReadDeadline(t time.Time) error
SetWriteDeadline(t time.Time) error

These methods are intended to act exactly like the similar methods on the various types in the net package that implement the net.Conn interface.
This will permit programs to set deadlines on pipes, TTYs, and perhaps other kinds of I/O objects most naturally handled via the os package.
This proposal replaces the earlier proposals #21834 and #22100, which had more complex contexts that need not be considered in the consideration of this proposal.

Are there plans to support context for file operations, specifically cancellation?
Use case: FUSE filesystem server, implemented using underlying files which may be remote (as in NFS etc) or just slow (disk spun down or IO errors), kernel can cancel FUSE requests at any time, including based on end user control-C.

@tv42 I'm certainly open to it but I don't understand how to implement it efficiently.

@ianlancetaylor Heh, I hoped it would have somehow followed from deadlines.

Not sure, whether this is the right place to ask (I've mentioned that idea in #22100), but since we have already internal/poll, is it possible to also have (in addition to the Deadline methods above) a io.File method that has prototype similar to:
Async() <-chan int
This method will make read/write on io.File non-blocking and returns a notification channel (int type could be some concrete type of io package).
Where the channel is used to receive notifications, like: FD is ready to read/write/is closed (or just close(channel)).

@juliandroid From your code it looks like you basically want an interruptible read. You can get that using the SetReadDeadline in this proposal.  Combine that with a goroutine doing the read and sending the results on a channel, and I think you  have everything you want.

SetReadDeadline timeouts the read() operation after some ms, while the channel will send a message only when there is something to read. I guess that is more efficient and also you get that message immediately only when available? With SetReadDeadline you have to poll yourself.
For me, SetReadDeadline have one purpose only: to give you an option to quit the goroutine.
If I want to read big amount of data over TTY, for example > 1k and set timeout to some small value: 0, or close to 0 (so, my select reacts as fast as possible on close signal over "done" channel), I guess the SetReadDeadline will cause the Read() to terminate although currently there are data for reading on the TTY? This my cause a lot of unnecessary Read() calls.
Alternatively, using channel a signal will be send and the read() will return whole data (that is currently available) and the SetReadDeadline will be just a last line of defense against stuck reads for too long.
So, both have their purpose and it is not one or the other :)

Set a deadline of one minute and start a goroutine that reads from the TTY and sends the data over the channel.  Handle your small timeout in a goroutine that reads from a timer channel and from the TTY read channel.
Yes, your Async suggestion can be useful, but it requires a lot of mechanism, it will be used very very rarely, and the operations that it supports can be done without it.

In any case this proposal is about the deadline methods.

I think you are mostly correct, although in some cases, I might be more interested in performance then I have to set bigger timeout and then lose flexibility to terminate the goroutine quickly.
If I set the timeout to some very small value then I have to do a lot of read calls unnecessary.
Both combined seems best approach.
One way to mitigate the issue is by passing second optional argument to the Set(Read)Deadline.
I.e. to have one argument with bigger timeout value that is the maximum timeout for whole read to complete and one argument that is a SoftTimeout that will interrupt the Read() operation only if the read buffer is empty. In other words, If there are still bytes coming, in less than SoftTimeout ms then Read() continues to deliver bytes. Every time you get data in less than SoftTimeout you have again another SoftTimeout for the next batch of bytes until EOF, close() or the global timeout expires.
Maybe it is a bit too complicated, although logical and maybe faster :)

What you are looking for is the termios VMIN and VTIME values.
Let's keep this issue about the proposal deadline methods.

SGTM. I'm assuming they will return a non-nil error for non-pollable files.


feature: ability to display source file path and line number with def…

closes #344

welp #344

Thanks!


add runbook url for MDSCacheUsageHigh alert

The PR adds the run book URL for the MDSCacheUsageHigh alert. It gives detailed call to action for the customers when they see this alert.

@sp98 is this still a draft PR?

/cherry-pick release-4.15

@sp98 is this still a draft PR?

Not any more. The dependent PR got merged.

Testing Looks good.


Despawning pokemon occasionally trigger false positive respawn in same location

Checklist

[x] I have read the FAQ
[x] I have checked if this issue already exists (both open and closed issues)
[x] I am using the most recent version of pogom from the master branch
[ ] If this is an installation issue, I am posting my OS, OS version, python version, pip version
[ ] I'm posting the complete log
[x] I'm using a descriptive title for this issue

My Issue
This has happened to me twice. A pokemon will appear on the map scanner and I will go out to collect it. Soon after the pokemon spawn timer expires, the map scanner will notify me again that a pokemon of the exact same type has spawned with a new timer in the exact same location. However, when I return to the location, there is nothing to be found. Could this be a misinterpretation of stale data?

Could be a double spawn.

I thought it was a legit spawn, so I physically went back to the spawn location to search for the double spawn and it never showed up in either Sightings or in the game itself. It only showed up on the map.

Double spawns have the same encounter id. They don't show up the second time, if you've caught it during the first spawn.

I've seen this in the past - back when PokeVision was operational.

Is it possible that it's just 30 minutes spawn?

#350


Thumbnails causes whole app to crash unexpectedly

On one of our devices, we had a huge address book (approx 2500 contacts), and our app wasn't able to read that at all, always crashing inside getAll(). When we commented out https://github.com/rt2zz/react-native-contacts/blob/master/ios/RCTContacts/RCTContacts.m#L162 line, it started to work. We haven't investigated it further yet.

Has to do with this method I believe:
-(NSString *) getABPersonThumbnailFilepath:(ABRecordRef) person
{
  if (ABPersonHasImageData(person)){
    CFDataRef photoDataRef = ABPersonCopyImageDataWithFormat(person, kABPersonImageFormatThumbnail);
    if(!photoDataRef){
      return @"";
    }

    NSData* data = (__bridge_transfer NSData*)photoDataRef;
    NSString* tempPath = [NSTemporaryDirectory()stringByStandardizingPath];
    NSError* err = nil;
    NSString* tempfilePath = [NSString stringWithFormat:@"%@/thumbimage_XXXXX", tempPath];
    char template[tempfilePath.length + 1];
    strcpy(template, [tempfilePath cStringUsingEncoding:NSASCIIStringEncoding]);
    mkstemp(template);
    tempfilePath = [[NSFileManager defaultManager]
    stringWithFileSystemRepresentation:template
    length:strlen(template)];

    tempfilePath = [tempfilePath<EMAIL_ADDRESS>
    [data writeToFile:tempfilePath options:NSAtomicWrite error:&err];
    CFRelease(photoDataRef);
    if(!err){
      return tempfilePath;
    }
  }
  return @"";
}

Looks like we're writing a file each time the contact is fetched. I wonder if there is a workaround, given iOS app constraints regarding containerization.

Yes, this is a known bug.  I'm in the process of testing a work around.  You can help if you like, point your code at my fork:
"react-native-contacts": "git@github.com:npomfret/react-native-contacts.git",


C_CPP: Multilevel Inheritance

Description 📜
Explained about Multilevel inheritance with an image attached. Also, performed a program explaining the same.

Type of change 📝

[x] Documentation (Content Creation in the form of codes or tutorials)

Domain of Contribution 📊

[x] C/CPP

Checklist ✅

[x] I follow Contributing Guidelines & Code of conduct of this project.
[x] I have performed a self-review of my own code or work.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[x] I'm GWOC'21 contributor

/assign


Cannot set properties of undefined (setting '$shepherd')

hi @malthauser
i got this when try to run the shepherd tour in my vue app

Here is my configuration

Because Vue.prototype has been replaced in Vue 3 (see here). This wrapper seems not supporting Vue3 with option API, maybe that's why composition API is suggested in the document.

所以这里在Vue2中应该怎么处理呢？


Improved sliding mechanics

The purpose of this task is to improve upon the mechanic where the player slides when walking over ice. Currently, when the player stops moving, they slide for a few moments. If they try to move while sliding, they won't move until the sliding stops. The main goal is to allow for movement, without it interrupting the slide either.

Minor changes have been made to make the issue less noticeable, but would require the physics of the player movement to be completely redone to fix this effectively


Error when trying to run peft prompt tuned model in del

Describe the bug
When loading a PeftPromptTuned model and then running inference on the model, this works successfully but there is an ignored exception that occurs at the end after running inference. This occurs in the __del__ function that runs at the end when trying to destroy the object.
What I run where the input is a model tuned with caikit-nlp prior and saved to a directory outside of caikit-nlp:
model = PeftPromptTuning.load(args.model)
inf = model.run("I do not like the train. It is slow and smelly.")

Output:
<function register_backend_type at 0x148ac4311820> is still in the BETA phase and subject to change!
Loading checkpoint shards: 100%|████████████████████████████████████████████████| 6/6 [02:41<00:00, 26.98s/it]
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
{
  "generated_text": "Tweet text : I do not like the train. It is slow and smelly. Label : no complaint",
  "generated_tokens": 23,
  "finish_reason": "EOS_TOKEN",
  "producer_id": {
    "name": "Peft generation",
    "version": "0.1.0"
  },
  "input_token_count": 21,
  "seed": null
}
Exception ignored in: <function PeftPromptTuning.__del__ at 0x148a17253af0>
Traceback (most recent call last):
  File "/dccstor/anhtest/caikit-nlp/caikit_nlp/modules/text_generation/peft_prompt_tuning.py", line 146, in __del__
TypeError: 'NoneType' object is not callable

The line specified is gc.collect() here
Platform
Please provide details about the environment you are using, including the following:

Python version: 3.9.13
Caikit-nlp library version: commit on main branch 50f1023ee24150ed98a48d263cb8fcb9070db10d

Not pip installed, but using local git cloned version

Expected behavior
No exception message should be printed.
We can expect the error if there is nothing to delete. Add a try-catch for exception:
try:
   gc.collect()
except:
   do nothing

thats a bit of weird since gc / collect should be there.. I wonder if this is coming from next line, i.e torch.cuda.empty_cache since when we are not running on GPUs, then this line might behave differently and we are only catching AttributeError and not TypeError 🤔


Resolved MEW Contract Conflict Bug

Resolution for bug #138

I accidentally merged from branch issue/139 rather than master, so this pull includes commits not yet in the master. Sorry about that.
I've basically played around a lot with this issue and the best solution to me seemed to test for the binding element that doesn't like being messed with and leaving it alone.
I've tried to make the logic as extensible as possible to allow other items in the future we may wish to exempt from the DOM manipulation.

All the changes unique to this issue are found in this commit : https://github.com/409H/EtherAddressLookup/pull/146/commits/43945084505fab95c9d9b4a9407060309810b77c

Will be testing this tonight

Sorry, I got a little busy over the last couple days.
Marked this PR as a priority and will be done either tonight or tomorrow.

No probs, just whenever.

@Samyoul - can you commit to manifest.json and push the version to 1.9.1.

@409H version updated

Also, I can't replicate this anything since v1.9.0.
Yes, I noticed that too. This fix still addresses some weird behaviour with altering angular elements

With 1.9.0 I added a change that forced the regex to only be applied to textNodes, and moved new elements into slots avoiding unintended manipulation when looping over live collections of nodes/elements. Sometimes the regex got applied to element attribute elements and messed the HTML up.
I think the really bad problems were fixed with the introduction of using slot elements, previously chrome attempted to correct the misplaced elements so you couldn't actually see where the HTML was going wrong, but it affected the js behaviour.


fix xcode 9.2 support

Related issue #0
Changes:

Removed Nodes that does not have a files

Notes:

Now you don't need care about folders and groups.

I've downloaded your version and found a bug when try to create new Surf MVP module with generamba:


Its possible to choose color ?

Hi, its possible to choose the color of the shadows?
For example:
Neumorphic(
            style: NeumorphicStyle(
              firstColor: Colors.white, <----
              secondColor: Colors.black, <----
            ),
            boxShape: NeumorphicBoxShape.circle(),
            child: Icon(Icons.star_border, size: 17, color: Theme.of(context).accentColor)
)

Shadows colors are fixed as white / dark on neumorphic ''guidelines'', btw I can make it mutable for your usecase, I'll make it today

I understand, it's because black doesn't appear enough, my background color is "Color.fromRGBO(39, 42, 46, 1)"
Model:

Flutter app:

you can play with style.intensity for your case

You can only see "black" with an intensity of 1.
Only white is too "white", that's why I wanted to change the colors and add an opacity only to the white color

Done, in theme & style you have now
const NeumorphicStyle({
    this.shape = _defaultShape,
    this.lightSource,
    this.color,
    this.shadowLightColor,
    this.shadowDarkColor,
    this.shadowLightColorEmboss,
    this.shadowDarkColorEmboss,

I just published a new release 1.0.5 with this, can you try ?
just remember by default the intensity is on 0.5, and it's a percentage of the alpha applied to the shadow color,
if you set the color to #FF FFFFFF with the default opacity, the shadow color applied will be #99 FFFFFF
if you want your exact color set into style, use intensity = 1

Thank you very much really!
It works on an element with depth greater than 0, only the custom color is not taken under 0.

below 0 it's shadowLightColorEmboss

< 0

0

Sorry, I didn't understand, it's okay
It works perfectly, thank you very much 🙏

;)


feat(*): use kubernetes 1.7

This change is

Test the image by running
helm init --image quay.io/adamreese/tiller:kube-v1.7

@adamreese Did you want us to review and test this now or wait until 1.7 releases?

We chatted in the dev meeting that we were testing 1.7 so we can catch things earlier in the release cycle, so this is WIP until 1.7 is released

tested this on a v1.6.4 cluster with draft v0.5.0. No issues here!

pushed up a new test image and ready for testing

NOTE: CircleCI has not been run on this last commit (according to the GitHub green box below)

manually kicked off circleci


Feature_Request: Where Is Messi?

Describe the feature you are requesting
So under this Heading, I will show the stats of "Goals scored by Messi Against Different Nations"
Additional context(if any)
"Where is Messi?" is used by Messi Haters during FIFA worlcup 2022 after Argentina lost to Saudi Arabia (which is not considered a Top class Football Nation).
Are you contributing under any open-source program ?

Apertre

@5h0ov have a look here!
Amazing changes


Green background

How is it possible to overlay the browser with the transparency of the green?

If you're using a browser source in OBS or something along those lines, make sure you add ?transparent to the URL. cf. https://github.com/drweissbrot/cs-hud/blob/d85dcd87db009d29bac9e2c34eab927d89690ab1/readme.md
You can't just open the website in your browser and put it on top of the game, if that's what you're trying to do; see https://github.com/drweissbrot/cs-hud/blob/d85dcd87db009d29bac9e2c34eab927d89690ab1/readme.md#fullscreen-windowed for instructions.
Does this help? If not, please describe how you're trying to use it.

Will close for now, feel free to reopen if you're still having trouble :)


Reduce prose for more concise answers

Chat should make the best use of the minimal sidebar space, keeping its answers short and on point, reducing extra prose ("As an AI pair programmer …", "I apologize if my previous response...").

Another example of prose that is often unnecessary is when it starts its answer by repeating your question:
Me: How do I programmatically trigger the onChange event of a dropdown after removing an option which had been the selected option?
Copilot: To programmatically trigger the onChange event of a dropdown after removing an option which had been the selected option...


fix(proto): use buf, like we do on dymension repo

PR Standards
Opening a pull request should be able to meet the following requirements

Closes #XXX
All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow-up issues.
For Author:

[ ]  Targeted PR against correct branch
[ ]  included the correct type prefix in the PR title
[ ]  Linked to Github issue with discussion and accepted design
[ ]  Targets only one github issue
[ ]  Wrote unit and integration tests
[ ]  All CI checks have passed
[ ]  Added relevant godoc comments

For Reviewer:

[ ]  confirmed the correct type prefix in the PR title
[ ]  Reviewers assigned
[ ]  confirmed all author checklist items have been addressed

After reviewer approval:

[ ]  In case targets main branch, PR should be squashed and merged.
[ ]  In case PR targets a release branch, PR should be rebased.

@danwt the Makefile missed setting version to variables:
https://github.com/dymensionxyz/dymension/blob/983dd904dacc3b5a6ff5c3122b02fdbf450cd730/Makefile#L13-L20

Thanks for pointing that out I was in a rush
I'll fix it


Update templates

This pull request updates the templates used by the babelizer to generate projects of wrapped BMI components.
Most of the changes are in the generated documentation:

docs use the furo theme
the babelizer generates csdms-styled logos (dark and light themed) for the new project
fleshed out the documentation to include installation, usage, virtual environments, API, and how to update the project with the babelizer

I've also added a nox file and a pre-commit hooks file for running routine maintenance tasks (building the docs, checking for lint, etc.).
I've set up towncrier in the generated project as well. This may be overkill but since we consider it good practice elsewhere I figured we could try it here as well.
I removed the recipe folder that held the generated conda recipe since we don't use that anymore.

One more thing. I removed the setuptools version requirement for the build system. I had required setuptools<60 because of a problem building Fortran projects (numpy doesn't not support newer version of setuptools) but older versions of setuptools don't support pyproject.toml files. Anyway, it appears that perhaps numpy fixed the issue since we're no longer seeing those fortran build errors we had been getting.


Update to rustc 0.13.0-nightly

(hopefully) does what it says on the tin

(I would merge, but the build seems to be failing because crates.io's regex is out of date, and I don't have the permissions publish the fixed version yet.)

Thanks!


update travis ci

herp derp lets auto deploy site when merging to master

hm, i meant to post this to my fork, whoops


[STORM-2283] Fix multi threading issues in DefaultStateSerializer

Raising a separate patch for the state issue fixed in https://github.com/apache/storm/pull/1693

+1 Confirmed this works well from #1693.

@arunmahadevan
Just realized you're using Optional from JDK 8.
Given that it's a bug across 1.x version lines, could you craft a patch for 1.x? Thanks in advance.

@HeartSaVioR guava's Optional did not have a public default ctor and Kryo was throwing errors during deserialization. I will try to address it and raise a different patch for 1.x branch.

@HeartSaVioR I made some change so that it works without any changes to RedisEncoder and compatible with 1.x-branch as well. Please take a look. I also applied the same changes to https://github.com/apache/storm/pull/1693

+1 Thanks for the quick fix.


Add ADR folder and template

Changes proposed in this pull request:

Add a new subfolder to hold ADR documents, and an ADR Decision log
Add a template to use when creating new ADRs.

security considerations
~N/A~ ADR pros and cons could provide a malicious actor with knowledge of system weaknesses, so we should be circumspect in our discussion, and possibly the hold back publication until after we've implemented a solution.

@pburkholder I like that idea. There are a few more issues kicking around that might also be good as a way to road test the template.


Example code needs small changes

Example code needs small changes
It looks like the example code in the readme needs some small changes.
SamplePlayer({Key key}) : super(key: key);
Needs to become
const SamplePlayer({super.key});
and
FlickManager flickManager;
Needs to become
late FlickManager flickManager;

We updated this.


build: Use qrc file content as source

Use rcc --list instead of also adding files in the Makefile.

Rebase?

I'll rebase and try to sharpen my lame makefile skillz to make it work.

Closing since workarounds/hacks are needed to address limitation configure substitutions are not allowed in _SOURCES variables.


Placeholder appear with no value

Placeholder appear if the value is empty ('') null or undefined

Thanks for the input


Zotero: Fix to use patchelf instead of FHS Env.

Don't use buildFHSUserEnv and use patchelf to fix up the binary and libs. This should fix issue #48814.
Motivation for this change
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

Thanks for this great patch!
I tested this:

19.03: builds
18.09: builds and runs (from command line)

I encountered one issue, though:  The desktop item which is copied from the upstream source does not work.  It does not start (the exec entry is trying to be clever), and the icon is also missing (which could be a kde issue on my setup, though)

I don't use a DE, so i didn't know the desktop file was broken. It should be fixed now. Let me know if it works for you. Thanks.

It works now.

This pull request has been mentioned on Nix community. There might be relevant details there:
https://discourse.nixos.org/t/prs-already-reviewed/2617/12


Option to treat timeouts as non-failure

I am currently using nextest via miri in Wasmi to run the entire Wasm spec testsuite (which is huge) nightly. Since miri execution with all of its checks is rather slow and since we use default GitHub action runners most tests timeout. Locally I ran the entire testsuite via miri with a timeout of 4h per test case and even then I had some timeouts, so that's how bad it really is. However, this is not as bad as it sounds since even timed out tests produced a ton of checks and validated their input until their timeout.
So what would be nice is to make it possible via config to treat timed out tests as non-failures since I only really want to be notified by this CRON miri run if it actually found an actual bug.
Is this maybe even already possible? In case you do not want to implement this feature what would be the best way to handling this on my side?
Link to the cron job if that is of interest: https://github.com/paritytech/wasmi/actions/runs/7771389011/job/21192633974
Thanks!

Hi, and thanks for the feature request!
This isn't something that's supported at the moment -- timeouts with nextest are meant to be for unexpected issues.
Would you be able to add a timeout within your test logic instead?

Hmm, I see, you're actually using miri.
I guess we can add an on-timeout = "success"/"failure" config, and then you can set that in the Miri-specific config. I guess it'll also be useful for tests that are meant to be run for a while and then cancelled (I've come across a few such tests over the years)

Yes, I am using nextest via miri. And yes, even running into timeouts for these kinds of tests is already extremely valuable since the tests have checked various safety properties of the program along the way and as long as they haven't found issues before running out of time, I am happy.
Obviously it would be preferred if no test has a time out, but that is simply infeasible with the current hardware and in particular the current performance of miri with all of its checks.
I ran those miri tests locally on 12 cores via nextest and a timeout of 4 hours per test case and still some test cases ran out of time.

Yeah, makes sense!


C1 Pro installation error

I tried to integrate my C1pro camera with v.45 (I used HACS), but I get an error (installation failed).
I use Home Assistant version : core-2021.11.5 , supervisor-2021.10.8 on RP4.
This is the error log:
Logger: homeassistant.config_entries
Source: helpers/network.py:116
First occurred: 20:41:13 (1 occurrences)
Last logged: 20:41:13
Error setting up entry overkapping for reolink_dev
Traceback (most recent call last):
File "/config/custom_components/reolink_dev/init.py", line 79, in async_setup_entry
push = hass.data[DOMAIN][base.push_manager]
KeyError: 'push_manager-B0411DD68D54'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/config_entries.py", line 304, in async_setup
result = await component.async_setup_entry(hass, self)  # type: ignore
File "/config/custom_components/reolink_dev/init.py", line 88, in async_setup_entry
await push.subscribe(base.event_id)
File "/config/custom_components/reolink_dev/base.py", line 335, in subscribe
get_url(self._hass, prefer_external=False),
File "/usr/src/homeassistant/homeassistant/helpers/network.py", line 116, in get_url
raise NoURLAvailableError
homeassistant.helpers.network.NoURLAvailableError

README.


unknown ptrSize for $GOARCH "wasm"

What version of Go are you using (go version)?

$ go version
go version go1.14.6 darwin/amd64

Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/huangwutao/Library/Caches/go-build"
GOENV="/Users/huangwutao/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/huangwutao/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/y4/5jgjdbn12rg03j1__vf0mg340000gn/T/go-build891741688=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

i trying build a wasm file:

$ GOOS=js GOARCH=wasm go build -o lib.wasm test.go

and test.go source code is :

package main

import "C"

import (
	"syscall/js"
)

func main() {
	var window = js.Global()
	var cb = js.FuncOf(func(this js.Value, args []js.Value) interface{} {
		return args[0].Int() + args[1].Int()
	})
	window.Set("test", cb)
	select {}
}

What did you expect to see?
got a wasm file
What did you see instead?

# runtime/cgo
unknown ptrSize for $GOARCH "wasm"

cgo is not supported for wasm. The error could be clearer, though.
Remove the import "C". You're not using it, and that should fix the problem.

cgo is not supported for wasm. The error could be clearer, though.
Remove the import "C". You're not using it, and that should fix the problem.

is cgo will support for wasm in future?

is cgo will support for wasm in future?

very unlikely as you would need a C to wasm compiler for you C dependencies.
I'm going to close this question as answered based on the answer from @randall77 above.


Nocode based AI which does nothing

I created a nocode based  AI which does nothing. As I support FOSS, I am releasing it for everyone to use. Please use it responsibly. Here's a look at the code

No tests was found this is a fantastic PR LGTM


Add latency logging

Use the log npm package.
Store json, not CSV. For example:
{ "time": "XXXXX", "numberOfPlayers": XX, "averagePing": XX }
{ "time": "XXXXX", "numberOfPlayers": XX, "averagePing": XX }
{ "time": "XXXXX", "numberOfPlayers": XX, "averagePing": XX }

Add an environment variable check before logging (so we can turn it off as needed via an ENV variable). This is particularly important for Heroku (or other PaaS stacks where you may not have access to the file system).


undefined: uuid.Must

Hi,
I am struggelig with dep and a dependency with this library. Unfortunately it seems like renstrom/shortuuid uses uuid.Must. Is it possible to get a release with this included so Dep can be satisfied ?

@satori No rush, any idea when the next release is going to get tagged?


anko kmart fan

hey mate
love your work wanted to know how I could go about adding a device I have Anko Kmart fan.
I have queried to dp ids have worked a couple out just wanted to find out how I can go about adding it
anko fan
id 1 power true ,false
id 2 mode normal
id 3 speed 1-8
id 4 osc auto , off
id 6 timer  1-9
regards
rhys

Take a look at the custom_components/tuya_local/devices directory.  It will need a yaml file similar to the ones in there now - probably closest to the goldair_fan.yaml. Minimal would be a primary_entity of entity:fan, and each dps specified with its id, type, and name which should match the names for specific functions in custom_components/tuya_local/generic/fan.py.
It is best if you can specify all DPS in the yaml file, even if you don't know what the functions are for some of them, as it will help with device detection. With just 1 - 6, I think there might be a heater or two that clash.

Generally I would write a test for it as well, to make sure everything works as expected, the tests are in tests/devices, but to get it going quickly, it can be done later. Also, if it cannot be auto-detected due to DPS clashing with another device, I think it will need some work in the config related files, as currently they have a fixed list of options, and haven't yet been modified to take that list from the config files.

Thank for the quick reply I had stated to look if I could edit the golder fan file. I will have a bit off a play with it post back if I have success so it can be added.

Based on the information you provided so far, and the manual here: https://www.kmart.com.au/wcsstore/Kmart/pdfs/42779896_Manual.pdf
I think the following should work (filename anko_fan.yaml)
name: Anko Fan
primary_entity:
  entity: fan
  dps:
    - id: 1
      type: boolean
      name: switch
    - id: 2
      type: string
      name: preset_mode
      mapping:
        - dps_val: normal
          value: normal
        - dps_val: nature
          value: nature
        - dps_val: sleep
          value: sleep
    - id: 3
      type: integer
      name: speed
      range:
        min: 1
        max: 8
      mapping:
        - scale: 0.08
    - id: 4
      type: string
      name: oscillate
      mapping:
        - dps_val: "auto"
          value: true
        - dps_val: "off"
          value: false
    - id: 6
      type: integer
      name: timer

hey mate
thank you for creating that file. had not played with it yet also  that is the fans manual
will try the file tonight
regards
rhys

Hey mate is their a log I can look at to make sure its using the new configuration thing its trying to use the goldair one as it does strange things

There should be some indication of what is going on in the Home Assistant Core log.  It may require INFO level logging to be enabled.
Make sure you set the type to Auto, not fan.  "fan" is for the Goldair fan, definitely, but I don't think it will misdetect this one as the Goldair fan if it is on auto, more likely is one of the heaters.

hey mate
I have managed to test your file I could do everything except for speed changing and the fan freaks out when you try change.
this what I manged to grab from the logs
ogger: homeassistant.components.websocket_api.http.connection
Source: custom_components/tuya_local/helpers/device_config.py:391
Integration: Home Assistant WebSocket API (documentation, issues)
First occurred: 9:14:38 AM (1 occurrences)
Last logged: 9:14:38 AM
[2747037296] Target speed (12) must be between 1 and 8
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 185, in handle_call_service
await hass.services.async_call(
File "/usr/src/homeassistant/homeassistant/core.py", line 1491, in async_call
task.result()
File "/usr/src/homeassistant/homeassistant/core.py", line 1526, in _execute_service
await handler.job.target(service_call)
File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 213, in handle_service
await self.hass.helpers.service.entity_service_call(
File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 658, in entity_service_call
future.result()  # pop exception if have
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 760, in async_request_call
await coro
File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 695, in _handle_entity_call
await result
File "/config/custom_components/tuya_local/generic/fan.py", line 158, in async_set_percentage
await self._speed_dps.async_set_value(self._device, percentage)
File "/config/custom_components/tuya_local/helpers/device_config.py", line 228, in async_set_value
settings = self.get_values_to_set(device, value)
File "/config/custom_components/tuya_local/helpers/device_config.py", line 391, in get_values_to_set
raise ValueError(
ValueError: Target speed (12) must be between 1 and 8
regards
rhys

How are you setting the speed?  By the UI, or by a service call or Developer UI?
The range of the speed should be 1 to 8, which is translated in the UI to 0-100% (since Google, Alexa and Apple Homekit all work with percentages, but their abilities vary in dealing with smaller ranges).  Or is the range 1-8 wrong for this fan, and the 12 comes from the fan itself?

it was just in the lovelace ui i will give it a go by a service call and test all the speeds again.

its strange here is the output i get from tuya-cli when list the device .
{ '1': true, '2': 'normal', '3': '3', '4': 'auto', '6': '0' }
{ '1': true, '2': 'normal', '3': '8', '4': 'auto', '6': '0' }
but when I try set the speed with tuya-cli set command --dps 3 --set 2
i get a  Error [ERR_UNHANDLED_ERROR]: Unhandled error

Can you check if setting the speed works if you add quotes around the 2?
tuya-cli set --id xxxx -key xxxx --dps 3 --set '2'

hi mate
sorry had a busy weekend just put in that command and it timesout. i am about to read through the goldair fan you have mentioned to see if I can find anything useful
regards
rhys

its strange i can set other dps but when I try do the speed it timesout

Thanks for trying anyway.  I guess I'll push out what I have with a note in the README about speed not working for now. If anyone figures it out, I can update it later.

that is alright mate I appreciate all the effort you have put in.
regards
rhys

Sorry to reopen, not sure if this helps at all.
fan:

platform: mqtt
name: "Bedroom Fan"
state_topic: "bedfan/stat/POWER1"
command_topic: "bedfan/cmnd/POWER1"
oscillation_state_topic: "bedfan/cmnd/TuyaSend1"
oscillation_command_topic: "bedfan/cmnd/TuyaSend1"
speed_state_topic: "bedfan/stat/speed"
speed_command_topic: "bedfan/cmnd/TuyaSend4"
qos: 0
payload_on: 'ON'
payload_off: 'OFF'
payload_oscillation_on: '5,1'
payload_oscillation_off: '5,0'
payload_low_speed: '3,0'
payload_medium_speed: '3,5'
payload_high_speed: '3,11'

payload_highest_speed '3,5'
payload_highester_speed: '3.4'
availability_topic: "bedfan/tele/LWT"
payload_available: Online
payload_not_available: Offline
speeds:
- 'off'
- low
- medium
- high
- highest
- highester

That mess of numbers doesn't really help, but this did trigger me to relook at this early issue, and I realised that the speed is a string, not integer, so the config is wrong, and only works because of some fault tolerance that tries to detect quoted integers and allow them to pass as integers.

Sorry wasn't trying to confuse things. Thanks for responding though.


MR24HPCB1_checkDevID.ino example

According to User Manual, this is not a correct product ID query:
0x53, 0x59, 0x02, 0xA1, 0x00, 0x01, 0x0F, 0x5F, 0x54, 0x43}
It should be
0x53, 0x59, 0x02, 0xA2, 0x00, 0x01, 0x0F, 0x60, 0x54, 0x43

Thanks for the heads up, I've corrected it


Resize fails with large image

Resize operation can fail with a large image, even after the newly added code to check for >4GB per slice.
Failure is here:
DirectXTexResize.cpp
	HRESULT PerformResizeUsingWIC(
		hr = pWIC->CreateBitmapFromMemory(static_cast<UINT>(srcImage.width), static_cast<UINT>(srcImage.height), pfGUID,

hr returns error: Invalid Args
Format: DXGI_FORMAT_R16G16B16A16_UNORM
Slice size:<PHONE_NUMBER> bytes (still less that UINT32_MAX of<PHONE_NUMBER>.)
Source Image: 20,000 x 20,000 x 16bit x2 channels Geo .TIFF file (763mb)
Can provide the image if I can find a place to upload to.
It appears that the new 4GB check is inadequate, instead WIC internal limit may be 2GB?
This .tiff ends up taking more memory due to 2 channels (elevation + alpha) being converted to 4 channels, even so, geotiff images can be very large.

Doing some digging into the WIC source code, it looks like you are hitting this issue:
if (SUCCEEDED(hr) && (nHeight >= INT_MAX / cbStride))
{
  / / E_INVALIDARG;
}

That makes sense.
I didn't see any limits mentioned in the docs on CreateBitmapFromMemory
Perhaps normal use expects some kind of capability query before use.
I considered using a *NON_WIC flag, but I thought the code path always used WIC and failed before reaching the filter configuration. I could be wrong.


Exclude foreign keys from being used to generate prediction problems

Change "index" to "primary_key"
Fixes duplicate prediction problem generation by hashing all possible prediction problems

Still need to add a few more unit tests and assertions


Add run_exports

The run_exports feature can be used to specify the versions that are ABI compatible with the built version. Here I assume that 4.0.0 is ABI compatible to 4.2.1 and so on.

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

@conda-forge-admin, please rerender

Based on https://conda-forge.org/docs/maintainer/pinning_deps.html#specifying-run-exports

Looks good to me, thank you!

As I do not have the permissions to merge the pull request - can you merge it ?


it doesn't work with storybook

I got a lot of errors like the following in version 1.9.0:
ERROR in ./node_modules/tailwind-merge/dist/lib/config-utils.mjs 9:7-23
Can't import the named export 'createClassUtils' from non EcmaScript module (only default export is available)
ERROR in ./node_modules/tailwind-merge/dist/lib/default-config.mjs 65:15-20
Can't import the named export 'isAny' from non EcmaScript module (only default export is available)

Hey @krzysztofpniak! 👋
That's a weird bug. 🤔 Those modules are all EcmaScript modules and none of them use default exports. E.g. here is the code from those files in<EMAIL_ADDRESS>// tailwind-merge/dist/lib/config-utils.mjs

import { createClassUtils } from './class-utils.mjs';
import { createLruCache } from './lru-cache.mjs';
import { createSplitModifiers } from './modifier-utils.mjs';

// internal code

export { createConfigUtils };
//# sourceMappingURL=config-utils.mjs.map

// tailwind-merge/dist/lib/class-utils.mjs

// internal code

export { createClassMap, createClassUtils };
//# sourceMappingURL=class-utils.mjs.map

Do you have a minimal reproduction example for your bug? Might be that something in your toolchain is converting the ES modules to CJS.

Hi, I've managed to create a minimal reproduction:
https://github.com/krzysztofpniak/twMergeStorybook
all you need to do is to install deps and run:
yarn storybook or npm run storybook
the line 2 of stories/Button.jsx is causing the build problem

Thanks! Will take a look.

Just checked out your repo and it seems that the issue is that Storybook is using webpack v4 by default which is quite old and doesn't understand ESM or modern JS syntax like optional chaining.
I tried using webpack5 and then it compiled fine. So if you can upgrade to webpack 5 in Storybook, this is the best option.
Related:

https://github.com/storybookjs/storybook/issues/17200
https://github.com/storybookjs/storybook/issues/16690

Thanks a lot for your time and hits. You were right, with webpack 5 it works like a charm!


App crashes due to Out of Memory Exception with PINRemote

I am using PINRemoteImage to download images and cached them in my app. My app uses lots of images, just like instagram so the images memory increases all the time. And I have lots of crashes due to out of memory.
Is there a way to set a limit of MB to store in cache? Is there a way to free memory once it's full before the app crashes?
Thanks!

The memory cache should be emptied in case of memory warning.  I think your issue is more likely to be parts of your app that are holding onto the image references directly - like view controllers or views that aren’t being released.
Have you tried looking at the Instruments Object Allocations / Leaks tools, or the Xcode 8 Memory Graph features?  It should help find it quickly.

On Sep 8, 2016, at 11:53 AM, Rodrigo Mato<EMAIL_ADDRESS>wrote:
I am using PINRemoteImage to download images and cached them in my app. My app uses lots of images, just like instagram so the images memory increases all the time. And I have lots of crashes due to out of memory.
Is there a way to set a limit of MB to store in cache? Is there a way to free memory once it's full before the app crashes?
Thanks!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/pinterest/PINRemoteImage/issues/243, or mute the thread https://github.com/notifications/unsubscribe-auth/AAigA7mVZdTqqp0uTeoCs2KpHQFzQUOfks5qoFnEgaJpZM4J4U8w.

So I am using PINRemoteImage to download and cache progressive images in a table view.  The memory balloons massively while I am scrolling, but quickly smashes itself back down when scrolling stops.  When testing on my phone, I can cause crashes with this.  On the simulator, I can't usually get it to crash, but I get horrible stuttering in scrolling the table and have seen memory usage jump as high as nearly 1GB before dumping back down to ~200MB (which is reasonable for what I'm doing).
So I think this is not due to having multiple images cached, as much as it is an issue with the download and caching process.  Once everything IS cached, everything is great.

Yeah I do think this might be a result of your implementation and not specifically PINRemoteImage. @lukeivers if you could drop some stack traces and maybe snippets of how your cells are using image loading etc that would be great 😄

Sure thing.  I don't have any stacktraces yet (I have some configuration wrong and am not properly capturing debugging info from running it on my phone, and it doesn't actually CRASH on the simulator, just uses massive amounts of memory).
A few related notes, first:

Telling it to prefetch all the URLs for my entire table data source made it jump to ~4GB of memory consumed
Because of that, I was able to find an old issue saying to use the costLimit to keep this from happening.
Using costLimit did not, in fact, change memory usage of prefetching at all
It DID, however, change memory usage and massively improve performance in the simulator for this table.
Unfortunately, it appears to have no effect when running the app on my phone.  It still crashes.
I did catch in the device logs that it was having problems allocating memory for an image just before it crashed (I'll see if I can find the exact error message here in a bit).

Anyway, here's some code I'm using (this is literally the entire cellForRowAtIndexPath code):
let cell = tableView.dequeueReusableCell(withIdentifier: "MyCell", for: indexPath as IndexPath) as! MyTableCell

cell.imageView.pin_updateWithProgress = true

cell.selectionStyle = .none

let itemForIndexPath = myCollection.sections[indexPath.section].items[indexPath.row]
if cell.item == nil || (cell.item != nil && cell.item != itemForIndexPath) {
    cell.item = itemForIndexPath
    cell.imageView.pin_setImage(from: cell.item.escapedWideScreenImageNSURL, placeholderImage: MyTableViewController.blankImage)
}

return cell

Also, it only has the super high memory usage / crashing behavior when it is first loading in the images.  Once they are locally cached, say on disk, then it behaves just fine.  And if I scroll slowly down the page, it's laggy but it has no issues.
It appears as though it's allocating tons of memory during the initial download process that it then immediately clears out after the image has been locally cached.  Maybe this is related to the progressive jpeg code where it's trying to create the lower resolution partially downloaded images for each cell's image?
Keep in mind that every one of my images, which I have 50-60 of in the test case I'm running (manually, not as an actual UITest or whatever), is at like 4-5k x 4-5k pixels, so it's a LOT of data / pixels.

One can see here that it's consuming MASSIVE amounts of memory in CG raster data:

Ok, the last one was on my simulator, this one is from my actual phone:

One final note: I'm not actually getting a crash on my phone.  The app just disappears.  I think that the system memory watchdog is killing off my app due to excessive memory usage, as opposed to it being a genuine crash from some direct bug in the code.

Update: if I set the defaultOptions for my imageView to include DownloadOptionsSkipDecode, it reduces the performance of scrolling on my table to... well, completely non-viable BUT it does completely prevent the crash / memory issues from occurring.
I found this because I finally got some error messages in a console to appear, and they looked like this:
CGBitmapContextInfoCreate: unable to allocate  bytes for bitmap data
I got about 10-15 of them, then the app was terminated due to memory pressure.
So I dug around to find out where CGBitmapContext objects were being created, bounced up the call chain until I found out that they're only created if it's decoding in the images in the background thread.
I'm not sure if it's leaking something until the image is finally done downloading, or if whatever function it is that prevents an image from calling it's completion block if it's no longer going to be displayed doesn't prevent it from continuing to render the data it has into bitmaps for display, but it's somewhere right down in that section of code.
Specifically, in lines 100-104 of PINRemoteImage/core/PINImage+DecodedImage.m, if it follows the code path on line 101, it doesn't crash.  If it follows the code path on line 103, it does.

You may need to differently configure your .memoryCache object, e.g. set a different cost limit for the cache.
I'm not too familiar with the intended behavior here — with AsyncDisplayKit (which closely integrates with PINRemoteImage), the framework proactively frees memory for images that are a certain distance from the screen.  I believe with a traditional UIKit approach, you either need to manually clear cache elements, or rely on hitting some maximum size (even if it's actually a lot more images than you need).
The disappearing without crash log behavior is definitely consistent with memory kills, so I think you're on the right track there.  The question is how you want to free the memory as infinite scrolling proceeds (the bounding factor).

On Sep 16, 2016, at 4:44 PM, Luke Ivers<EMAIL_ADDRESS>wrote:
Update: if I set the defaultOptions for my imageView to include DownloadOptionsSkipDecode, it reduces the performance of scrolling on my table to... well, completely non-viable BUT it does completely prevent the crash / memory issues from occurring.
I found this because I finally got some error messages in a console to appear, and they looked like this:
CGBitmapContextInfoCreate: unable to allocate bytes for bitmap data
I got about 10-15 of them, then the app was terminated due to memory pressure.
So I dug around to find out where CGBitmapContext objects were being created, bounced up the call chain until I found out that they're only created if it's decoding in the images in the background thread.
I'm not sure if it's leaking something until the image is finally done downloading, or if whatever function it is that prevents an image from calling it's completion block if it's no longer going to be displayed doesn't prevent it from continuing to render the data it has into bitmaps for display, but it's somewhere right down in that section of code.
Specifically, in lines 100-104 of PINRemoteImage/core/PINImage+DecodedImage.m, if it follows the code path on line 101, it doesn't crash. If it follows the code path on line 103, it does.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/pinterest/PINRemoteImage/issues/243#issuecomment-247733191, or mute the thread https://github.com/notifications/unsubscribe-auth/AAigA6bl6YvznDagvzkIEawWzXBBilteks5qqynngaJpZM4J4U8w.

So, I think that part of the issues it that (from what I can tell) it doesn't appear to calculate the cost and validate that against the costLimit until after it has fully materialized the remote URL locally.  In the mean time, it's building the progressive images in memory (obviously), and those are consuming nearly or as much memory as the fully materialized object will in some cases.
So when you're doing progressive images, the costLimit is not really an effective method to manage memory consumption.
I really think the issue is largely hinged on progressive images.  I understand that the frameworks that layer on top of this have their own methods for managing the memory that may prevent this issue in those cases, but as a separate library overall, there seems to be something missing when it comes to managing memory capacity when downloading progressive images.
Luke

@lukeivers so it's not perfect, but there are several ways of limiting the amount of images being rendered at once:

You can limit the number of concurrent downloads with setMaxNumberOfConcurrentDownloads:completion:
You can limit the number of concurrent operations (rendering a progressive scan is an operation) with setMaxNumberOfConcurrentOperations:completion:. This is probably what you want.
You can reduce the number of scans rendered per progressive image with setProgressThresholds:completion:. The default is [0.00, 0.35, 0.65] which will result in up to 3 scans. You could reduce this by dropping one or two of those.

Let me know how that works!

I ended up actually switching over to AsyncDisplayKit (which I was mostly only avoiding because I didn’t want to rebuild my UITableViewController, and then I had to do it for other reasons), which totally fixed my issues.
I did have max concurrent downloads set.  I did not have max concurrent operations set.  However, I think the primary issue is due to the way that cancelling operations/tasks works.  It seemed like it would only allow a certain number of tasks to happen simultaneously, but as you scroll quickly, it will start and then cancel multiple tasks repeatedly.  However, each of those tasks has started downloading data for a progressive image, and when it receives the data, it checks whether it has enough to render at least one scan of the progressive image.  Each one of those consumes some amount of memory, and I don’t think it knows to not try to composite the image from the data it receives (i.e. that the task has been cancelled) until after it’s already done so.
So, if you’re on a slow connection, I think this isn’t as much of an issue, because you won’t get enough data to create even the first progressive scan with each chunk of received data.  If you have everything cached, it goes straight to a full image and doesn’t build the intermediate bitmap/contexts, so also doesn’t have issues.
But if you have a fast connection and nothing cached, and scroll a table with a lot of rows (I think I had around 50 or 60), and each image has a large pixel space (most of my images were 4000x5000 or so pixels), it exposes this as an issue.
So, not that everyone is doing that exact thing, I understand that.  But I think there’s probably a relatively straightforward method to ensure that cancelling a task guarantees that NO further operations that consume memory (creating bitmaps, etc) will take place for that task, and that would have completely solved the problem for me.
If I had time I’d dig far enough into the task management/scheduling to figure out where to put it in myself and submit a PR but the code was too dense for me to do that in the time I had available.
Luke

On Sep 21, 2016, at 4:13 PM, Garrett Moon<EMAIL_ADDRESS>wrote:
@lukeivers https://github.com/lukeivers so it's not perfect, but there are several ways of limiting the amount of images being rendered at once:
You can limit the number of concurrent downloads with setMaxNumberOfConcurrentDownloads:completion:
You can limit the number of concurrent operations (rendering a progressive scan is an operation) with setMaxNumberOfConcurrentOperations:completion:. This is probably what you want.
You can reduce the number of scans rendered per progressive image with setProgressThresholds:completion:. The default is [0.00, 0.35, 0.65] which will result in up to 3 scans. You could reduce this by dropping one or two of those.
Let me know how that works!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/pinterest/PINRemoteImage/issues/243#issuecomment-248770393, or mute the thread https://github.com/notifications/unsubscribe-auth/AA6wiFXLj86Sx-pkf2O9OTmEM-HcjYo1ks5qsboagaJpZM4J4U8w.

So, after working a lot more with this, I'm pretty sure I've identified what was my base issue.  In one case I worked around it via AsyncDisplayKit, but in another I couldn't figure out how to use it (can't figure out how to do ASScrollNode with zooming).
So, the place where I was using it most recently was in an UIPageViewController.  I was doing imageView.pin_setImage in viewDidLoad of the child view controllers for the page view controller, and in viewDidDisappear, I was calling pin_cancelImageDownload.  Now, for images I didn't have cached, this worked great.
But for images I DID have cached, a quirk of the implementation of UIPageViewController when you're using UIPageViewControllerDataSource (so viewControllerAfter/viewControllerBefore) is that if you swipe VERY quickly between pages, it doesn't properly deinit the ones that are moving off screen.  I think a similar thing is happening with a UITableViewController and its cells.
What this meant in practice is that, despite the fact that under normal circumstances I would only have 3 controllers loaded which wasn't too much of a memory issue, if a user swiped quickly, it would call viewDidLoad on multiple (8-10, lets say) controllers and wouldn't deinitialize them, and the cancel download didn't have any effect because it was loading them directly in from the cache.
So it would load 8-10 (in my case, very large) decode 8-10 cached images into memory simultaneously and subsequently get memory killed.
Similarly with UITableViews and scrolling very fast through a table where you have most images cached and they are large images.
Setting a max concurrent operations and max concurrent downloads didn't really seem to have an impact on this.
Is there some way to a) serialize pulling images from the cache (as a configuration option, obviously) and b) have those serialized requests respect calls to cancelImageDownload (or some other function call specifically designed for this)?
I think that would allow this sort of use of PINRemoteImage by just setting a few configuration options, instead of the crazy workaround I had to do (basically I have now created my own queue for calling pin_setImage on the image views in my page view controller, and I check to ensure that the view in question is still within rendering distance (on screen, or just off screen) by the time it gets to calling setImage for it)

Thanks for the awesome description Luke, Garrett and I really appreciate the opportunity to learn from your use case.
It's certainly concerning to think that UIPageViewController may have a bug related to this.  As you may have found with AsyncDisplayKit, there is a formal -clearContents and -clearFetchedData callback, which can be set to trigger a predefined distance away from the screen.
If you haven't tried ASPagerNode for this (with ASNetworkImageNodes, powered by PINRemoteImage) — it should be a tremendous help for managing the memory footprint and loading strategy for large images like you describe.  You may want to tweak the default values for the range tuning parameters.
Have you considered using a CATiledLayer if you are displaying content that is higher resolution than the screen?  Especially if you want to implement zooming, this would be the correct approach.  You could even create an ASDisplayNode with +layerClass of CATiledLayer, and use its -didEnterPrefetchState, -didEnterDisplayState, and the clear methods to manage loading into the tiled layer.

On Sep 29, 2016, at 2:14 PM, Luke Ivers<EMAIL_ADDRESS>wrote:
So, after working a lot more with this, I'm pretty sure I've identified what was my base issue. In one case I worked around it via AsyncDisplayKit, but in another I couldn't figure out how to use it (can't figure out how to do ASScrollNode with zooming).
So, the place where I was using it most recently was in an UIPageViewController. I was doing imageView.pin_setImage in viewDidLoad of the child view controllers for the page view controller, and in viewDidDisappear, I was calling pin_cancelImageDownload. Now, for images I didn't have cached, this worked great.
But for images I DID have cached, a quirk of the implementation of UIPageViewController when you're using UIPageViewControllerDataSource (so viewControllerAfter/viewControllerBefore) is that if you swipe VERY quickly between pages, it doesn't properly deinit the ones that are moving off screen. I think a similar thing is happening with a UITableViewController and its cells.
What this meant in practice is that, despite the fact that under normal circumstances I would only have 3 controllers loaded which wasn't too much of a memory issue, if a user swiped quickly, it would call viewDidLoad on multiple (8-10, lets say) controllers and wouldn't deinitialize them, and the cancel download didn't have any effect because it was loading them directly in from the cache.
So it would load 8-10 (in my case, very large) decode 8-10 cached images into memory simultaneously and subsequently get memory killed.
Similarly with UITableViews and scrolling very fast through a table where you have most images cached and they are large images.
Setting a max concurrent operations and max concurrent downloads didn't really seem to have an impact on this.
Is there some way to a) serialize pulling images from the cache (as a configuration option, obviously) and b) have those serialized requests respect calls to cancelImageDownload (or some other function call specifically designed for this)?
I think that would allow this sort of use of PINRemoteImage by just setting a few configuration options, instead of the crazy workaround I had to do (basically I have now created my own queue for calling pin_setImage on the image views in my page view controller, and I check to ensure that the view in question is still within rendering distance (on screen, or just off screen) by the time it gets to calling setImage for it)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/pinterest/PINRemoteImage/issues/243#issuecomment-250593255, or mute the thread https://github.com/notifications/unsubscribe-auth/AAigA-7TH68N-MxRAt7QmL9OTFzgNMK1ks5qvCoYgaJpZM4J4U8w.

Going to close this one out due to a lack of activity.


Some fixes to validate_ima_policy_data

This PR fixes a recently introduced bug (by me) where the return parameters of the function were adjusted to only return an error message instead of the redundant is_valid parameter.

Hmmm... I've ended up with a somewhat confusing commit message while squashing. Should I revert @mpeters ?

It seems a bit redundant and long, but not inaccurate, right? I'm inclined to just leave it.
Btw, we shouldn't ever do a force push on the main branch as that will mess up everyone who has a fork or checkout.


Handle empty dates

Currently when adding a date field to a template and calling a document that doesnt yet have that date filled out the SDK raises an error because it cant convert an empty string or nil to a Date/Time.
It will still raise an error for malformed dates.
This fixes it by returning an empty text fragment (so calling .value) still works.
I thought about a few other approaches but this seems to have the lowest impact. Open to feedback

Btw looks like these same tests that failed are failing in master

What are your thoughts on this @faresd ?

@zjwhitehead thanks for your contribution and apologies for the massive delay in responding. While it works it feels slightly icky that we would return a Prismic::Fragments::Text instance when we are expecting a Prismic::Fragments::Date instance. I think it would be better if we adjust Prismic::Fragments::Date#to_html to simply do an early return with an empty string ('') if the value is nil.


Change speed of switching slides in preview mood

How can I change speed of switching slides in preview mood? In documentation there is no such option
Overriding for transitions style not fully solve the issue
@lukasz-galka FYI

Hi @olshansky, unfortunately no. You can only disable animation using option previewAnimation. I will try to add this as a option.

@lukasz-galka Any update on this?

Would be a nice feature!


joshdholtz fix init and spaceship tunes find

Motivation and Context
TBD
Description
TBD
Testing Steps
TBD

          1 Warning

:warning:
Big PR

  Generated by :no_entry_sign: Danger

This test didn't work


LoginFailed

POST /bus/createEventBus HTTP/1.1
Host: demo.eventbridge.com
Content-Type: application/json; charset=utf-8
{
"eventBusName":"demo-bus",
"description":"a demo bus."
}
error message:
{
"code": "LoginFailed"
}

use the latest main branch run again


Typo in creating-a-cache.mdx

Typo import -> important

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Questions about the set of batch_size and epoch_size

Hi david,
I notice that you set batch_size=90 and epoch_size=1000 in the training and iterates about 80 times totally, but the whole number of images used for training is about 490k, so I dont know whether every image is used 80×90×1000/490k=14 times ?
What's more, I train on my own dataset which contains 9000 images with these parameters:
max_nrof_epochs 10
epoch_size 1000
batch_size 9
But a problem occurs:
OutOfRangeError (see above for traceback): FIFOQueue '_1_batch_join/fifo_queue' is closed and has insufficient elements (requested 9, current size 0)
[[Node: batch_join = QueueDequeueUpToV2[component_types=[DT_FLOAT, DT_INT64], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](batch_join/fifo_queue, _recv_batch_size_0)]]
I find the same error in the closed issue but their solution doesn't fit  me.
#472
#338
#273
could you please help about this?

The problem above is solved, and the reason is that I put a .csv file into the dataset directory, so it cannot read this file....
But a new error occurs, every time I run the train_softmax.py ,it stuck at the half of the iteration like this:

Here I set epoch_size=100, and it shows that it stucks at 50, I have no idea what is going on...

omg, I realize I put a .csv file into the dataset directory...so it cannot read as a normal np.array format...


removes regenerator-runtime from cdn.ts

resolves #2868

thanks! don't forget the changeset so that we can publish a bug fix
I'm surprised it's still there! I'm pretty sure the browserslist we have now (is our browserslist up to date?)  means this is no longer required.


fix(build): migrate to the main lodash package

What's the problem this PR addresses?
The per method packages for lodash has been deprecated and the use of them is discouraged.
See https://lodash.com/per-method-packages for more
How did you fix it?
Replaced the per method packages with lodash and required the method directly

Thanks! I'm going change this to target the v6 branch instead, as that's likely to be our next release.

Actually, that ends up being pretty messy when targeted against v6. If you don't mind, I can just close this PR and create a similar one myself using v6 as the base—or alternatively, if you'd prefer, you could do the same.
Just let me know.

I just rebased the PR instead :)

Thank you!