SyntaxError: Unexpected identifier

C:\Users\Administrator\Desktop\node-steamcardbot-master\node-steamcardbot-master

node bot.js
C:\Users\Administrator\Desktop\node-steamcardbot-master\node-steamcardbot-master
\config.js:19
welcome: 'Welcome to Rick's Level up Service, Our current rate i
s 16:1 (Set > Key) Use !help for more informaton to start.',
^

SyntaxError: Unexpected identifier
at createScript (vm.js:56:10)
at Object.runInThisContext (vm.js:97:10)
at Module._compile (module.js:542:28)
at Object.Module._extensions..js (module.js:579:10)
at Module.load (module.js:487:32)
at tryModuleLoad (module.js:446:12)
at Function.Module._load (module.js:438:3)
at Module.require (module.js:497:17)
at require (internal/module.js:20:19)
at Object. (C:\Users\Administrator\Desktop\node-steamcardbot-mast
er\node-steamcardbot-master\bot.js:25:14)
C:\Users\Administrator\Desktop\node-steamcardbot-master\node-steamcardbot-master

pause
Press any key to continue . . .

'Welcome to Rick's Level up Service, Our current rate i
s 16:1 (Set > Key) Use !help for more informaton to start.',
You basically closed the string at Rick'
Anything else after that isn't apart of the string and naturally, you'll get an error.
You could try enclosing the string with double quotes in order to facilitate the apostrophe. Like this:
"Welcome to Rick's Level up Service, Our current rate i
s 16:1 (Set > Key) Use !help for more information to start."

Derp, Got it. Had that multiple times throughout the list. Fixed but now I have a separate spam error. Me and GGrandma were trying to figure out what goes where.
C:\Users\Administrator\Desktop\node-steamcardbot-master\node-steamcardbot-master

node bot.js
Sun Nov 05 2017 12:51:27 GMT-0800 (Pacific Standard Time) - Connected...
C:\Users\Administrator\Desktop\node-steamcardbot-master\node-steamcardbot-master
\node_modules\long\dist\long.js:310
return fromBits(val.low, val.high, val.unsigned);
^

TypeError: Cannot read property 'low' of undefined
at Function.fromValue (C:\Users\Administrator\Desktop\node-steamcardbot-mast
er\node-steamcardbot-master\node_modules\long\dist\long.js:310:28)
at Function.SteamID.decode (C:\Users\Administrator\Desktop\node-steamcardbot
-master\node-steamcardbot-master\node_modules\steam\lib\steamID.js:19:18)
at SteamFriends.sendMessage (C:\Users\Administrator\Desktop\node-steamcardbo
t-master\node-steamcardbot-master\node_modules\steam\lib\handlers\friends\index.
js:59:20)
at log (C:\Users\Administrator\Desktop\node-steamcardbot-master\node-steamca
rdbot-master\bot.js:38:15)
at SteamClient. (C:\Users\Administrator\Desktop\node-steamcardbot
-master\node-steamcardbot-master\bot.js:50:3)
at emitNone (events.js:86:13)
at SteamClient.emit (events.js:185:7)
at SteamClient.handlers.(anonymous function) (C:\Users\Administrator\Desktop
\node-steamcardbot-master\node-steamcardbot-master\node_modules\steam\lib\steam_
client.js:243:8)
at SteamClient._netMsgReceived (C:\Users\Administrator\Desktop\node-steamcar
dbot-master\node-steamcardbot-master\node_modules\steam\lib\steam_client.js:157:
26)
at emitOne (events.js:96:13)
C:\Users\Administrator\Desktop\node-steamcardbot-master\node-steamcardbot-master

pause
Press any key to continue . . .

Let me see if I can help with this in the other open issue


Can not find any working java library for send/receive dogecoin

Hi,
I am looking for java library for send/receive dogecoin. Can anyone suggest me working java library, please ?

I haven't used it for a while but this library might work:
https://github.com/dogecoin/libdohj
Make sure to use testnet coins when developing! :)

Ok. I will try. Thanks :)

Hi,
I added libdohj-core-0.14-SNAPSHOT.jar &  below   dependencies
<dependencies>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>1.7.7</version>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
            <type>jar</type>
        </dependency>
        <dependency>
            <groupId>com.google.protobuf</groupId>
            <artifactId>protobuf-java</artifactId>
            <version>2.5.0</version>
        </dependency>
        <dependency>
            <groupId>com.lambdaworks</groupId>
            <artifactId>scrypt</artifactId>
            <version>1.4.0</version>
        </dependency>
        <dependency>
            <groupId>org.bitcoinj</groupId>
            <artifactId>**bitcoinj-core**</artifactId>
            <version>**0.14.2**</version>
        </dependency>
    </dependencies>

Now receive coins working but send not working i.e tx hash show tx not
found on blockchain.
Please check below log.
2018-08-01 13:23:58,172 ERROR [org.bitcoinj.core.Context] (EJB default - 3)
Performing thread fixup: you are accessing bitcoinj via a thread that has
not had any context set on it.
2018-08-01 13:23:58,172 ERROR [org.bitcoinj.core.Context] (EJB default - 3)
This error has been corrected for, but doing this makes your app less
robust.
2018-08-01 13:23:58,172 ERROR [org.bitcoinj.core.Context] (EJB default - 3)
You should use Context.propagate() or a ContextPropagatingThreadFactory.
2018-08-01 13:23:58,172 ERROR [org.bitcoinj.core.Context] (EJB default - 3)
Please refer to the user guide for more information about this.
2018-08-01 13:23:58,172 ERROR [org.bitcoinj.core.Context] (EJB default - 3)
Thread name is EJB default - 3.
2018-08-01 13:23:58,173 INFO  [com.eno.doge.DOGEConfig] (EJB default - 3)
DOGEConfig_sendCoin:Balance=1005.00 BTC
2018-08-01 13:23:58,386 INFO  [org.bitcoinj.crypto.KeyCrypterScrypt] (EJB
default - 3) Deriving key took 206.7 ms for 16384 scrypt iterations.
2018-08-01 13:23:58,387 INFO  [org.bitcoinj.wallet.Wallet] (EJB default -
3) Completing send tx with 1 outputs totalling 3.00 BTC and a fee of 2.00
BTC/kB
2018-08-01 13:23:58,398 INFO  [org.bitcoinj.wallet.Wallet] (EJB default -
3)   with 996.55 BTC change
2018-08-01 13:23:58,421 INFO  [org.bitcoin.Secp256k1Context] (EJB default -
3) java.lang.UnsatisfiedLinkError: no secp256k1 in java.library.path
2018-08-01 13:23:58,436 INFO  [org.bitcoinj.wallet.Wallet] (EJB default -
3)   completed:
 a2838d507fcf1df990cf0017e00d73a4ebb2cd57b2f69df75bfd5660144b8717
     in
 PUSHDATA(71)[3044022047c0b6134e0a93cdea2a490ace87773da901a1f89e5300b76d59da7ae3c2a6d7022003f568d96638d4d7a8a3f7cc3019e0663c8526e872e5ebd912e904d184804ea301]
PUSHDATA(33)[02072314acce50e737791e490bff7ff44a66a325680d00f60898674a191e355f96]
1000.00 BTC

outpoint:459bf5db26759e08f515b36caf284258776219c676f796f8a760aa3fc86fd140:0
hash160:d0108b2243988bb583e731ed0d03e25c85038f48
     out  DUP HASH160
PUSHDATA(20)[143754b58fcbf5504ad7f4833c2b62d5b2867c80] EQUALVERIFY CHECKSIG
996.55 BTC
     out  HASH160 PUSHDATA(20)[f1fcaab99bddd4171a1756850d95510c4bc05355]
EQUAL 3.00 BTC
     fee  2.01793721 BTC/kB, 0.45 BTC for 223 bytes
     prps USER_PAYMENT

2018-08-01 13:23:58,437 INFO  [org.bitcoinj.wallet.Wallet] (EJB default -
3) commitTx of
a2838d507fcf1df990cf0017e00d73a4ebb2cd57b2f69df75bfd5660144b8717
2018-08-01 13:23:58,438 INFO  [org.bitcoinj.wallet.Wallet] (EJB default -
3)   marked
459bf5db26759e08f515b36caf284258776219c676f796f8a760aa3fc86fd140:0 as spent
by a2838d507fcf1df990cf0017e00d73a4ebb2cd57b2f69df75bfd5660144b8717
2018-08-01 13:23:58,438 INFO  [org.bitcoinj.wallet.Wallet] (EJB default -
3)   459bf5db26759e08f515b36caf284258776219c676f796f8a760aa3fc86fd140
prevtx <-unspent ->spent
2018-08-01 13:23:58,439 INFO  [org.bitcoinj.wallet.Wallet] (EJB default -
3) ->pending:
a2838d507fcf1df990cf0017e00d73a4ebb2cd57b2f69df75bfd5660144b8717

Can you help me to find issue, please ?
On Tue, Jul 31, 2018 at 7:38 PM, Cameron Clough<EMAIL_ADDRESS>wrote:

I haven't used it for a while but this library might work:
https://github.com/dogecoin/libdohj
Make sure to use testnet coins when developing! :)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/dogecoin/dogecoin/issues/1509#issuecomment-409233790,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKHjXo8VgBYEtc7f8LKN0-phWyt072Viks5uMGTXgaJpZM4VoR7h
.

--
Regards,
Krunal S. Ransing.

I'd wager a guess that you're not connected to any peers. Did peer discovery succeed?

Hi,
Coin receive working well so i guess peers are connected. Also below log
shows that 8 connected.
2018-08-02 10:50:46,946 INFO  [org.bitcoinj.core.PeerGroup]
(NioClientManager) [<IP_ADDRESS>]:44556: Peer died      (8 connected, 1
pending, 12 max)
What do you think?
On Thu, Aug 2, 2018 at 4:20 PM, Max K<EMAIL_ADDRESS>wrote:

I'd wager a guess that you're not connected to any peers. Did peer
discovery succeed?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/dogecoin/dogecoin/issues/1509#issuecomment-409886579,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKHjXn9jpbCrUmAYD0IVN-A8z24Owb7Jks5uMtl-gaJpZM4VoR7h
.

--
Regards,
Krunal S. Ransing.

Hi,
Ok. I will try to run testnet node locally & check.
On Thu, Aug 2, 2018 at 4:38 PM, Max K<EMAIL_ADDRESS>wrote:

Hm right. Normally the log should give a bit more info about the state of
the transaction and repeat the sending after a while if it failed. The
question would be if the peers actually accept the tx. Do you happen to
have a local Dogecoin Core node running you can test against? Then you
could connect to it and check its log.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/dogecoin/dogecoin/issues/1509#issuecomment-409890767,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKHjXhbfoN01q4mK4mIqhKy94pA6AU54ks5uMt3GgaJpZM4VoR7h
.

--
Regards,
Krunal S. Ransing.

Hi,
2018-08-02 11:40:52,622 INFO  [org.bitcoinj.core.TransactionBroadcast]
(EJB default - 36) Waiting for 10 peers required for broadcast, we have 8
For testnet only one dns seed mentioned i.e "testseed.jrn.me.uk".
Do we have more ?
This can be issue ?
On Thu, Aug 2, 2018 at 5:07 PM, Kunal Ransing<EMAIL_ADDRESS>wrote:

Hi,
Ok. I will try to run testnet node locally & check.
On Thu, Aug 2, 2018 at 4:38 PM, Max K<EMAIL_ADDRESS>wrote:

Hm right. Normally the log should give a bit more info about the state of
the transaction and repeat the sending after a while if it failed. The
question would be if the peers actually accept the tx. Do you happen to
have a local Dogecoin Core node running you can test against? Then you
could connect to it and check its log.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/dogecoin/dogecoin/issues/1509#issuecomment-409890767,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKHjXhbfoN01q4mK4mIqhKy94pA6AU54ks5uMt3GgaJpZM4VoR7h
.

--
Regards,
Krunal S. Ransing.

--
Regards,
Krunal S. Ransing.

Testnet only has very limited peers available. You could try lowering that limit, I just don't know where exactly that is right now.

I guess issue solved. In code for testnet dns seeds contains only one entry. For broadcast 10 peers connection required & I had only 8 connected. Now I added few more dns seeds then peers connection crossed 10 & it broadcasts txs.
Thanks.

Hi @langerhans
When I receive/send its shows BTC , is it because used bitcoin core, so its show BTC string ?

Hi,
Now it's working fine. Thank you very much.
On Thu, Aug 2, 2018 at 5:44 PM, Max K<EMAIL_ADDRESS>wrote:

Testnet only has very limited peers available. You could try lowering that
limit, I just don't know where exactly that is right now.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/dogecoin/dogecoin/issues/1509#issuecomment-409905347,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKHjXjbKU16RUoPHHTJjY6vtunZ5-TxBks5uMu0OgaJpZM4VoR7h
.

--
Regards,
Krunal S. Ransing.


Is playlist functional?

Thank you for this great little player!
I don't understand if the playlist feature is ready in version 3, and, if so, how I can use it.
Can you provide an example, please?
G.

Not yet. There's a (very old) issue that's tracking this but it's not something I've started on yet.

@gborgonovo check this out ;) https://github.com/sampotts/plyr/issues/1276


Add 'Package info' group button #91

Also added CSS in main.css to slightly reduce buttons' left/right
padding.

Signed-off-by: John M. Horan<EMAIL_ADDRESS>

:+1:  Will tackle this afternoon, @jdaguil .

LGTM!


Add entry for theemcoe.org

Official Website
Course for Bachelors' in Computer Engineering

@Omkaragrawal First stage of verification passed, official domain for Theem College of Engineering is theemcoe.org as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision.

@Omkaragrawal Thank you for your patience while waiting for a review and processing your request. Pull request merged.
Please start requesting the licenses in about an hour, to let the changes to propagate through our system.


Targeting - properties with special characters

Review special characters support for:

[x] Content tests
[ ] Overwrites
[x] Assign variables

Linked to #365


Update for Flarum beta 8

Changes made:

Modified all the JS directories accordingly
Updated `bootstrap.php to utilize new Extenders
Removed unnecessary Listeners
Update icons for FontAwesome 5
Did something to make the default translations API endpoint work again (don't ask me why it works)
Removed TranslationLock as a result of ^
Fixed unable to set All translation because of type string in validation (removed from payload through JS if non-existent)
Removed scrollbar if not needed in Dropdown buttons (was caused by other style applying margin)

@datitisev first thanks for the work on this, it saves me a lot of time :hugs:
I'm really confused by something you changed in there. I don't understand the need to remove the ConfigureLocales event...
Your solution does work (kind of, see below), but I don't find it pretty at all. You in fact delayed the loading of the Linguist string loader until serialization and removed the use of the event and the locking system. Which understandably achieves the same behavior as before, allowing the default string repository to access the default string before Linguist kicks in while still loading the strings "every time".
But there are multiple issues:
If the request does not involve serialization (like the password reset page) or if the translated strings are retrieved before serialization (sending emails), the custom loader isn't registered yet and custom translations aren't used.
If multiple models are serialized, the custom loader is included multiple times and therefore runs multiple times, causing more database access than required.
I'm going to try to revert that part to how it was before. Maybe you have more insights to share on the subject ? Thanks :wink:

@datitisev I merged your changes to master along with a new strategy for loading strings that I think solves all issues raised above. Want to have a look before I tag a release ?
A simple revert wasn't possible because it was all built around replaying the events for configuring locales. But now that Flarum uses extenders which interact with the container directly, there was no way to keep it working that way.
However instead of trying to not add my resources, I can simply make my custom loader not run when I want the default translations list. Which is a lot simpler. This solution could even already have been used in beta 7.
I'm just not sure whether translation caching could mess with this solution (or the previous one as well). Flarum has a storage/locale folder but whether I'm in debug mode or not the folders seems to stay empty.


Add support for importing an existing model

This implements import support for juju_model resource.

 Add support for importing an existing model


fs/epoll: fix compile break about change type of pollevent_t

Summary
fs/epoll: fix compile break about change type of pollevent_t
Signed-off-by: Jiuzhu Dong<EMAIL_ADDRESS>Impact
N/A
Testing
CI

This patch is associated with https://github.com/apache/incubator-nuttx/pull/5937
@xiaoxiang781216 please commit this PR first, thanks.


gitignore is not coding practice for beginners.

Hello, please can you be more explicit

Hello, please can you be more explicit

Under coding practice, you have gitignore listed, but it is more of a general resource rather than a place to learn and practice code.

Under coding practice, you have gitignore listed, but it is more of a general resource rather than a place to learn and practice code.

Yeah you are right, thanks for reporting, if you can open a PR to move it into another it will be great!

Yeah you are right, thanks for reporting, if you can open a PR to move it into another it will be great!


fix(cursor): Always return pandas.DataFrame

Description

Currently cursor.fetch_dataframe() returns a None object when there are no results returned by the query, this is also backed up by the type hint typing.Optional["pandas.DataFrame"] on this line. My changes makes sure that cursor.fetch_dataframe() always returns a pandas.DataFrame even if if the DataFrame is empty (but with column headers)
Motivation and Context

Without reading too deeply into the source code, when a user calls cursor.fetch_dataframe() they expect a pandas.DataFrame to be returned, but there is an case when the SQL query returns no results then fetch_dataframe returns None instead of the expected empty DataFrame. This causes issues for users who migrate away from things like psycopg2 and manual creation of DataFrames, they would expect to test empty datasets not by df is None but using the .empty attribute or even:
df = cursor.fetch_dataframe()
if df:
    print("DataFrame has data")
else:
    print("DataFrame is empty")

Testing

Screenshots (if appropriate)
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)

Checklist

[ ] Local run of ./build.sh succeeds
[ ] Code changes have been run against the repository's pre-commit hooks
[x] Commit messages follow Conventional Commit Specification
[x] I have read the README document
[ ] I have added tests to cover my changes
[ ] I have run all unit tests using pytest test/unit and they are passing.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Happy New Year @Brooke-white, have you been able to take a look into this?

Thanks @Brooke-white! Glad it was a simple fix, look forward to the next release


Integration in SDD EventManager

Just wanted to let you know that I was able to integrate this EventVisualizer with the SDD EventManager.
Awesome work, just a few code changes were needed and most features are still usable.

Awesome! :-) Thanks for letting me know.
https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail
Virus-free.
www.avast.com
https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
On 17 February 2018 at 20:55, Nils Brinkmann<EMAIL_ADDRESS>wrote:

Just wanted to let you know that I was able to integrate this
EventVisualizer with the SDD EventManager
https://github.com/UbimaxFrontline/EventManager.
Awesome work, just a few code changes were needed and most features are
still usable.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/MephestoKhaan/UnityEventVisualizer/issues/7, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AEvwqr3werLunnhQCK3wPN65_4kmsV_Gks5tVzy-gaJpZM4SJdVx
.


Generate builtin getters for Date object

The Date Prototype Object  (15.9.5) has some getters (around 26), that are very similar. For now, I implemented most of them separately in [1]. Maybe we could generate them. The most different parts are in a try-catch block.
Could you give some ideas how I should generate these getter functions.
Some typically differences:

the helper functions return value is: int or ecma_number_t
Every getter function has a simple and an UTC version.

CC: @egavrin @ruben-ayrapetyan  @galpeter @zherczeg @LaszloLango
[1] https://github.com/szledan/jerryscript/commit/51275ff06474372660db783d5789834a351041f4

Closing issue, as this was implemented in https://github.com/Samsung/jerryscript/pull/334 and landed in: https://github.com/Samsung/jerryscript/commit/8d7cdebee09d9fcccb85b47efa4e84341f065607


[SPARK-15508][Streaming][Tests]Fix flaky test: JavaKafkaStreamSuite.testKafkaStream

What changes were proposed in this pull request?
JavaKafkaStreamSuite.testKafkaStream assumes when sent.size == result.size, the contents of sent and result should be same. However, that's not true. The content of result may not be the final content.
This PR modified the test to always retry the assertions even if the contents of sent and result are not same.
Here is the failure in Jenkins: http://spark-tests.appspot.com/tests/org.apache.spark.streaming.kafka.JavaKafkaStreamSuite/testKafkaStream
How was this patch tested?
Jenkins unit tests.

Test build #59216 has started for PR 13281 at commit b9f5fae.

Test build #59216 has finished for PR 13281 at commit b9f5fae.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59216/
Test PASSed.

cc @jerryshao since you wrote this test

Thanks for the fix, LGTM :)

Thanks for reviewing. Merging to master / 2.0.


No mailing list name found at http://mm.icann.org/pipermail/accountability-coordination-group

No mailing list name found at http://mm.icann.org/pipermail/accountability-coordination-group
There's a blank archive for this list:
http://mm.icann.org/pipermail/accountability-coordination-group/
Eventually this hits a MissingDataException
bigbang.mailman.MissingDataException
bigbang.mailman.MissingDataException: 'No messages in archives under http://mm.icann.org/pipermail/accountability-coordination-group. Did you run the collect_mail.py script?'
But this should be caught and dealt with earlier so it doesn't break the script.

https://github.com/nllz/bigbang/commit/74171a8553ec0957d7d7b0966092b5ee24b56e8e


refactor: Use path alias instead of relative imports

What it solves
Uses path alias @/... instead of relative imports
Note: There are still some (short) relative imports which my IDE didn't catch but they are not causing warnings.
How to test

All tests should still run

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

234 of 248 (94.35%)  changed or added relevant lines in 150 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage remained the same at 92.495%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/routes/about/tests/get-about.e2e-spec.ts
0
1
0.0%

src/routes/health/tests/get-health.e2e-spec.ts
0
2
0.0%

src/routes/owners/tests/get-safes-by-owner.e2e-spec.ts
0
2
0.0%

src/routes/contracts/tests/get-contract.e2e-spec.ts
0
3
0.0%

src/routes/data-decode/tests/data-decode.e2e-spec.ts
0
3
0.0%

src/routes/safe-apps/tests/get-safe-apps.e2e-spec.ts
0
3
0.0%

Files with Coverage Reduction
New Missed Lines
%

src/routes/data-decode/tests/data-decode.e2e-spec.ts
1
0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
4353

Relevant Lines:
4606

💛  - Coveralls


Remote code execution vulnerability in NLTK

The current and earlier versions of NLTK are vulnerable to a remote code execution vulnerability when using the integrated data package download functionality. A man-in-the-middle attacker or an attacker with control over the NLTK data index can force users which use data packages with pickled Python code to download a new version of the package which executes arbitrary code upon unpickling.
Data packages which have been identified to be vulnerable are "averaged_perceptron_tagger" and "punkt". For code to be vulnerable it has to download a data package and use functionality in NLTK which causes the data package to be unpickled.
Here is an example of vulnerable code for the "averaged_perceptron_tagger" data package:
import nltk
nltk.download("averaged_perceptron_tagger")
nltk.pos_tag(["hello", "world"])

This vulnerability was reported together with POC code to exploit it multiple times to the NLTK team via the email address mentioned in SECURITY.md (and later directly to some maintainers as well). It was reported first on 2024-05-19. So far there has been no response from the NLTK maintainers to these reports.
https://github.com/nltk/nltk/issues/2522 already pointed out the security implications of using pickled code a few years ago, but didn't receive a response either.

@Dunedan, what would you suggest?
Maybe print a warning and ask for confirmation before loading a pickle?
The pickles in question contain Python classes with executable functions. They are not data but programs. And,as a compiled format, they are not ideal for an open-source project.
But an eventual attacker needs to trick the user into loading a malicious pickle.

The short answer to solve this issue is to remove the download of code via network at runtime.
If that's not feasible for NLTK I have no clear answer how to solve this issue. However, there are some measures I'd implement, which would at least improve the situation:
Remove use of pickled code wherever possible
While I haven't checked what the code in in the data packages with pickled code does, I can imagine that it's not the code which is large, but rather data associated with it. Maybe the algorithms implemented in there can be moved into NLTK, removing the need for pickled code.
Also instead of using pickle, a domain-specific format could be used (depending on the use case something like ONNX (https://onnx.ai/) might fit for that).
Make pickled code reproducible
When an update for a data package with pickled code is issued, it's difficult to figure out what changed. Having the pickle files be reproducibly built would be helpful for assessing if they're legit.
Make documentation clear for download once during setup
While the documentation details various ways to install data packages right now, making it more prominent that data packages should be installed once during installation/setup, instead of including Downloader.download() in application code, could be helpful to make users aware of the implications downloading data packages in their application code has.
Opt-in for auto-updating installed data packages
Right now calling Downloader().download() always results in the data index being fetched and potential updates to already installed packages being downloaded and installed. Instead I believe the download functionality should by default only do network requests and install data packages if they're not already installed. Automatic updating could then be an additional parameter users have to explicitly set.
Opt-in for installing data packages containing code
Similar to having users explicitly opt-in for auto-updates, I'd do the same for downloads of data packages which contain code which is going to be executed.
In addition I'd always log a warning when a package with pickled code gets downloaded or used.
Verify downloaded packages
Right now the data index contains a checksum per package. However this checksum is only used to check whether a downloaded package differs from a previously downloaded version. It is not used right now to check if a downloaded package is the one referenced by the data index. This should be done together with the move to a cryptographical strong hash function for the checksums.

Thanks @dunedan for your detailed suggestions.
Should this issue be labelled critical? I wonder if it could make NLTK deemed usafe for use in schools, or inclusion in some software distributions. But on the other hand, has any exploit been reported since #2522, back in 2020? When nobody else than the administrator controls nltk_data, the pickles should be safe to load.
It could be nice to remove all the pickles completely, for ex. by moving the code into the classes of the corresponding corpus reader in NLTK, and reducing the data package, for ex. to CSV files for tabular data, and JSON or XML for complex data.

This is now being tracked as CVE-2024-39705.

In addition to the two packages already mentioned, the following also contain pickles:

chunkers/maxent_ne_chunker.zip
help/tagsets.zip
taggers/maxent_treebank_pos_tagger.zip

In total nltk_data contains 52 pickles, where half are Python 2, which is not supported anymore, so these may just be deleted.
Starting at the easy end, the pickles in help/tagsets.zip are very easy to decompile, since they are just Python dictionaries that associate each tag symbol with a tuple consisting in the tag name, and a string of instances. This is just tabular data, that would fit very well in the CSV or TAB format. One may wonder what made it seem attractive to compile this data into pickles back then.

Hey team!!!
We hit a critical dependabot alert regarding this vulnerability. So I'm now tracking this thread quite closely!
I was trying to trace down into its sub dependencies but got super lost with where to be looking in order to verify that these vulnerable data packages are being loaded into our codebase;
Sorry for the silly questions here; but would love to understand this a little more;

A man-in-the-middle attacker or an attacker with control over the NLTK data index can force users which use data packages with pickled Python code to download a new version of the package which executes arbitrary code upon unpickling.

This is the first time I've heard of pickling! So thank you for raising this and introducing me to this new concept for me.
So, my understanding of the vulnerability here is that the pickled byte-stream can be manipulated before unpickling the content which may include malicious executable code?

We're currently using the LazyCorpusLoader and we download the words package. My understanding here is that the download() function is a main vulnerable entry-point here?
Part of the issue here is we use this within an AWS lambda function which means we would run a download every single run.. making this really open :/

How can I tell whether the words package is a pickled download? Or for a broader question; what's the best way to trace which datasets are prone to this vulnerability?

#!/bin/sh
for f in ~/nltk_data/*/*zip
do
unzip -l $f | grep -i pickle >> Nltk_Pickles.txt
done

Nltk_Pickles.txt

On Mon Jul 1, 2024 at 1:35 PM CEST, DhenPadilla wrote:

This is the first time I've heard of pickling! So thank you for raising this and introducing me to this new concept for me.
So, my understanding of the vulnerability here is that the pickled byte-stream can be manipulated before unpickling the content which may include malicious executable code?

See https://docs.python.org/3/library/pickle.html and big red warning there. Pickle can contain ANY Python data including but not limited to executable functions. So, pickle should NEVER be used for data which are outside of your control. Especially, downloading pickles from Internet carries with itself serious risk of downloading and executing malicious code.

I understand this is just an emergency maintenance brutal workaround, but would anything outside of the downloading stuff from Internet break with this patch?
---
 nltk/app/chartparser_app.py    |   13 +++++++++++++
 nltk/corpus/reader/util.py     |    2 ++
 nltk/data.py                   |    2 ++
 nltk/parse/transitionparser.py |    2 ++
 nltk/tbl/demo.py               |    4 +++-
 5 files changed, 22 insertions(+), 1 deletion(-)

--- a/nltk/app/chartparser_app.py
+++ b/nltk/app/chartparser_app.py
@@ -800,6 +800,10 @@ class ChartComparer:
             showerror("Error Saving Chart", f"Unable to open file: {filename!r}\n{e}")

     def load_chart_dialog(self, *args):
+        showerror("Security Error",
+                  "Due to gh#nltk/nltk#3266, deserializing from " +
+                  "a pickle is forbidden.")
+        return
         filename = askopenfilename(
             filetypes=self.CHART_FILE_TYPES, defaultextension=".pickle"
         )
@@ -811,6 +815,8 @@ class ChartComparer:
             showerror("Error Loading Chart", f"Unable to open file: {filename!r}\n{e}")

     def load_chart(self, filename):
+        raise RuntimeError("Due to gh#nltk/nltk#3266, deserializing from " +
+                           "a pickle is forbidden.")
         with open(filename, "rb") as infile:
             chart = pickle.load(infile)
         name = os.path.basename(filename)
@@ -2268,6 +2274,10 @@ class ChartParserApp:
         if not filename:
             return
         try:
+            showerror("Security Error",
+                      "Due to gh#nltk/nltk#3266, deserializing from " +
+                      "a pickle is forbidden.")
+            return
             with open(filename, "rb") as infile:
                 chart = pickle.load(infile)
             self._chart = chart
@@ -2306,6 +2316,9 @@ class ChartParserApp:
             return
         try:
             if filename.endswith(".pickle"):
+                showerror("Due to gh#nltk/nltk#3266, deserializing from " +
+                          "a pickle is forbidden.")
+                return
                 with open(filename, "rb") as infile:
                     grammar = pickle.load(infile)
             else:
--- a/nltk/corpus/reader/util.py
+++ b/nltk/corpus/reader/util.py
@@ -521,6 +521,8 @@ class PickleCorpusView(StreamBackedCorpu

     def read_block(self, stream):
         result = []
+        raise RuntimeError("Due to gh#nltk/nltk#3266, deserializing from " +
+                           "a pickle is forbidden.")
         for i in range(self.BLOCK_SIZE):
             try:
                 result.append(pickle.load(stream))
--- a/nltk/data.py
+++ b/nltk/data.py
@@ -752,6 +752,8 @@ def load(
     if format == "raw":
         resource_val = opened_resource.read()
     elif format == "pickle":
+        raise RuntimeError("Due to gh#nltk/nltk#3266, deserializing from " +
+                           "a pickle is forbidden.")
         resource_val = pickle.load(opened_resource)
     elif format == "json":
         import json
--- a/nltk/parse/transitionparser.py
+++ b/nltk/parse/transitionparser.py
@@ -553,6 +553,8 @@ class TransitionParser(ParserI):
         """
         result = []
         # First load the model
+        raise RuntimeError("Due to gh#nltk/nltk#3266, deserializing from " +
+                           "a pickle is forbidden.")
         model = pickle.load(open(modelFile, "rb"))
         operation = Transition(self._algorithm)

--- a/nltk/tbl/demo.py
+++ b/nltk/tbl/demo.py
@@ -253,6 +253,8 @@ def postag(
                 )
             )
         with open(cache_baseline_tagger) as print_rules:
+            raise RuntimeError("Due to gh#nltk/nltk#3266, deserializing from " +
+                               "a pickle is forbidden.")
             baseline_tagger = pickle.load(print_rules)
             print(f"Reloaded pickled tagger from {cache_baseline_tagger}")
     else:
@@ -327,7 +329,7 @@ def postag(
         with open(serialize_output) as print_rules:
             brill_tagger_reloaded = pickle.load(print_rules)
         print(f"Reloaded pickled tagger from {serialize_output}")
-        taggedtest_reloaded = brill_tagger.tag_sents(testing_data)
+        taggedtest_reloaded = brill_tagger_reloaded.tag_sents(testing_data)
         if taggedtest == taggedtest_reloaded:
             print("Reloaded tagger tried on test set, results identical")
         else:

Hello, is there an ETA for a fix?

Before getting too alarmed, we may want to wait for a sober analysis of this vulnerability, bearing in mind that it has been known for several years, without any known exploitation yet.
Still, pickles are not open source, which is in itself a good reason to avoid them. In addition to repackaging the data, a complete fix would require deep modifications in several Python files:

#!/bin/sh

tree=nltk
q=pickl
of="$tree"_$q.txt
echo -n > $of
mxd=`ls -R $tree|grep /|awk -F "/" '{print NF}'|sort -nr|head -1`

for f in `find -L $tree -maxdepth $mxd -name "*py"`
do
if [ -n "`grep pickle $f`" ]
then
echo $f >> $of
echo -------------------------------- >> $of
grep -ni $q $f >> $of
echo \\n--------------------------------------------------------------------- >> $of
fi
done

nltk_pickl.txt

+1 we need a fix on this...many repos on https://github.com/amazon-science/ use older NLTK and are affected.

A possible mitigation is to not use NLTK's downloader functionality to download vulnerable data packages until this issue is fixed. Instead you could download the data packages your application needs manually once, verify they are legit and ship them bundled with your application.

As @Dunedan mentioned above. A workaround for now is to consider making your own package index.
Considering the package index that is used is https://github.com/nltk/nltk_data. You can create your own fork and validate the packages you need and customise the mirror used by using --url

Now that it has a public NIST CVE, people will exploit it. This isn't something you "wait" on. This is something you address right away...

@nicolaschaillan,  before even starting NLTK, you load Python, which already exposes the user to the risk of being tricked into downloading malicious pickles anyway. In that context, wouldn't a fix in NLTK only marginally improve the user's security?

@ekaf I agree with @nicolaschaillan and let me explain that I've gotten a high-severity ticket (both internally at Amazon and from GitHub's dependabot) because of this vulnerability.
Given the options of (a) not resolving this ticket and (b) removing all dependencies on nltk from my codebase, I am forced to choose (b), in order to comply with my organization's security requirements. It's either that or take down my repository.

I inspected a typical pickle in each series, looking at their type() and eventual __dict__.
The tagsets and averaged_perceptron_tagger packages contain only simple data structures, and can easily be translated to a text-based data format, which solves the problem radically.
The rest of the pickles contain complex classes. A good translation may only be possible with help from the original package authors, and would require more time.
So at this moment, users who find themselves in an emergency, and need very strict settings, may need to consider some of the mitigations suggested above, like forbidding unpickling, or using their own secure fork.

The solution is simple. Whitelist which pickles can be loaded. Give control to the tenant to decide which ones are fine or not. Maybe by name or something. Right now, the fear is that third party can manage to load some that are malicious.

I don’t think, that’s good enough. As long as there is an opportunity how to make NLTK download pickle from the Internet (by using some internal undocumented functions, there are really no private and really inaccessible functions in Python), somebody can make download malicious content from the Internet.
Pickle should really never be used on any content which is not 100% under the control of the admin of the machine.
That’s the reason I suggested so harsh solution as above.

Please send an email with the CVE URL to me (@gmail.com) and also the email in security.md
I can take a look at it for the next few days

I think whitelist/allowlist is not that useful, since it's a bit cumbersome to maintain. It becomes a file in the user's home directly, which hurts code portability (you need the file everywhere the code is run).
HuggingFace recently faced a similar remote code execution vulnerability. Their solution was in two parts:

Add a flag trust_remote_code=True at the  model/tokenizer level, e.g. AutoModelForCausalLM.from_pretrained('gpt2-xl', ..., trust_remote_code=True)
Add a TRUST_REMOTE_CODE environment variable (defaulting to False), which allowed remote code execution for all modules. Here is an example

I feel like (1) is a better long-term fix, but it seems like it could take a while to implement, given the number of modules.
@ekaf @stevenbird does (2) seem like a good idea and doable?
Concrete proposal: A single env variable TRUST_REMOTE_NLTK_CODE, which the user must manually enable in their code via os.environ["TRUST_REMOTE_NLTK_CODE"] = "True", prior to calling nltk modules.

If enabled, then module pickles can be downloaded as usual.
If not enabled, then remote code download/execution is blocked, using something like the patch suggested by @mcepl. Already-downloaded code modules should be executable (but maybe you want to introduce a STRICT_* flag to control this).

I feel like this is a good solution because it gives users control on when to execute remote code, without changing more than 1 line of code.

Experimentation code (in Jupyter notebooks) will probably be okay to set TRUST_REMOTE_NLTK_CODE=True.
Production code (running in Docker/etc) will probably pre-download the module pickles during the build phase, but run live using TRUST_REMOTE_NLTK_CODE=False.

@alvations, it is great to hear that you believe in the "deep clean" solution, where the pickles are completely removed forever.
There is also excellent news from gpt-4o, suggesting to serialize the more complex data types using Python's joblib module:
chatgpt-4o-remedy.txt
That is much easier than previously believed, and I have verified that the serialization works on the three complex pickle classes used with puntkt, maxent_ne_chunker, and maxent_treebank_pos_tagger.
We need to verify that adjusted  loaders can work as expected with the joblib dumps, and if everything goes well, it seems an ETA is within close reach now.

There is also excellent news from gpt-4o, suggesting to serialize the more complex data types using Python's joblib module

joblib uses pickle under the hood as well, so this wouldn't solve the issue.

joblib uses pickle under the hood as well, so this wouldn't solve the issue.

Thanks @Dunedan! Instead, there seems to be a possibility to translate the pickles into Protobuf messages:
protobuf.txt
Would that help?

https://nvd.nist.gov/vuln/detail/CVE-2024-39705. Hi, Is there an ETA?
NVD doesn’t care how we use it, and doesn’t care how we avoid risks. They only care that there may be risks if we use it.

@Dunedan I see the CVSS base metrics says "Privileges required: None". I believe that's inaccurate and has elevated the CVSS score beyond what it should be.
You mentioned 2 ways this could be exploited, and they both seem to require privileges.

Controlling the NLTK data index presumably requires privileges.
Editing pickles in transit as a MITM presumably requires breaking TLS1.3, which requires knowledge of a privileged cryptographic key. If your POC was able to successfully edit pickles in transit, then that's a vulnerability that extends beyond just using pickles to insufficient encryption of data in transit.

Having a built-in downloader makes it harder than IMHO necessary to depend on nltk. I understand that some dependencies are several gigabytes in size, but even that should be manageable with a package manager like poetry/nix reusing a local download.
My vote would be to optionally allow depending on individual nltk data as packages, so that I can e.g. depend on nltk-punkt-english and know that a certain version of this data has been "locked" in my poetry lockfile.

https://github.com/nltk/nltk/issues/2228#issuecomment-1648551210

@Dunedan I see the CVSS base metrics says "Privileges required: None". I believe that's inaccurate and has elevated the CVSS score beyond what it should be.

I wasn't involved in calculating the severity. I believe that was done by Github's Security Curators team. If you believe that metric isn't accurate, there is a link on the page to suggest improvements for the vulnerability details.
That said, from my perspective "Privileges required: None" seems to be correct, as it refers to privileges on the targeted system prior to the attack. See https://www.first.org/cvss/v4.0/specification-document#Privileges-Required-PR for details.

According to our SLA, we would like to have a fix for this in the next 7 days. Otherwise, we are forced to rip out NLTK

@Dunedan, to tamper with a user's files, the attacker needs at least the same privileges as that user.

The attacker gains these permissions as part of the attack. What the "Privileges required" metric is about are privileges the attacker needs prior to the attack. For this vulnerability an attacker doesn't need any privileges on a targeted system prior to an attack.

According to our SLA, we would like to have a fix for this in the next 7 days. Otherwise, we are forced to rip out NLTK

How much are you willing to pay to have a fix in less than 7 days ?

According to our SLA, we would like to have a fix for this in the next 7 days. Otherwise, we are forced to rip out NLTK

Investigating removal and replacement today, not a level of risk we are willing to accept.

Am I right in my understanding that if you are not explicitly running nltk.download then you are not at risk of this vulnerability? When we included nltk in our codebase a year ago, we did a one time download of the tokenizers we needed and then stored that in a location we control. We point NLTK to the data directory using the NLTK_DATA environment variable. If we assume that there has not been any exploitation of this vulnerability yet and the pickles from a year ago are safe, then I should be able to accept this risk? Is there a way to validate whether or not the existing pickles are safe and do not contain malicious code? And then checksum against known secure versions? Not talking about an actual fix just wondering whether I can suppress my vulnerability alert while we wait for a better fix.

following thread...

There are now PRs available, with fixed data packages and nltk handlers for all the suspect packages in nltk_data. So an ETA for the fix should be near.
Meanwhile, it could be useful if more users would review the PRs,
and eventually discuss alternatives. In particular, it would be helpful to confirm that nltk.data loads Json and Yaml data in a safe way.

@ekaf Thank you for the update!
Is this the fixing PR? https://github.com/nltk/nltk/pull/3286/
and is there an expected 3.8.2 release of nltk?

Out of 6 data packages containing pickles, 3 have been replaced by Json and their new handlers were already  merged into the develop branch last week:

help/tagsets
taggers/averaged_perceptron_tagger
taggers/averaged_perceptron_tagger_ru

The following have been replaced by Tab-files and are awaiting review:

tokenizers/punkt (#3283)
chunkers/maxent_ne_chunker (#3286)
taggers/maxent_treebank_pos_tagger

The last package is fixed, but has no handler yet, since that data was not used from nltk anyway.
A release is planned, though I don't know about its version number.

As per Aqua scan this is
CVSS Score NVD CVSSv3 9.8
NLTK through 3.8.1 allows remote code execution if untrusted packages have pickled Python code, and the integrated data package download functionality is used. This affects, for example, averaged_perceptron_tagger and punkt.
There is No Running Workloads, When this fix is expected ?

According to our SLA, we would like to have a fix for this in the next 7 days. Otherwise, we are forced to rip out NLTK

What is alternate you have ?

Hello !
I just saw this CVE pop in my alerts,
I download punkt and bcp47, if I stop downloading them from nltk_data but use my local file am I avoiding the problem ?

Out of 6 data packages containing pickles, 3 have been replaced by Json and their new handlers were already merged into the develop branch last week:

Does it mean that the whole pickle opening will be removed from NLTK?

A release is planned, though I don't know about its version number.

@ekaf Any updates on when we can expect a release that resolves this CVE?

@ekaf An entire week of silence on this issue leaves everyone affected by this CVE scrambling...not knowing whether they have to eliminate the use of this library in their code by the end of July or not. Please respond with an outlook for the release that resolves this CVE.

@ekaf An entire week of silence on this issue leaves everyone affected by this CVE scrambling...not knowing whether they have to eliminate the use of this library in their code by the end of July or not. Please respond with an outlook for the release that resolves this CVE.

This is OSS so the maintainers don't have to give you an ETA. I don't think they gain a dollar for the great work they do (do you want to be a sponsor ?)
FWIW they could be in holidays for a month or this bug could be never fixed.
If it's not acceptable for you, you can:

remove the dependency
live with the CVE an accept the risk (personally I would not download packages at runtime)
fork and fix it yourself
I'm sure the maintainers do what they can. They know very well that their project reputation is at stake.

Thanks @cbornet, I have also wondered about how people could talk about an SLA when this is free software.
However, I am sorry for not responding sooner to @cherylking. The reason is that there is no update for the moment, and your guess is as good as mine concerning the release date.
But the outlook seems reasonable: if we can assume that Json is safe, it should be possible to close this issue before the end of July,

@ekaf Will the remote code execution vulnerability of the NLTK cause problems if we are not explicitly using nltk.download() within the project?
In my project, I am using only its regexp_tokenize (from nltk.tokenize import regexp_tokenize) module and noting else. Will the NTLK vulnerability still cause any troubles in the production environments?

@vishwvir-singh, please ask someone else. I prefer not to make statements about the future, or assume that anything is safe.

Hello, is the fixed version expected to be available on pyPI sometime soon? Thanks!

Hello, is the fixed version expected to be available on pyPI sometime soon? Thanks!

It looks to be being built and run now so yes https://github.com/nltk/nltk/actions/runs/10088968161/job/27895608352

Hi, what's the alternative to be able to work with nltk.sent_toknize() which does load(f"tokenizers/punkt/{language}.pickle") internally?

Hi, what's the alternative to be able to work with nltk.sent_toknize() which does load(f"tokenizers/punkt/{language}.pickle") internally?

@sadra-barikbin, the alternative is #3283. It redefines sent_tokenize() without pickles. Please test it if you can and report any problems.

Any news on this ?

Checking in, as #3283 seems to be approved for a merge as of yesterday.

On Pypi, it's still the 3.8.1 version. Does the nltk.download works again ?

Is there a way to download the "averaged_perceptron_tagger" from nltk now please ?

@nikky78, the most convenient is to use NLTK's high-level pos_tag function, which is unchanged:
from nltk.tag import pos_tag
But you can also call the PerceptronTagger explicitly:
from nltk.tag import PerceptronTagger
tagger = PerceptronTagger()

The same also applies to the other replaced packages.

@nikky78, the most convenient is to use NLTK's high-level pos_tag function, which is unchanged:
from nltk.tag import pos_tag
But you can also call the PerceptronTagger explicitly:
from nltk.tag import PerceptronTagger
tagger = PerceptronTagger()

The same also applies to the other replaced packages.

I'm trying to use langchain UnstructuredMarkdownLoader. But it requires nltk averaged_perceptron_tagger. So I can't use your way I believe.

@nikky78, there should be no difference between the old and the new averaged perceptron tagger object. Please read the code, and report actual differences, instead of vague beliefs.

@nikky78, there should be no difference between the old and the new averaged perceptron tagger object. Please read the code, and report actual differences, instead of vague beliefs.

I do :
loader = UnstructuredMarkdownLoader(markdown_path)
documents = loader.load()
It does not seem to be take a tagger in arguments

It looks like this and the related PRs are merged. But the fix has not yet reached pypi. pypi is still 3.8.1. Could this be released as 3.8.2 or are there any blockers? @stevenbird @ekaf ?

Thanks @noren95,actually #3283 still needs to be merged.

I can do it on Friday.

@stevenbird thank you! we are keeping an eye out for the new release.

Now that NLTK 3.8.2 is released, users need definitively to upgrade. Those who are not yet convinced that this vulnerability is highly critical will soon be, if they have a look at this detailed analysis.
It contains a full Proof-of-Concept of the attack, which is specially scary for those who are using systems that allow you by default to sudo tasks without asking for the root password. It sounds weird, but such systems are actually common.

Can the CVE be informed that 3.8.2 fixes the vulnerability so checkers are not triggered anymore ?

Can the CVE be informed that 3.8.2 fixes the vulnerability so checkers are not triggered anymore ?

Certainly, but how do you inform the CVE?

No idea 😅

You can directly edit/"improve" it at https://github.com/advisories/ghsa-cgvx-9447-vcch/
(which would be https://github.com/advisories/ghsa-cgvx-9447-vcch/improve )

Actually the conf in the NVD is "lessThanOrEqual":"3.8.1" so checkers are OK now.

Now that NLTK 3.8.2 is released, users need definitively to upgrade. Those who are not yet convinced that this vulnerability is highly critical will soon be, if they have a look at this detailed analysis. It contains a full Proof-of-Concept of the attack, which is specially scary for those who are using systems that allow you by default to sudo tasks without asking for the root password. It sounds weird, but such systems are actually common.

The lack of HTTPS takes the vulnerability from “bad practice” to “lethal”

The lack of HTTPS takes the vulnerability from “bad practice” to “lethal”

Thanks @jdwhitaker, #3295 should ensure that the downloader only uses HTTPS.

Hi @ekaf, this package was on pypi yesterday, seems like it has been removed now. Any reason for the same?

The "fix"-release is now 3.9b1, see https://github.com/nltk/nltk/issues/3301#issuecomment-2292549011 for reference.

The "fix"-release is now 3.9b1, see https://github.com/nltk/nltk/issues/3301#issuecomment-2292549011 for reference.

Would it be possible to cut a "non-beta" release ? 3.9.0 ?

Please see 3.9.1

For 3.9.1 to address the NIST CVE, it would mean that the download of unsafe pickles is no longer possible, right? If that's the case, it wouldn't be necessary to start using nltk.download("punkt_tab"), right?
nltk.download("punkt") should now be pulling punkt_tab instead?

@InterferencePattern, the package switching is only implemented for the load() method. But you need to download('punkt_tab').

Hi @ekaf, sorry for asking, but if I was simply using [sent_tokenize()](https://github.com/nltk/nltk/blob/develop/nltk/tokenize/__init__.py#L109) without explicitly downloading anything, do I have to take action too? That function also uses the punkt tokenizer but I couldn't quite understand if it also downloads a pickle when initializing the PunktTokenizer...

Yes @cornzz, sent_tokenize() used to rely on a pickle, and was thus vulnerable to a potentially lethal attack. So nobody should be on 3.8.1. You need to upgrade to NLTK 3.9.1.
Concerning a migration guide, if you use high-level functions nothing changes, as shown in these examples.
And if you need to use low-level load functions, please have a look at how the switch_* functions in data.py are called (L818-829) and implemented (L667-731).

I would like to use langchain UnstructuredMarkdownLoader like this:
from langchain.document_loaders import UnstructuredMarkdownLoader
loader = UnstructuredMarkdownLoader(markdown_path)
documents = loader.load()

But I get the error:
  Resource punkt_tab not found.
  Please use the NLTK Downloader to obtain the resource:

Is there a way with the new version to do nltk.download('punkt_tab') ?

I would like to use langchain UnstructuredMarkdownLoader like this:
from langchain.document_loaders import UnstructuredMarkdownLoader
loader = UnstructuredMarkdownLoader(markdown_path)
documents = loader.load()

But I get the error:
  Resource punkt_tab not found.
  Please use the NLTK Downloader to obtain the resource:

Is there a way with the new version to do nltk.download('punkt_tab') ?

@nikky78
Did you get a similar error mentioned here https://github.com/langchain-ai/langchain/issues/25609
nltk.download('punkt_tab') should work.

Sharing my upgrade from 3.8.1 to 3.9.1, tokenizer use case:
NLTK 3.8.1:
# download data
import nltk
nltk.download()

# put pickles to a local folder manually

# at runtime
import nltk
file_name = f'{NLTK_MODEL_PATH}/{lang}.pickle'  # lang e.g. 'english'
tokenizer = nltk.data.load(file_name)
tokenizer.tokenize(text)

NLTK 3.9.1:
# download data
import nltk
nltk.download('punkt_tab')

# put the nltk_data folder contents (with folder structure preserved) to a local folder
# e.g. local_path/tokenizers/punkt_tab/english/<four files here>

# at runtime
import nltk
from nltk.tokenize import PunktTokenizer
nltk.data.path.append(NLTK_MODEL_PATH)
tokenizer = PunktTokenizer(lang)  # lang e.g. 'english'
tokenizer.tokenize(text)

Hello! I've upgrade to 3.9.1, but still cant complete the download. Please suggest, what I might be missing!


Comment out workunit code and tests

A pull request to turn off the work unit to job association that seems somewhat problematic.

Should we backport it to 1.1.6_wmagent branch? @ticoann

I thought this stuff wasn't in that branch. This is a placeholder in the (likely) case we don't fix the underlying problem before I leave. :-)

Yes workunit code change is not in wmagent branch

@ticoann I guess we'll have to merge it today such that we can cut the new wmagent release candidate tag.


dataset

excuse me, can you  introduce how to get the dataset needed for training? i did not get it . thanks!

we will release it when the paper is accept!


unify form controls

Reticketing from #3220.

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/9962545-unify-form-controls?utm_campaign=plugin&utm_content=tracker%2F85909&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F85909&utm_medium=issues&utm_source=github).

How is this different from #3267 ?

It's not, sorry.


[Usage] AttributeError: 'Parameter' object has no attribute 'quant_state'

Describe the issue
Issue: Trying to do inference on a LoRA adapter created from finetuning on  liuhaotian/llava-v1.5-13b base model.
Due to an error, I have modified llava/server/cli.py so that
        with torch.inference_mode():
            output_ids = model.generate(
                input_ids,

is changed to
        with torch.inference_mode():
            output_ids = model.generate(
                input_ids=input_ids,

But there is now a new error shown below.
Here are my installed pip packages
torch                     2.0.1
bitsandbytes              0.41.0
flash-attn                2.3.2
llava                     1.1.3        /home/x/LLaVA
peft                      0.4.0
transformers              4.31.0

Command:
python -m llava.serve.cli --model-path ./checkpoints/llava-v1.5-13b-lora-test --model-base ./base/llava-v1.5-13b --image-file ./data/testimg.jpg --load-4bit

Log:
[2024-01-29 22:03:11,676] [INFO] [real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda (auto detect)
Loading LLaVA from base model...
Loading checkpoint shards: 100%|██████████████████████████████████| 3/3 [00:14<00:00,  4.73s/it]
Loading additional LLaVA weights...
Loading LoRA weights...
Model is loaded...
USER: Foo bar baz
ASSISTANT: FP4 quantization state not initialized. Please call .cuda() or .to(device) on the LinearFP4 layer first.

Traceback (most recent call last):
  File "/home/x/anaconda3/envs/llava/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/x/anaconda3/envs/llava/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/x/temp/LLaVA/llava/serve/cli.py", line 125, in <module>
    main(args)
  File "/home/x/temp/LLaVA/llava/serve/cli.py", line 94, in main
    output_ids = model.generate(
  File "/home/x/anaconda3/envs/llava/lib/python3.10/site-packages/peft/peft_model.py", line 977, in generate
    outputs = self.base_model.generate(**kwargs)
  File "/home/x/anaconda3/envs/llava/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/x/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/generation/utils.py", line 1588, in generate
    return self.sample(
  File "/home/x/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/generation/utils.py", line 2642, in sample
    outputs = self(
  File "/home/x/anaconda3/envs/llava/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/x/anaconda3/envs/llava/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "/home/x/temp/LLaVA/llava/model/language_model/llava_llama.py", line 88, in forward
    return super().forward(
  File "/home/x/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 824, in forward
    logits = self.lm_head(hidden_states)
  File "/home/x/anaconda3/envs/llava/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/x/anaconda3/envs/llava/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "/home/x/anaconda3/envs/llava/lib/python3.10/site-packages/bitsandbytes/nn/modules.py", line 249, in forward
    out = bnb.matmul_4bit(x, self.weight.t(), bias=bias, quant_state=self.weight.quant_state)
AttributeError: 'Parameter' object has no attribute 'quant_state'

It might be lm_head raising this error. Strangely I checked a few layers and they are all 0s.
# model.base_model.model.lm_head.weight
torch.Size([32000, 5120])

# (model.base_model.model.lm_head.weight == 0).all()
tensor(True, device='cuda:0')

# model.base_model.model.lm_head.weight.shape
Parameter containing:
tensor([[0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.],
        ...,
        [0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.]], device='cuda:0', dtype=torch.float16)

# getattr(model.base_model.model.lm_head.weight, 'quant_state')
AttributeError: 'Parameter' object has no attribute 'quant_state'

I dont remember if the LoRA adapter was trained using the same pip packages as the inference env.
Anyone knows a fix? Thanks

Same quesion here.

same question here. Anyone has solved?

Any updates here?

I am also facing the same error. Did anyone solve it ?


[Error] Not enough columns given to draw the requested chart

Version
react-google-charts 1.5.3
react ^15.3.0
Expected behaviour
Render area chart without any error.
Actual behaviour
Throwing error while rendering area chart
Steps to reproduce

Provide below sample data

[
    ['CL1', 'CL2, 'CL3', 'CL4', 'CL5'],
    ["String", 100, 100, 100, 10],
    ["String", 1231, 123, 12, 123]
]

Use Chart component

<Chart
        chartType="ScatterChart"
        data={this.state.data}
        options={this.state.options}
        graph_id="ScatterChart"
        width="100%"
        height="400px"
        loader="Loading Chart..."
      />

Error Screenshot

Please check example scatter chart props here : https://github.com/RakanNimer/react-google-charts/blob/master/src/constants/SAMPLE_DATA.js#L115
and in the examples.
Going to close this issue, please feel free to re-open if it persists.

@RakanNimer
Just updated the content, this is related to AreaChart. Chart gets rendered but throws an error as well.

Thanks for the clarification !
If you could provide a link to a repo that reproduces your issue then it would make it easier for me or one of the other contributors to help.

This issue is related to google charts.
Thanks.


Allow Running the script with root priviledge

Hello,
I'm using kali linux which comes with a single root user. Whenever I try to run wine-portable-5.2-staging-amd64.sh I get this message:

Do not run this app as root!

Can you change this behaviour?

Hello. I've added such ability and reuploaded the 5.2 build. You can now run it as root like this:
ALLOW_ROOT=1 ./wine-portable-5.2-staging-amd64.sh


"E3035 DeletionPolicy should be only one of Delete, Retain, Snapshot" need addition of

CloudFormation Lint Version
cfn-lint 0.79.7
What operating system are you using?
MasOS 13.5.1
Describe the bug
E3035 finds "DeletionPolicy: RetainExceptOnCreate" as an error but it should be allowed as it has been added to Cloudformation.
https://aws.amazon.com/about-aws/whats-new/2023/07/aws-cloudformation-deletion-policies-dev-test-cycle/
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-deletionpolicy.html
Expected behavior
I expected to not error as RetainExceptOnCreate is currently a valid DeletionPolicy: value.
Reproduction template
---
AWSTemplateFormatVersion: 2010-09-09

Parameters:

  AccessSubnetCidrs:
    Type: CommaDelimitedList

  EnterpriseVpcId:
    Type: String

Resources:
  AZaAccessSubnet:
    Properties:
      AvailabilityZone:
        Fn::Select:
        - 0
        - Fn::GetAZs: ''
      CidrBlock: !Select [0, !Ref AccessSubnetCidrs]
      VpcId: !Ref EnterpriseVpcId
    DeletionPolicy: RetainExceptOnCreate
    UpdateReplacePolicy: Retain
    Type: AWS::EC2::Subnet

You sure you are on v0.79.7?
This was fixed in #2834  which was included in the v0.79.7 release.
Willing to reopen as needed.


VPC module - PSA configurable service producer

Checklist

I applicable, I acknowledge that I have:

[X] Read the contributing guide
[X] Ran terraform fmt on all modified files
[X] Regenerated the relevant README.md files using tools/tfdoc.py
[X] Made sure all relevant tests pass

@spica29 the PR looks reasonable but I'm wondering what's the use case for this. I've never seen any other value besides "servicenetworking.googleapis.com".

@spica29 the PR looks reasonable but I'm wondering what's the use case for this. I've never seen any other value besides "servicenetworking.googleapis.com".

Hi, we are trying to set up a PSA for NetApp and it uses another service: "netapp.servicenetworking.goog". Here is the documentation. I believe that we often just see GCP as a service producer, but we can also have third parties. There is not a lot of documentation about it though.


How to use scroll bar

I'm using this library to create a database diagram, everything works, however I have tables with many columns and they end up being very large, would it be possible to use a scroll bar while maintaining the connections?


remove uneeded itemgroup

Addresses #12249

There are two PR's for this. We can merge whichever one passes the CI first 😎

This one came in second! https://github.com/aspnet/AspNetCore/pull/12247 was the winner


Example for reading an RTSP stream and republishing all tracks to a server

This may be a weird usecase, but I'm trying to use gortsplib to read an RTSP stream from a camera, then publish it to an endpoint configured on rtsp-simple-server. I've attempted to do so myself, but my naive attempt isn't working correctly. See kaiiorg/client-read-publish. For the record, I'm new to Go, so I may have committed some cardinal sin without realizing; sorry for that.
Additional information:

I'm using rtsp-simple-server 0.17.13.
rtsp-simple-server's config is the default provided with the linux binary, but the paths key was modified:

paths:
  PublishedStream:
    source: publisher
  ExampleStream:
    # Just using rtsp-simple-server for consistency, a realistic usage would be connecting directly to a camera.
    source<EMAIL_ADDRESS>

ffplay works when I stream the ExampleStream:

ffplay.exe -x 500 -rtsp_transport tcp "rtsp://<IP_ADDRESS>:8554/ExampleStream"

ffplay does not work when I attempt to stream my republished stream:

ffplay.exe -x 500 -rtsp_transport tcp "rtsp://<IP_ADDRESS>:8554/PublishedStream"

Setting both rtsp-simple-server and my test program to use TCP as the transport makes no change

Full Usecase
One of the requirements of the program I'm writing is to:

Get request via HTTP or MQTT
Read the stream from the IP camera

This may require adding a header to the RTSP request(s) to stream stored files on the camera instead of the live stream. Its weird, I know.

Publish the stream to an instance of rtsp-simple-server. The publish url would be in the HTTP/MQTT message.
This issue would handle parts of 2 and 3.

I have also tried using the unmodified client-publish-h264 example, publishing to rtsp-simple-server, and am getting the same errors and similar image distortion via both ffplay and vlc.

Hello, this can be done with gortsplib with a little work, but in my opinion a simpler solutions consists in using ffmpeg, that allows to perform this without writing any code:
ffmpeg -re -i rtsp://original-camera -c copy -f rtsp rtsp://rtsp-server

I'll see if I can get that to work, but it might be better for my particular usecase if I don't have any additional binaries or subprocesses to worry about.

A working example that shows how to read and republish tracks has been added here.

Awesome. Thanks!


Bug fixes and changes for cypress tests

Reolved https://github.com/LiveChurchSolutions/ChumsApp/issues/33


Bug report downloading lemonde.fr video

Hi everyone,
I launched this command :
 youtube-dl --verbose http://www.lemonde.fr/les-decodeurs/article/2016/10/18/tout-comprendre-du-ceta-le-petit-cousin-du-traite-transatlantique_5015920_4355770.html
Here is the error log :
[debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'--verbose', u'http://www.lemonde.fr/les-decodeurs/article/2016/10/18/tout-comprendre-du-ceta-le-petit-cousin-du-traite-transatlantique_5015920_4355770.html'] [debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2017.02.11 [debug] Python version 2.7.12 - Linux-4.4.0-62-generic-x86_64-with-Ubuntu-16.04-xenial [debug] exe versions: ffmpeg 2.8.10-0ubuntu<IP_ADDRESS>, ffprobe 2.8.10-0ubuntu<IP_ADDRESS> [debug] Proxy map: {} [Lemonde] tout-comprendre-du-ceta-le-petit-cousin-du-traite-transatlantique_5015920_4355770: Downloading webpage ERROR: Unable to extract digiteka url; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. Traceback (most recent call last):   File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 696, in extract_info     ie_result = ie.extract(url)   File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 369, in extract     return self._real_extract(url)   File "/usr/local/bin/youtube-dl/youtube_dl/extractor/lemonde.py", line 33, in _real_extract     webpage, 'digiteka url', group='url'))   File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 672, in _search_regex     raise RegexNotFoundError('Unable to extract %s' % _name) RegexNotFoundError: Unable to extract digiteka url; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like that [x])
Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.02.11. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

[x ] I've verified and I assure that I'm running youtube-dl 2017.02.11

Before submitting an issue make sure you have:

[x ] At least skimmed through README and most notably FAQ and BUGS sections
[x ] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[x ] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
$ youtube-dl -v <your command line>
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'http://www.youtube.com/watch?v=BaW_jenozKcj']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version 2017.02.11
[debug] Python version 2.7.11 - Windows-2003Server-5.2.3790-SP2
[debug] exe versions: ffmpeg N-75573-g1d0487f, ffprobe N-75573-g1d0487f, rtmpdump 2.4
[debug] Proxy map: {}
...
<end of log>

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Single video: https://www.youtube.com/watch?v=BaW_jenozKc
Single video: https://youtu.be/BaW_jenozKc
Playlist: https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc

Note that youtube-dl does not support sites dedicated to copyright infringement. In order for site support request to be accepted all provided example URLs should not violate any copyrights.

Description of your issue, suggested solution and other information
Explanation of your issue in arbitrary form goes here. Please make sure the description is worded well enough to be understood. Provide as much context and examples as possible.
If work on your issue requires account credentials please provide them or explain how one can obtain them.

Fixed in PR #12116
Waiting for Review...

If you need the video right now you can use the following file 'lemonde.py':
https://raw.githubusercontent.com/ThomasChr/youtube-dl/fixlemonde/youtube_dl/extractor/lemonde.py


assign_initiatives_identity.bicep fails

 az deployment mg create --template-file ./infra-as-code/bicep/assign_initiatives_identity.bicep --location $location --management-group-id $identityManagementGroup --parameters ./infra-as-code/bicep/parameters-complete-identity.json

{"status":"Failed","error":{"code":"DeploymentFailed","target":"/providers/Microsoft.Management/managementGroups/alz-Identity/providers/Microsoft.Resources/deployments/assign_initiatives_identity","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.","details":[{"code":"ResourceDeploymentFailure","target":"/providers/Microsoft.Management/managementGroups/alz-Identity/providers/Microsoft.Resources/deployments/zpzxwaie64mx2-Alerting-Identity","message":"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.","details":[{"code":"DeploymentFailed","target":"/providers/Microsoft.Management/managementGroups/alz-Identity/providers/Microsoft.Resources/deployments/zpzxwaie64mx2-Alerting-Identity","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.","details":[{"code":"InvalidCreatePolicyAssignmentRequest","message":"The policy definition specified in policy assignment 'ALZ-Monitor_Identity' is out of scope. Policy definitions should be specified only at or above the policy assignment scope. If the management groups hierarchy changed recently or if assigning a management group policy to new subscription, please allow up to 30 minutes for the hierarchy changes to apply and try again."}]}]}]}}

Fixed myself


Update BCV control for field user testing

The UX team needs to be able to run this from Windows. Creating a build for them will be helpful.  The testing will happen on April 27, so it would be important to have this done and verified working by the UX team by around April 25 at the latest.

[ ] Add a BCV control inside of a tab so it is working (globally)
[ ] Global BCV in the same style as the one inside the tab
[ ] Scroll groups working so that the BCV in tabs may control things globally or just control content inside the tab  Scroll groups are in #788. Talk with UX when you start working on this to make sure it is clear.

Package 1

[ ] per-tab BCV navigation
[ ] independent scroll (optional)
[ ] scroll groups (optional)

Package 2

[x] global BCV navigation (done)
[ ] shadcn/UI (optional)

Package 3 (optional)

[ ] both per-tab and global BCV navigation
[ ] scroll groups

When you feel confident about what's achievable on this task, please could you let us know you think will be implemented? What gets implemented will determine how we will write our test scripts.
(Also I don't know what "independent scroll" refers to in @merchako 's "package 1". Can anyone clarify?)

If there’s no scroll groups, we risk getting unuseful reactions from users. So when there are multiple tabs, should they use the same reference or be separate?

if separate, maybe users get scared of no scroll sharing
if together, maybe the flexibility isn’t apparent

@mikelothers says most people will want to have synchronized scrolling. It only makes sense to sync scrolling if we have multiple resources.

@paranext/paranext-ux the builds are now linked in the issue description! Let me know if you have any problems. At this point, I will likely only be able to respond to major issues that break the tests.

@tjcouch-sil Thanks for these, they look great.
There's just one behaviour which I think might be problematic enough to be worth fixing if you are able: The first time I click in the BCV control, it opens the dropdown with GEN at the top of the list (not with the current book and chapter in view). If I click away and then click back in the control, or if I press the down arrow on the keyboard, the current book and chapter appear located and selected, as expected.


[bug?] X/Y co-ordinate on TouchEvent/iOS button nulls the action

this is something ive just come across. after doing some lite tests. but using X/Y co-ordinates for the position of a SimpleButton makes the button null in its use.
once i take away the X/Y co-ordinates, it works.
though using it on desktop/neko, works fine as well. with or without the X/Y
but this is testing on an iOS device running latest public iOS. i think that 8.4
here is the code i have drummed up to test. super simple. but just uses SimpleButton & TouchEvent
package;

import openfl.display.Sprite;
import openfl.display.SimpleButton;
import openfl.display.Shape;
import openfl.events.TouchEvent;

class Main extends Sprite {
  public function new (){
    super();

    var button:SimpleButton = new SimpleButton();
    //this works but if you...
    //uncomment these below then run again on a iOS device, and it doesnt work
    // button.x = 300;
    // button.y = 300;

    button.upState = createShape(0x0ff0f0, 300, 150);
    button.overState = createShape(0x0ffff0, 300, 150);
    button.downState = createShape(0xfff0ff, 300, 150);
    button.hitTestState = createShape(0xfff, 300, 150);
    button.addEventListener(TouchEvent.TOUCH_BEGIN, onTouchBegin);
    addChild(button);
  }

  private function createShape(color:UInt, width:Int, height:Int){
    var shape:Shape = new Shape();
    shape.graphics.lineStyle(1, 0x000000);
    shape.graphics.beginFill(color);
    shape.graphics.drawRect(x,y,width,height);
    shape.graphics.endFill();
    return shape;
  }

  private function onTouchBegin(event:TouchEvent):Void {
    trace('TOUCH BUTTON PRESSED');
  }
}

ok. so i tried another test. this time using nape physics. because i also need that for this project ;)
but that also is in the same camp as the button. though when i do test through xcode, using the ipad, it does register the touch, but does not move the object
here is the nape code i was using. and yes, does print out its being grabbed, but not moving at all. vi etried a few ways, and am just stuck here
package char;

import openfl.display.Sprite;
import openfl.ui.Mouse;
import openfl.events.MouseEvent;
import openfl.events.TouchEvent;
import openfl.events.Event;
import openfl.Assets;
import openfl.Lib;

import nape.phys.Body;
import nape.phys.BodyType;
import nape.constraint.PivotJoint;
import nape.space.Space;
import nape.shape.Circle;
import nape.geom.Vec2;
import nape.util.ShapeDebug;

class Player extends Sprite {
  private var _player:Body;
  private var _playerJoint:PivotJoint;
  private var _playerPoint:Vec2;

  public function new(space:Space){
    super();
    _player = new Body(BodyType.DYNAMIC);
		_player.shapes.add(new Circle(100));
		// _player.position.setxy(300,300);
		_player.space = space;

    _playerJoint = new PivotJoint(space.world, null, Vec2.weak(), Vec2.weak());
    _playerJoint.space = space;
    _playerJoint.active = false;
    _playerJoint.stiff = false;
    // this.graphics.beginFill(0xfff);
    // this.graphics.drawCircle(x,y,30);
    Lib.current.stage.addEventListener(TouchEvent.TOUCH_BEGIN, onTouchDown);
    Lib.current.stage.addEventListener(TouchEvent.TOUCH_END, onTouchUp);
    Lib.current.stage.addEventListener(Event.ENTER_FRAME, onEnterFrame);
  }

  private function onEnterFrame(event:Event):Void {
    if (_playerJoint.active){
      _playerJoint.anchor1.setxy(mouseX, mouseY);
    }
  }

  private function onTouchDown(event:TouchEvent):Void {
    _playerPoint = Vec2.get(mouseX, mouseY);
    _playerJoint.body2 = _player;
    _playerJoint.anchor2.set(_player.worldPointToLocal(_playerPoint, true));
    _playerJoint.active = true;
    trace('PLAYER GRABBED');
  }

  private function onTouchUp(event:TouchEvent):Void {
    _playerJoint.active = false;
  }
}

also the touch on my track pad does work with neko, and it moves if i just tap. just touch is not working for movement n the pad

Maybe this bug is related to #776 . It's so confusing.
Core contributors, Will you please give me some clues? I would like to digg into the source.
This bug make iOS target almost unusable with -Dnext.

@zengjie thanks. ill take a look


Toggle shortcut

Is there an easy way to TOGGLE remote explorer window ON and OFF, by a shortcut?

Yes, there is :)
https://github.com/mgrenier/remote-ftp#keyboard-shortcuts


AttributeError: 'module' has no attribute 'get_installed_distributions'

Context
When deploying a project with "zappa deploy [project-name]" I get the error: AttributeError: 'module' has no attribute 'get_installed_distributions'
I honestly do not know what is wrong or what I am missing. Any help please provide simple step by step instructions to solve this.

Upgrading virtualenv fixed this for me.
sudo pip install virtualenv --upgrade
Please try that and see if the problem is fixed.

Thanks, but I did upgrade virtualenv but it didn't work. I still get the error message.

You probably have a newer pip, they made changes to their public api top level functions recently. Try downgrading with pip install pip==9.0.1 or to 8.1.2.

I also got this to work by running sudo pip3 install virtualenv --upgrade. Thank you for your response @varunbalupuri !

Closing. Comment and we'll reopen if it's a Zappa problem.

AttributeError: 'pip' has no attribute 'get_installed_distributions'
For pip == 10.0.0
You can downgrade pip to >= 9.0.0 to fix this

Thank you @neotheicebird!  I downgraded and it fixed the issue for me.

@scoates how come you don't consider this as a Zappa issue? adjusting to pip is better than downgrading to Zappa

There are already two open issues about adapting to pip's new api (aka subprocess): #1478 #1446

I'm running into this issue. I tried downgrading pip and can confirm it is done because pip --version returns 9.0.3 but i still get the same error from zappa

@shivampaw sudo pip3 install virtualenv --upgrade

nvm, turns out rebooting does fix problems...

Installing pip==9.0.3 after activating the virtualenv works for me:
pip install virtualenv
virtualenv venv
. ./venv/bin/activate
pip install pip==9.0.3

I know this issue is closed, but this is obviously still ongoing, and I'd love if we could find a way to patch it up. Anyone clear on the mechanics of what's going wrong here? I'm happy to get my hands dirty, but only if I know what I'm doing.

My comment above was indeed in error, but the root of this problem is the pip changes, I believe, now.
#1478 + #1481

Hello all.
The prob is def pip. Broken in 10.x.
However, depending on your environment, you may need to downgrade pip or pip3. For my fedora box it was pip and my windows machine was pip3. Just depends which interpreter you're using for the problem file/package.

Downgrading to pip 9.0.3 (both globally and in virutualenv) worked for me too. But this sounds like a Zappa issue, not a pip issue.

True. It’s that pip updated location/structure of some internal resources. Other libraries that leverage resources internal to pip can become broken when pip makes changes.

I too have had this issue, downgrading pip in the virtualenv resolved this

For pip 10 use from pip._internal.utils.misc import get_installed_distributions

Breaks Zappa. None of the above suggestions worked for me.

@MoBattah Exact same issue, yet you couldn't get it to find the APIs even when you're running the old version? Mind posting up prints of pip -V inside and outside your virtualenvs so we can troubleshoot?

This 100% doesn't work with pip==10.0.1 and definitely does work with pip==9.0.

Following WonderMr's post, this worked for me:
if pip.__version__ >= "10.0.0":     from pip._internal.utils.misc import get_installed_distributions else:     from pip import get_installed_distributions

downgrade pip and remove slim-handler if you have that option set also

For me it was setting the runtime properly in zappa_settings.json


chore(metadata): ensure build metadata is up-to-date

Description
Please explain the changes you made here.
Which issue(s) does this PR fix

Fixes https://issues.redhat.com/browse/RHIDP-1414

PR acceptance criteria
Please make sure that the following steps are complete:

[ ] GitHub Actions are completed and successful
[ ] Unit Tests are updated and passing
[ ] E2E Tests are updated and passing
[ ] Documentation is updated if necessary (requirement for new features)
[ ] Add a screenshot if the change is UX/UI related

How to test changes / Special notes to the reviewer

/cherry-pick 1.1.x


Decompose fit

I am attempting to decompose_fit, which results in the following error.
OSError: Cannot save file into a non-existent directory: '/Users/yi1/Desktop/sigpro_decompose/Decompose_Solution/Signatures'
I can work around this by manually creating the said directory, but this is not ideal. Could you fix this?
I am using Sonoma 14.6.1 and running the analysis on venv. Here is the pip freeze output.
about-time==4.2.1
alive-progress==3.1.5
chardet==5.2.0
contourpy==1.3.0
cycler==0.12.1
fonttools==4.53.1
grapheme==0.6.0
joblib==1.4.2
kiwisolver==1.4.5
matplotlib==3.9.2
numpy==1.26.4
packaging==24.1
pandas==1.5.3
patsy==0.5.6
pillow==10.4.0
PyMuPDF==1.24.9
PyMuPDFb==1.24.9
pyparsing==3.1.4
pypdf==4.3.1
python-dateutil==2.9.0.post0
pytz==2024.1
reportlab==4.2.2
scikit-learn==1.5.1
scipy==1.14.1
setuptools==74.0.0
SigProfilerAssignment==0.1.8
SigProfilerMatrixGenerator==1.2.28
sigProfilerPlotting==1.3.24
six==1.16.0
statsmodels==0.14.2
threadpoolctl==3.5.0

Sorry, I fixed this by myself, but this could be useful for someone else.
The error happens when I execute this:
Analyze.decompose_fit(samples = "path-to-matrix", output="~/Desktop/sigpro_decompose/", signatures="path-to-sigs", genome_build="GRCh37")
By changing the output argument to "sigpro_decompose", I was able to run this.


Best way to close/unmount ViroARscene without performance issues

Development OS: Mac
Device OS & Version: Android / iOS (all)
Version: ViroReact 2.9.1 and React Native 0.55.3

Description
I am having performance issues when i want to unmount my AR scene. It seems to happen on all devices (Android and iOS) in prod/release mode. When i "close" the scene, the experience slows down to a halt and sometimes causes the app to crash.
I am using React Navigation and the AR is initiated from the one stack which launches another. The Launch doesn't seem to be be the issue, rather the closing.
This is a basic example of my setup;
The ARScreen index.js
class ARScreen extends Component {
  getARNavigator = () => (
    <ViroARSceneNavigator
      apiKey={Config.VIRO_API_KEY}
      initialScene={{ scene: InitialARScene }}
      style={styles.flex}
    />
  )

  render = () => {
    const { navigation } = this.props
    return (
      <View style={styles.flex}>
        <StatusBar hidden />
        {this.getARNavigator()}
        <CloseButton onPress={() => navigation.navigate('Home')} />
      </View>
    )
  }
}

<CloseButton /> is positioned over the scene itself which takes over the entire screen.
I think, ideally, i need the onPress to somehow unmount or close or stop the AR scene before attempting to navigate to 'Home'

Hi @alexborton ,
Can you provide how your stack / routes are setup? What kind of navigator are you using? What do you mean by "the AR is initiated from the one stack which launches another"? Another stack? or Another route? Does the user navigate to ARScreen from Home screen, or some place else?
We use React Navigation along with ViroARScene in our Paramount AR app -> Play Store, App Store. We havent seen any memory leaks / performance issues with react navigation there.
It'd be good to see how your routes/screens are laid out so we can try to reproduce on our end.
Thanks

I got the same issue, also using React Navigation.
On navigating back from the AR-View to another, using navigator.pop(). The app slows down extremly till the AR-View gets unmounted, but this take 5-10 seconds.

My app crashes when I try to remove ViroARSceneNavigator from scene. Did you have this error?

Hi @MarvinErnst ,  @kesha-antonov ,
What version of ViroReact and React-Native are you guys using? Can you elaborate how your stack / routes are set up? As I commented earlier, we haven't seen memory leaks / degraded performance issues with React Navigation and ViroARScene in our Paramount AR App. Having source code / stack setup where you guys are seeing these issues would help us a lot in reproducing them on our end and looking into any possible memory leaks.
Thanks

Hi @manbod ,
react-viro: 2.10.0,
react-native: 0.55.4
Base of the App is a  StackNavigator from 'react-navigation'.
 <RootStackstyle={generalStyle.container} />
From there the user navigates to a list page  or map page (the result is the same for both) and from there to the details page where the AR is embeded
 this.props.navigation.navigate('Detail', {       itemId: id,    });`
This the the AR component
     <ViroARSceneNavigator {...sharedProps} initialScene={{ scene: () => (             <ViroARScene>               <ViroARImageMarker               target="tafel1">               <ViroVideo                 source={moenchVideo}                 rotation={[-90, 0, 0]}                 position={[1, 1, 0]}                 scale={[1, 1, 1]}                 opacity={0.85}                 volume={0.5}                 materials={['chromaKeyFilteredVideo']}               />               <ViroSound paused={false} muted={false} source={sound} loop volume={1.0} />             </ViroARImageMarker>             </ViroARScene>           ),         }}       />
If i then go back using   this.props.navigation.pop(); the app is lagging till the component gets unmounted (5-10s);
I hope you can help :)

My app crashes when I try to remove ViroARSceneNavigator from scene. Did you have this error?

same!
You fixed?

Hi all, have anyone managed to solve this issue? I'm experiencing app irresponsibility even in @achuvm 's NavTest when navigating back from the second screen with ViroARSceneNavigator. Sometimes it works correctly, but most times I'm not able to interact with the app at all. iPhone SE, iOS 12.

I'm having the same issue here. I put my ViroARSceneNavigator in a modal and when i close the modal the modal animation becomes very laggy

I'm having the same issue here. I put my ViroVRSceneNavigator in a modal and when i close the modal the modal animation becomes very laggy

@SatyaFariz +1, my app becomes unusably slow after closing a modal containing ViroARSceneNavigator.

Also having the same issue, my app freezes up after navigating to a different screen

@kesha-antonov https://github.com/viromedia/viro/issues/248#issuecomment-385126647 fixed it for me

Hi @bryantAXS,
Actually, I think I've come up with a workaround, it seems like this is caused when running the app in debug mode while connected to Xcode.
If you open up your app after disconnecting from the debug session, then it no longer deadlocks. Another way you can do this is to click on your target virodemo next to the Stop button -> Edit Schemes -> Run -> Options -> GPU Frame Capture -> Set to Disabled.
Once you do that, your app should no longer freeze after leaving the AR view.
It looks like there's some interaction with Xcode's debugging tools and Viro that's causing this deadlock when destroying the ViroARSceneNavigator.
See image below:

I'm experiencing a similar problem.
When I close the screen with viro using react-navigation, everything works fine. However, when the ViroArSceneNavigator is removed while remaining on the same screen, the app crashes.
To reproduce:
constructor(props) {
  super(props)

  this.state = {
    show: true
  }
}

componentDidMount() {
  setTimeout(() => this.setState({ show: false }), 5000)
}

render() {
  return (
    <>
      {this.state.show && (
          <ViroARSceneNavigator ..... scene with some ARMarkers.... />
      )}
    </>
  )
}

@kesha-antonov #248 (comment) fixed it for me

Hi @bryantAXS,
Actually, I think I've come up with a workaround, it seems like this is caused when running the app in debug mode while connected to Xcode.
If you open up your app after disconnecting from the debug session, then it no longer deadlocks. Another way you can do this is to click on your target virodemo next to the Stop button -> Edit Schemes -> Run -> Options -> GPU Frame Capture -> Set to Disabled.
Once you do that, your app should no longer freeze after leaving the AR view.
It looks like there's some interaction with Xcode's debugging tools and Viro that's causing this deadlock when destroying the ViroARSceneNavigator.
See image below:

it work for me! thánks.


Display toplist for the last 30 days

In order for new IT students to be able to climb to the top, there should be a list which only uses data from the last 30 days. That means you need to be in the Hub in order to keep the points.

Duplicate of issue #24


How to test if a thunk has been dispatched inside another thunk?

Is it possible to test if a thunk has been dispatched inside another thunk without checking all of its actions creators?
I created this example on Codesandbox to express what I mean (src/utils/functions directory). I can't mock the function bar, if it's called from the same file/function. I think it is because of closure.
I found out this question, but it doesn't suit me, because foo, bar and other thunks can be wrapped by another function. The wrapper function accepts configuration parameters, that should be shared among all the inner functions. You can see such example in src/utils/wrappedFunctions. Also the solution in the question looks like hack, workaround.
Thank you in advance.

Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.


imagemin minifies images in src folder, not destination

I just wanted to say:
In my opinion, there was no bug. It worked correct, src files were not touched – at least on windows ;-)
Our configuration was exactly the imagemin example.


VSCodeInline template html element selection brokenish

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.79.2
OS Version:  Ventura 13.4

Steps to Reproduce:

use an inline HTML template (doing this with angular currently)
click on an element and see that it highlights the whole template string rather than just the matching html elements

I've disabled all extension and using the default theme and it still does it, it's not really a blocking visual bud but kinda annoying when editing inline html.  any help fixing this would be greatly appreciated!
See attached file

Edit:
I've also tried installing the highlight matching tag extension and didn't see to help the issue. but in regualr html files VScode does this out of the box so I'm curious how to get it working for inline templates...

Ok I figured out this is an issues with the occurrencesHighlight setting and doesn't seem to play nicely with inline templates, if I turn the setting off the border goes away but then I loose out on highlighting occurrences...

Occurrences highlighting now seems to be disabled on multi-line template strings like this