Authenticate with instance profile

If you leave aws_access_key_id and aws_secret_access_key out of the config file you'll get this error:
Required configuration option for an AWS account (aws_access_key_id) in section "AWS account1" is not configured!

It's nice not to leave passwords on the filesystem - typically boto will authenticate with the instance profile if key and keyid are left off the connection.

Removing the access key altogether would make #16 much more difficult.  You'd have to give the machine iam assumerole access and swap between accounts to query the ec2 data.
Not impossible, just more setup work and if not done properly security risk.

If you don't care about the multi account issue, you should be able to just toggle the config options to not required.
https://github.com/TerbiumLabs/check-reserved-instances/blob/master/src/check_reserved_instances/config.py#L48
just change to False on lines 48-49  and rebuild/install

I think that's possible with one IAM role.  These docs cover the basic idea:
http://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_cross-account-with-roles.html
http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_policy-examples.html
Basically, the script would still use one machine role/set of credentials.  But that role would have access to the other accounts.

Yep.  The code would need to be updated to use iam assumerole in the boto section.
http://boto3.readthedocs.io/en/latest/reference/services/sts.html#STS.Client.assume_role
And be very careful when making IAM assume role policy to be extremely specific in the roles you give it access to ( that guide shows a very specific role selection so it's pretty good ) or you are exposing major security risk.

fixed in release 0.0.7, thanks


For DIY server kickoff need to default to add "http://" or change expected input format

Right now if I dont enter http://node:port it complains. But when kicking off my DIY server the string format is only node:port. not HTTP://node:port. It is a minor issue but would be nice for it to match the expected string that the DIY server gives.

Thanks Sita.
(from slack convo) It's hard to check that user input for a URL is valid if it's just the host and port. Also, perhaps it will be https at some point, or someone from outside of Hutch runs the app and uses https . So I figured we should just ask for the entire url.
I'll do a pull request to the diy repo to change the output if amy's okay with that
rcromwell also expects the http part

changes ready to go in diy repo, will open a PR and when that's merged will close this

pr here https://github.com/FredHutch/diy-cromwell-server/pull/28


Align downstream 10-7

Aligning with upstream, bringing l2 status and frr-k8s on a separate ns

This is gonna fail as we don't have the crds in the operator yet

/hold

/retest

/hold cancel

/lgtm

/retest


Fix settings load

The supportedAccountDomains array is serialized with a "url" property but is expected to have a "accountDomain" property on load. I chose to make load match save, so that existing files behave as was presumably intended.

/azp run


[QA] Fix flaky data_manager unit tests

Description
See issue for more.
Changes introduced in this PR:

deterministically solve the problem of unstable data_manager tests
speed-up data_manager unit tests a lot

Screenshot
Notice

[x] I acknowledge and agree that, by checking this box and clicking "Submit Pull Request":

I submit this contribution under the Apache 2.0 license and represent that I am entitled to do so on behalf of myself, my employer, or relevant third parties, as applicable.
I certify that (a) this contribution is my original creation and / or (b) to the extent it is not my original creation, I am authorized to submit this contribution on behalf of the original creator(s) or their licensees.
I certify that the use of this contribution as authorized by the Apache 2.0 license does not violate the intellectual property rights of anyone else.
I have not referenced individuals, products or companies in any commits, directly or indirectly.
I have not added data or restricted code in any commits, directly or indirectly.

@antonymilne thanks for taking the PR to a new level. I really like how the tests cases are now much more realistic (with timeouts of 300s and 100s). I agree with all your bullets from the comment above, but unfortunately I can't approve the PR I created 😄.


Removing camelize method.

There's not much justification for the camelize method, which is only used in a handful of places and can be replaced by knowledge. This patch removes the method, the tests for the method, and corrects usage of CSS-in-javascript elsewhere.

@tmcw we have decided on the mailing list to move the deprecated code to deprecated.js. This happens in the main development branch (master), so in the context of 2.x. With this we won't break apps that rely on deprecated code. As long as deprecated.js is loaded things should work as before.

All right, moved over to deprecated.

I just realized that this function is an API function, so there may be applications relying on it. I think it's fine to have a release note mentioning that this function is deprecated and has been moved to deprecated.js. Can we just make it output a deprecating warning message? After that I think we can merge. Thanks @tmcw.

@ahocevar please merge this if you agree with the changes made by @tmcw. I think they address your comments. Once merged I'll go ahead and update the 2.12 notes. Thanks.

Is anybody working on this? If we don't get a fix before the feature freeze later today, I'm going to revert the merge.

I'd like to take a look tonight.

For the reason mentioned in my last comment I think we can revert acf6a8a, 5875b50, and 43ef092.


[MODULES-2281] Add acceptance tests for rabbitmq_parameter

This is an addition to PR 390

@hunner Added an acceptance test, sorry I did not get this in with the origin PR.
Is it possible to release a new version of rabbitmq module to the forge? Last update was in Jan.

Passed on trusty. :+1:
http://ci.puppet.community/buildlogs/puppetlabs+puppetlabs-rabbitmq+395+1443558577+PASS
I'd like to get pcci green-ball voting on this repo if other maintainers agree.

@nibalizer I agree.


Remove pyclingo

Skimming through the PR, it seems this is dropping the support for Python 2 and making CFFI mandatory for Python extensions - am I correct on that?

Yes, cffi and at least Python 3.6 are required now. In principle cffi supports Python 2 but I really wanted to use Python's type annotations without any crude workarounds.


Docker images for SAS

This PR updates lock file to be used for generating Docker image to be delivered in cal/val release.

It looks like the tests using lock file has passed, but on-the-fly dependency solution using environment.yaml needs to be revised.
That means the RTC-S1 SAS is good to release, but we will need to update environment.yaml as time permits.


Add message to frontend with found nodes

Add simple list
Add loading spinner

[ ] Clean up UIStore (with get & set state interface)
[ ] Move results list to separate component


Unknown error <class 'twscrape.queue_client.ApiError'>: ApiError on 49/50 - *** (code 200): Dependency: Internal error. (131) ~ Dependency: Internal error.

At the beginning, I didn't encounter this problem until I scraped a certain amount of data.
I have tried the following solutions:

Use a new account that has not been used for crawling
Using a new IP node
Using a new runtime environment (conda env)
Using a new hardware device (from my personal computer to my work computer)
It is not possible to completely solve the problem.
After crawling 1-2 accounts, the above error will still occur.

https://github.com/vladkens/twscrape/issues/82#issue-1940124529
It seems that others have also encountered this problem.

a new error

I've also encountered this problem, twitter API guideline says it's a problem on their side but my guess it's a shadow ban

This is unknown error from Twitter. Should be fixed by PR https://github.com/vladkens/twscrape/pull/80 from v0.9


Use log in vach-cli

This replaces (most) println!'s with log calls on appropriate levels and initializes pretty_env_logger in the cli.
solves #47

Does this logger interfere with the progress bar?

I have a 4 hour long electronics class, will review when back

Does this logger interfere with the progress bar?

No, it still println!'s there


[BUG] Missing file viz/report.tpl

Comes from https://github.com/poldracklab/niworkflows/issues/99
The file is in the repo, but it is missing in the package_data of setup.py

Appears to be fixed with commit e6c7408c70fa27e5f6e7c37873d722d9263c3f50


Refactor how session/background services are gated

Goal
Refactors how the session/background activity services are gated by creating a ConfigGate class. This is responsible for checking whether a feature is enabled & holding state so that either a service is returned, or null is returned if not. The aim is to free up the session & background activity service from having to hold state about whether it's enabled or not & gate every single function call.
This is a little problematic in that the service can be disabled at any time in the process lifecycle, so it'd be great to get a detailed review to see whether I've missed anywhere this might cause us state management pain. Having said that, the existing services don't do a great job of this either.
Testing
Added to existing test coverage.

EmbraceImpl should never have a null reference to the service after the changes to SessionModuleImpl. Tagging for re-review as I made a few changes to the unit tests + EmbraceImpl to reflect this.


v2: return ETag with ACL and raw ACL responses

This allows clients to know which ETag to provide when updating an ACL.
Updates #119

(both are just FYIs, not blockers)


No brackets autocompletion with python.analysis.completeFunctionParens set to true when python.languageServer is set to Pylance

Environment data

VS Code
Version: 1.74.3 (system setup)
Commit: 97dec172d3256f8ca4bfb2143f3f76b503ca0534
Date: 2023-01-09T16:59:02.252Z
Electron: 19.1.8
Chromium: 102.0.5005.167
Node.js: 16.14.2
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.22621
Sandboxed: No
Language Server version: 2023.1.20
OS and version: Windows 11 Pro 22H2
Python version (& distribution if applicable, e.g. Anaconda): 3.11.1

I believe it has happened after VS Code or extension update, previously it worked fine.
Code Snippet

Expected behavior
Brackets are added when python.languageServer is set to "Pylance".
Actual behavior
Brackets are added only when python.languageServer is set to "Jedi".

Thanks for the issue.
I can definitely repro. Not sure what's happening yet.

It was a mistake introduced in 2023.1.20. Should be easy to fix. Hope to have it in 2023.1.31 (next week)

This issue has been fixed in prerelease version 2023.1.31, which we've just released. You can find the changelog here: CHANGELOG.md


fix: date format for posts list, chore:  create Japanese template

Type
Message

Bug Fix
fix: output date by "Site.params.dateFormat" except for archive pages.

Chore
chore(i18n): created Japanese template.

Thanks for your contributions. 👍
BTW, I reword your commit message, since the following should be the git commit messages.

Type	Message
Bug Fix	fix: output date by "Site.params.dateFormat" except for archive pages.
Chore	chore(i18n): created Japanese template.


Can't use more than one autocomplete on a page

When used more than once on a page, only the first autocomplete field behaves the way it was intended. Any autocomplete after the first doesn't display the drop down.

Thanks for raising this.
As far as I know, the accessible autocomplete should support this. The example page for example has a number of autocompletes working on the same page.
Are you able to share an example of the code you're using at the minute? That might help us understand what's going on.

Realised it was an issue on our end but thank you for getting back so quickly.


Add Possibility to update the firmware when connected to an AP

What happened?
After moving the http update to the config AP it is no longer possible to update the FW while connected to the home AP.

All

Which Inverter are you using?

All

fixed by #289


Alignment in Table 3 of columns is off

Numbers in columns of Table 3 (total catch per sector) should be right aligned.

@cgrandin i am guessing that center alignment was used because of the dash for missing numbers. We have three options

Get rid of the dash and change the caption to empty cells represent missing values
Leave the dash right aligned as I did here

leave the columns center aligned so the dash is in the center but the numbers are not right aligned which imo makes them harder to compare.

That looks fine to me and an improvement. No-one's going to complain about the dashes not being in the centre. Hopefully...

I agree with what you're showing. I replicated it by changing this:
https://github.com/pacific-hake/hake-assessment/blob/7fd0ebaeff4e04d885a1dfdb485981bbb1fb31fa/R/catches.R#L255
to this:
align = c("l", rep("r", ncol(tab)))),
and had a quick look round to see if we could center the dashes but nothing easy as you already found out. So I will leave it to you


fix(cloudflare-kv-http): hasItem and getItem

The current cloudflare-kv-http driver implementation of getItem does not seem to follow the Cloudflare API response documented here, which returns the value directly.

👍

Thanks for fixes @qin-guan and sorry took long to review. I've pushed few improvements to use metadata (it seems initial implementation was indented to use it for success)

No worries @pi0, thanks for reviewing and making the changes! 🎉


lis2dw12: Add support for set/clear int cfg bits

Set/Clear individual interrupt cfg bits
Change config in sensor creator
Change int_active to int_active_low to be more explicit
Change lis2dw12_set_int_active() to lis2dw12_set_int_active_low()

I know I already mentioned this, but I get a little nervous about tracking the count of enabled interrupts without knowing which ones are actually enabled.  I don't know this sensor code very well, so I may well be worrying over something that doesn't actually matter.  I would just be sure that it isn't possible to disable the same interrupt twice, and/or disable interrupts too many times and cause the interrupt count to roll over.  There, I said my piece :).

@ccollins476ad changed all the comments to match doxygen format. Please check it out.


name of the preference extensions.yslow.beaconInfo not the same as used in documentation

In the YSlow documentation( http://developer.yahoo.com/yslow/help/beacons.html#version2 ) the preference extensions.yslow.beaconInfo is mentioned. This is inconsistent with the preference name used in the code.
In the code /src/common/view.js line 413
info = pref.getPref('info', 'basic'),
This uses the preference extensions.yslow.info.
I'm not sure if the documentation or the code needs changing.

fixed, the correct was beaconInfo, the issue is #5 not #7


timeframe=bt.TimeFrame.Minutes and sleep 3 seconds

New environment, pip freeze:
alpaca-backtrader-api==0.10.1
alpaca-trade-api==0.50.1
backtrader==<IP_ADDRESS>
certifi==2020.6.20
chardet==3.0.4
cycler==0.10.0
idna==2.10
kiwisolver==1.2.0
matplotlib==3.3.2
numpy==1.19.2
pandas==1.1.3
Pillow==7.2.0
pyparsing==2.4.7
python-dateutil==2.8.1
pytz==2020.1
requests==2.24.0
six==1.15.0
toolz==0.11.1
trading-calendars==1.11.11
urllib3==1.25.10
websocket-client==0.57.0
websockets==8.1

Trying to use this sample code https://github.com/alpacahq/alpaca-backtrader-api/blob/master/sample/strategy_sma_crossover.py, but with one difference, instead of bt.TimeFrame.Days I am trying  bt.TimeFrame.Minutes:
data0 = DataFactory(dataname=symbol,
                            historical=False,
                            timeframe=bt.TimeFrame.Minutes,
                            backfill_start=True,
                            )

And looks like forever logging:
Starting Portfolio Value: 99920.71
***** DATA NOTIF: DELAYED
2020-10-08 10:55:41,651 connected to: wss://paper-api.alpaca.markets/stream
2020-10-08 10:55:42,294 connected to: wss://data.alpaca.markets/stream
2020-10-08 10:56:39,196 sleep 3 seconds and retrying https://paper-api.alpaca.markets/v2/account 3 more time(s)...
2020-10-08 10:56:47,027 sleep 3 seconds and retrying https://paper-api.alpaca.markets/v2/account 3 more time(s)...
2020-10-08 10:56:55,404 sleep 3 seconds and retrying https://paper-api.alpaca.markets/v2/account 3 more time(s)...
2020-10-08 10:57:03,546 sleep 3 seconds and retrying https://paper-api.alpaca.markets/v2/account 3 more time(s)...
2020-10-08 10:57:13,243 sleep 3 seconds and retrying https://paper-api.alpaca.markets/v2/account 3 more time(s)...
2020-10-08 10:57:22,087 sleep 3 seconds and retrying https://paper-api.alpaca.markets/v2/account 3 more time(s)...
2020-10-08 10:57:29,888 sleep 3 seconds and retrying https://paper-api.alpaca.markets/v2/account 3 more time(s)...
2020-10-08 10:57:39,282 sleep 3 seconds and retrying https://paper-api.alpaca.markets/v2/account 3 more time(s)...
2020-10-08 10:57:47,317 sleep 3 seconds and retrying https://paper-api.alpaca.markets/v2/account 3 more time(s)...
2020-10-08 10:57:55,757 sleep 3 seconds and retrying https://paper-api.alpaca.markets/v2/account 3 more time(s)...
2020-10-08 10:58:04,196 sleep 3 seconds and retrying https://paper-api.alpaca.markets/v2/account 3 more time(s)...
2020-10-08 10:58:04,489 sleep 3 seconds and retrying https://paper-api.alpaca.markets/v2/account 3 more time(s)...
2020-10-08 10:58:14,343 sleep 3 seconds and retrying https://paper-api.alpaca.markets/v2/account 3 more time(s)...
2020-10-08 10:58:22,433 sleep 3 seconds and retrying https://paper-api.alpaca.markets/v2/account 3 more time(s)...
2020-10-08 10:58:31,479 sleep 3 seconds and retrying https://paper-api.alpaca.markets/v2/account 3 more time(s)...
2020-10-08 10:58:39,858 sleep 3 seconds and retrying https://paper-api.alpaca.markets/v2/account 3 more time(s)...

What I am doing wrong?

I am sorry, just not read all code of the example. Looks like all fine.
Anyway, would be cool if creators add some samples with real-time intraday timeframes. It would be useful for new users.

Only one question, how to use in live mode resample()?

Also, how to control the backfill_start range? Looks like too much data loading for the first run.

https://github.com/alpacahq/alpaca-backtrader-api/issues/41
In this topic above, I understood that the latest version of the library something changes (maybe I am wrong) in real-time mode.
Let me explain what I mean. In the previous versions, I was using the polygon real-time data feed with cerebro.resampledata() function and first resampled data feed of this function worked as a range for next() function (time-driven, not event), right? And it's worked for me.
Now, I am trying to use paper live and I am a little confused. I understood in the above's topic that next() in paper live mode is event-driven. When I am using cerebro.resampledata() I get an error like this:
Error while connecting to wss://data.alpaca.markets/stream:your connection is rejected while another connection is open under the same account
and as I understand it's normal.
So, looks like, I need to change thinking about how it works in live mode with Alpaca API (not Polygon).
But I can't understand how to control next() with the time range of my resample() list of datas? I hope my question for knowing people is understandable what I mean.

Hi,
you have here a mix of a few things:

your connection is rejected while another connection is open under the same account - this is caused by running more than 1 algorithms at once. you can only execute one strategy per account. you could use this project to overcome this: https://github.com/shlomikushchi/alpaca-proxy-agent
as you read in: https://github.com/alpacahq/alpaca-backtrader-api/issues/41 the next() method is event driven. you could check the time inside next() and decide if you want to do something at this time. if not, just return and wait the next call
backfill takes time if you request a lot of data. you can control it with fromdate in DataFactory

your connection is rejected while another connection is open under the same account - this is caused by running more than 1 algorithms at once. you can only execute one strategy per account. you could use this project to overcome this: https://github.com/shlomikushchi/alpaca-proxy-agent

When I try to use resample data I see a rejection error too. It's ok? With live mode no sense to use resampling already before?

as you read in: #41 the next() method is event driven. you could check the time inside next() and decide if you want to do something at this time. if not, just return and wait the next call

Maybe you have some examples of how to divide event by timeframe?

I have the same issue when I am doing paper trading. When backfill is running, in next method we have the following check
 def next(self):
        if not self.live_bars and not IS_BACKTEST:
            # only run code if we have live bars (today's bars).
            # ignore if we are backtesting
            return

However, any backfill data which comes to next() method triggers /account API. It doesn't matter event if we return from next() method without performing any operations. Why do we need to check an account API if data still backfilling?

I have the same issue when I am doing paper trading. When backfill is running, in next method we have the following check
 def next(self):
        if not self.live_bars and not IS_BACKTEST:
            # only run code if we have live bars (today's bars).
            # ignore if we are backtesting
            return

However, any backfill data which comes to next() method triggers /account API. It doesn't matter event if we return from next() method without performing any operations. Why do we need to check an account API if data still backfilling?

Yes, it's hard to understand without good examples. All samples are very poor to use in reality. Just my opinion,

Today, the same test code doesn't work at all. No calling next() at all. No events - no logs. Enabled/disabled backfill_start param - no matter. Active symbol $SPY. Magic.

@observer-user

However, any backfill data which comes to next() method triggers /account API. It doesn't matter event if we return from next() method without performing any operations. Why do we need to check an account API if data still backfilling?

we need to feed the data into the backtrader platform because we added this integration on top of backtrader.
so every data that is fed into backtrader goes through the backtrader pipeline, no matter if it's a backfill or a live stream

@x777

Maybe you have some examples of how to divide event by timeframe?

yes you could do something like this:
    def next():
        import datetime
        if datetime.datetime.utcnow() - self.timer < timedelta(minutes=5):
            return
        self.timer = datetime.datetime.utcnow()
        <YOUR-CODE-HERE>

and initialize self.timer in the init function

Also, I think the alpaca data stream is a bit thin this morning.
I also don't get as many messages as usual.
when switching to the polygon data stream, everything works fine

Also, I think the alpaca data stream is a bit thin this morning.
I also don't get as many messages as usual.
when switching to the polygon data stream, everything works fine

Thank you for your code, I will try it.
But now, an interesting thing happens with the data feed. How this moment to debug or only wait? Is any status of servers where to check?

try using a different stock, maybe its data feed will be better.
otherwise, you need to wait

Ok, hope it will work later.
Another question about multi-data. I must use alpaca-proxy-agent if I add more than 1 data feed to my algo? For example, I need for indicators data feed with a different timeframe.

if you want more than one data you need the proxy project
more than one indicator, but only one data - you don't need it

if you want more than one data you need the proxy project
more than one indicator, but only one data - you don't need it

The compression parameter in DataFactory uses as a timeframe for the indicator or no?

The compression is used to get the backfill data from the servers, so it used for the indicators for daily bars (1D, 5D, 10D etc..)
once the live data starts, it's event driven with quote data.

So, if I want add specific timeframe for indicator, what best way - use
additional data feed with proxy or calculate it manually by each event?
сб, 10 окт. 2020 г., 12:26 Shlomi Kushchi<EMAIL_ADDRESS>
The compression is used to get the backfill data from the servers, so it
used for the indicators for daily bars (1D, 5D, 10D etc..)
once the live data starts, it's event driven with quote data.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/alpacahq/alpaca-backtrader-api/issues/122#issuecomment-706519041,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABBXRR7CBQ2ASVWVXSLIXNTSKASD7ANCNFSM4SI4NVRQ
.

you will still receive events to the indicators, but they will be the same events you get for the next() method.
if you want to calculate a different timeframe, calculate manually, yes

Maybe you can show where I can find useful examples for live mode, because
I see that too many questions.
Also, how do you think, it's possible to use with Alpaca backtrader library
with native Alpaca API library and make execution? Because resampling data
was useful thing and it worked for me for live mode, but execution part was
not for Alpaca.
сб, 10 окт. 2020 г., 15:01 Shlomi Kushchi<EMAIL_ADDRESS>
you will still receive events to the indicators, but they will be the same
events you get for the next() method.
if you want to calculate a different timeframe, calculate manually, yes
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/alpacahq/alpaca-backtrader-api/issues/122#issuecomment-706538243,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABBXRR22CP3ILZRS7ITPTSLSKBEIPANCNFSM4SI4NVRQ
.

@shlomikushchi I won't (can't) use proxy agent, but need a different timeframe for each indicator.
Your code with timedelta is good, but now how I can calculate for each indicator different timeframe? Still struggle with understanding how to do such simple things without resampledata().

I am going to work on the resample function in the future.
in the meantime you could use talib (or other TA packages, there are a few) to calculate the indicators on samples.
for samples you could do one of the two options

store the samples you get from the next function
use the alpaca api to request data with any timeframe that you need.

I am going to work on the resample function in the future.
in the meantime you could use talib (or other TA packages, there are a few) to calculate the indicators on samples.
for samples you could do one of the two options

store the samples you get from the next function
use the alpaca api to request data with any timeframe that you need.

Maybe I can use some previous versions. What previous version is good for real-time but with resample function?

you could try versions before introducing the alpaca data api (0.7.1 and below) but I don't believe it will change anything since the data stream was always quote based

you could try versions before introducing the alpaca data api (0.7.1 and below) but I don't believe it will change anything since the data stream was always quote based

I understand, but resampling data isn't working with even data in previous versions? I just trying to develop bar based strategy and I need to play with different timeframes.

try working with the polygon data stream, it's more stable. let's see if you get a better result.

try working with the polygon data stream, it's more stable. let's see if you get a better result.

Latest version you mean or that I worked with Polygon already?

resampling is a backtrader mechanism. it receives ticks/aggs and resamples them. but messages are not coming as clock events.
one minute you may receive many quotes, another minute you may not receive any.
it depends on many things (were there any traders, did the communication worked properly, etc) there's no way to know that every time, which means it's hard to predict the behavior.
one thing you could do, is change the qcheck parameter as described here: https://www.backtrader.com/docu/live/ib/ib/#live-feeds-and-resamplingreplaying
and/or try to work with the different data streams available

as I said, I don't think you will get a better result with an older version since this is a backtrader code.
but maybe I missed something, so you could try it too

as I said, I don't think you will get a better result with an older version since this is a backtrader code.
but maybe I missed something, so you could try it too

I just try to avoid using proxy agent. Because resampling data for live feed is working fine in previous versions.
But I confused now, maybe it's not working already)
Looks like advantages of using this library for live feed not too much and maybe make sense to work with native alpaca trade api. Just thinking for perspective strategies developing.

I just tested the latest code with polygon data. doing a resample like so, and it works:

data0 = DataFactory(dataname=symbol,
                    historical=False,
                    timeframe=bt.TimeFrame.Minutes,
                    qcheck=2.0,
                    backfill_start=True,
                    fromdate=datetime.utcnow().date() - timedelta(minutes=200)
                   )

cerebro.resampledata(data0,
              compression=1,
              timeframe=bt.TimeFrame.Minutes)

changing the quote data to minute data

@shlomikushchi can you please explain why this happens that only with polygon data it's working? What difference between Alpaca's feed and Polygon?

it works with alpaca data too, it just might not be as consistent.
could be different things:

amount of messages
delays
different exchanges collected from

the default quote data is the bidprice but you could specify to get the askprice

here's an example code using the resample functionality:
https://github.com/alpacahq/alpaca-backtrader-api/blob/master/sample/strategy_resampling_example.py

@shlomikushchi But earlier you said that using resample functionality requires proxy agent or not?
About quotes. How I can switch between bidprice and askprice? Can I switch to last trade?

proxy-agent is completely different thing. nothing to do with resampling.
you could pass a param useask (default is False.)
you can't use last trade for now. only quotes.

proxy-agent is completely different thing. nothing to do with resampling.
When I used resampling in earlier messages I get error: Error while connecting to wss://data.alpaca.markets/stream:your connection is rejected while another connection is open under the same account
So, I understood that I must to use proxy-agent. What will be if I add more resample data functions that in your example?

if you use more than one data you need the proxy-agent. it doesn't matter if you resample or not

if you use more than one data you need the proxy-agent. it doesn't matter if you resample or not

This is from all my problems began.

I just tested the latest code with polygon data. doing a resample like so, and it works:
data0 = DataFactory(dataname=symbol,
                    historical=False,
                    timeframe=bt.TimeFrame.Minutes,
                    qcheck=2.0,
                    backfill_start=True,
                    fromdate=datetime.utcnow().date() - timedelta(minutes=200)
                   )

cerebro.resampledata(data0,
              compression=1,
              timeframe=bt.TimeFrame.Minutes)

changing the quote data to minute data

I was trying to use your example with proxy-agent:
***** DATA NOTIF: DELAYED
2020-10-16 15:02:29,362 error while consuming ws messages: [Errno 110] Connect call failed ('<IP_ADDRESS>', 8765)
2020-10-16 15:04:40,444 error while consuming ws messages: [Errno 110] Connect call failed ('<IP_ADDRESS>', 8765)
2020-10-16 15:06:51,516 error while consuming ws messages: [Errno 110] Connect call failed ('<IP_ADDRESS>', 8765)
2020-10-16 15:09:02,578 error while consuming ws messages: [Errno 110] Connect call failed ('<IP_ADDRESS>', 8765)

Strategy without resampling works well through proxy-agent, but second, that use resampling shows such errors.

@shlomikushchi It's possible to convert live feed manually without resampedata() to data feed type that can be used later in indicators like after resampling? Now I am getting needed data from Alpaca Data API by using a timer and calculating simple indicators manually, but it's not comfortable. Maybe you have some examples?

are you running the docker container on '<IP_ADDRESS>'?
it doesn't seem like something is listening there

@shlomikushchi It's possible to convert live feed manually without resampedata() to data feed type that can be used later in indicators like after resampling? Now I am getting needed data from Alpaca Data API by using a timer and calculating simple indicators manually, but it's not comfortable. Maybe you have some examples?

this is are the 2 options you have:

resample
get the data manually using the api

are you running the docker container on '<IP_ADDRESS>'?
it doesn't seem like something is listening there

Yes, I don't see any logs in docker terminal when starting strategy.

@shlomikushchi It's possible to convert live feed manually without resampedata() to data feed type that can be used later in indicators like after resampling? Now I am getting needed data from Alpaca Data API by using a timer and calculating simple indicators manually, but it's not comfortable. Maybe you have some examples?

these are the 2 options you have:

resample
get the data manually using the api

Problem with second way is how to convert response to format (type) that supports backtrader indicators or talib library. Can you show some examples?
I only can use with new library btalib by backtrader developers, but there not too much supported indicators yet.

are you running the docker container on '<IP_ADDRESS>'?
it doesn't seem like something is listening there

Yes, I don't see any logs in docker terminal when starting strategy.

then you are probably not running on <IP_ADDRESS>.
are you using virtualbox? if not you are probably running on <IP_ADDRESS>:8765

Problem with second way is how to convert response to format (type) that supports backtrader indicators or talib library. Can you show some examples?
I only can use with new library btalib by backtrader developers, but there not too much supported indicators yet.

btalib is fine, it's the same indicators you have in backtrader
but why not use the resample?

are you running the docker container on '<IP_ADDRESS>'?
it doesn't seem like something is listening there

Yes, I don't see any logs in docker terminal when starting strategy.

then you are probably not running on <IP_ADDRESS>.
are you using virtualbox? if not you are probably running on <IP_ADDRESS>:8765

Changing the env variable with <IP_ADDRESS>:8765 address solved problem.

Problem with second way is how to convert response to format (type) that supports backtrader indicators or talib library. Can you show some examples?
I only can use with new library btalib by backtrader developers, but there not too much supported indicators yet.

btalib is fine, it's the same indicators you have in backtrader
but why not use the resample?

If the timeframe is not one and hard to use proxy-agent it would be helpful to understand how to convert response API to a data feed that is supported by indicators.

are you running the docker container on '<IP_ADDRESS>'?
it doesn't seem like something is listening there

Yes, I don't see any logs in docker terminal when starting strategy.

then you are probably not running on <IP_ADDRESS>.
are you using virtualbox? if not you are probably running on <IP_ADDRESS>:8765

Changing the env variable with <IP_ADDRESS>:8765 address solved problem.

But when I starting the same script through proxy-agent it only connected but no events after that. I see that it connected to docker only.
And looks like because I have problems with resampling.

In my opinion, the main problem to understand how it works is to be sure that the stream from the server is stable.
When you trying to play with resampling or with proxy agent you never know what you are doing wrong because always your script that worked a few minutes ago, now not receiving any data by the stream now. It's a very strange behavior. I am always testing only high liquid stocks and etf.

In resample example
line 50 not working for me:
fromdate=datetime.utcnow() - timedelta(minutes=20)
Only this code starting resample data:
fromdate=datetime.now() - timedelta(minutes=5)

In my opinion, the main problem to understand how it works is to be sure that the stream from the server is stable.
When you trying to play with resampling or with proxy agent you never know what you are doing wrong because always your script that worked a few minutes ago, now not receiving any data by the stream now. It's a very strange behavior. I am always testing only high liquid stocks and etf.

start with one stock. test it with resample (without proxy-agent). when you're sure it works start complicating it

Problem with second way is how to convert response to format (type) that supports backtrader indicators or talib library. Can you show some examples?
I only can use with new library btalib by backtrader developers, but there not too much supported indicators yet.

btalib is fine, it's the same indicators you have in backtrader
but why not use the resample?

If the timeframe is not one and hard to use proxy-agent it would be helpful to understand how to convert response API to a data feed that is supported by indicators.

if you want to retrieve data every minute from the api servers then the pylivetrader is a project that is designed to work like this. to do it in backtrader is possible, but it's not by design

In my opinion, the main problem to understand how it works is to be sure that the stream from the server is stable.
When you trying to play with resampling or with proxy agent you never know what you are doing wrong because always your script that worked a few minutes ago, now not receiving any data by the stream now. It's a very strange behavior. I am always testing only high liquid stocks and etf.

start with one stock. test it with resample (without proxy-agent). when you're sure it works start complicating it

It's funny, I just want to run on live feed a few basics indicators and really can't to do it)
Sometimes it takes a lot of time in ***** DATA NOTIF: DELAYED, then when it switched to ***** DATA NOTIF: LIVE nothing happens much more than in delay mode.
Only one question: such kind of problems have all users of alpaca-backtrader or only mine?

Problem with second way is how to convert response to format (type) that supports backtrader indicators or talib library. Can you show some examples?
I only can use with new library btalib by backtrader developers, but there not too much supported indicators yet.

btalib is fine, it's the same indicators you have in backtrader
but why not use the resample?

If the timeframe is not one and hard to use proxy-agent it would be helpful to understand how to convert response API to a data feed that is supported by indicators.

if you want to retrieve data every minute from the api servers then the pylivetrader is a project that is designed to work like this. to do it in backtrader is possible, but it's not by design

I don't want to do such converting if resampling works well. But...

use polygon data, it works better with resampling

use polygon data, it works better with resampling

Any perspective that it will work better with Alpaca feed in the future?

I believe it will be more and more like polygon yes

I believe it will be more and more like polygon yes

Hope too. Thx!

@shlomikushchi any updates about resamping with Alpaca?

no


Make varlink API more idiomatic

See individual commits. This is a first round of fixes I've found when I started working with this API for cockpit-podman. I have some more, but these were the ones I could easily implement.
I hope that you don't consider the varlink API to be stable, yet? As far as I know, cockpit-podman is the only user so far and I'm fine updating that on the next release.

Hm, I don't understand the latest error on the "gating" check. It says "9 commits to fix", but not what the fixes should be...

Gitvalidation is complaining that you need a signoff on your commits
@baude Will the time format changes here mean we need changes in adapter to read times back in as RFC3339?

/ok-to-test
bot, add author to whitelist

@larskarlitski cockpit is not the only user of the API ... the remote client for darwin I am writing is the a major one.  Unfortunately we don't quite have integration tests running for those yet as we are still working on a couple of container related ones.  Until that is resolved, the tests cannot be run.  So you will have to make sure you don't break that either.

please dont remove endpoints that are not implemented.  it is the only list we have for what needs to be done still.

@larskarlitski great start ... i like the changes for images and containers naming. if you can address my comments, that would be great.

@larskarlitski Have you had a chance to address @baude Issues?

Gitvalidation is complaining that you need a signoff on your commits

Thanks. Added those and addressed @baude's comments. Thanks for the reviews!

please dont remove endpoints that are not implemented. it is the only list we have for what needs to be done still.

We can't commit to a stable API if some of the functions return a NotImplemented type. Better to leave out the things that aren't there yet. to track missing functions, would you be ok with me adding a list in a TODO file or similar?

Gitvalidation is complaining that you need a signoff on your commits

Thanks. Added those and addressed @baude's comments. Thanks for the reviews!

please dont remove endpoints that are not implemented. it is the only list we have for what needs to be done still.

We can't commit to a stable API if some of the functions return a NotImplemented type. Better to leave out the things that aren't there yet. to track missing functions, would you be ok with me adding a list in a TODO file or similar?

there are several i dont mind pruning.  Ill remove those in a different PR.  Can we compromise and instead of removing, comment them out?

there are several i dont mind pruning. Ill remove those in a different PR. Can we compromise and instead of removing, comment them out?

Good idea! Done.
Are these test failures related to my changes? I can't see how they would be...

Restarted one of the cirrus tests to see if this is just flakes, or we have a real issue
@baude Mind taking another look at this?

kind of baffled by this one; the tests look like their legitimately failing?

kind of baffled by this one; the tests look like their legitimately failing?

I don't understand all the tests and it's hard to get them working locally. I've found one place where the remoteClient panics in splitStringDate, because it still assumed the old time format. I could remove this function, because we're now just passing along standard time stamps.

@baude @jwhonce Tests are passing PTAL

please rebase and I will merge it @larskarlitski

Rebased.

Rebased, again.

@larskarlitski I will watch your PR .. .btw, join us on #podman on freenode!

/approved

/lgtm


Accessibility improvements

To improve site accessibility, we need to do an audit of the current state of the application, then fix it as much as possible to enable better accessibility.
Tab ordering is the easiest to tackle and should be done first.

https://developer.mozilla.org/en-US/docs/Web/HTML/Global_attributes/tabindex


fresh install of single-spa-react is missing lib and parcel directories

Hi, I'm working on a project that uses single-spa and single-spa-react.  I had to remove my node_modules folder and do a fresh install.  After doing so, my project started throwing errors on start, specifically complaining about this:
ERROR in ./src/redacted.js 3:0-46
Module not found: Error: Can't resolve 'single-spa-react' in 'C:\projects\redacted\src'
resolve 'single-spa-react' in 'C:\projects\redacted\src'
  Parsed request is a module
  using description file: C:\projects\redacted\package.json (relative path: ./src)
    Field 'browser'

Here's how I'm attempting to access single-spa-react in the file it's complaining about:
...
import singleSpaReact from 'single-spa-react';
...
const lifecycles = singleSpaReact({
    React,
    ReactDOM,
    rootComponent: Root,
    errorBoundary(err, info, props) {
        // Customize the root error boundary for your microfrontend here.
        return null;
    },
});
...

I tried several fresh installs, cleaning my cache, etc., and couldn't get rid of this error.  I have several other projects that use single-spa-react, so I decided to investigate their node_modules/single-spa-react directories to see if there were any discrepancies.  Here's what I found.  In an older project, node_modules/single-spa-react looks like this:
node_modules
--single-spa-react
----lib
----parcel
----types
--LICENSE
--package.json
--README.md

However, the repo with the fresh install looks like this:
node_modules
--single-spa-react
----types
--LICENSE
--package.json
--README.md

The lib and parcel directories are missing.  To make sure this is what was causing the issue, I replaced node_modules/single-spa-react with the copy from the older repo, and that did fix the problem.
In case there was an issue with the machine I'm working on, I recreated this issue on a different computer in a completely fresh npm project.  The package.json looks like this:
{
  "name": "single-spa-react-test",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "single-spa-react": "^5.1.0"
  }
}

This results in the same thing - single-spa-react-test/node_modules/single-spa-react is missing the lib and parcel directories.  Can I get any guidance on what I can/should do to fix this?  Thanks!

You are not the only one who has faced this problem. There is a solution to your question in thisthread.

Duplicate of #173


Closes #53: Fix CSS inconsistencies (as of 22 Aug)

Closes #53. White edges in the background of the task popup scene are now removed when both adding or editing tasks. Current state of task scene is in the screenshot below.

I like the changes Maybe we should make the scene which is not in focus grey out a little bit so that it is obvious to the user which scene they can interact with?

Yes. I think this would be a good issue to open. Would help with overall maintainability.


Nested  stops working after re-render

Using react-router 1.0.0-rc3, I have the following routes:
<Route component={Application}>
   <Route path="/" component={Home} />
   <Route path="gallery/:navigationId" component={Gallery}>
      <Route path="detail/:articleId" component={Detail} />
   </Route>
</Route>

I have a  in a thumbnail component in Gallery like this:
<Link to={`/gallery/${this.props.params.navigationId}/detail/${this.props.article.id}`}>
The link works, but once it goes to the detail page (with the Thumbs still below) the links stop working (nothing happens on click)
If I refresh or go directly to the Detail page it works again, but only once. After a re-render they are broken.
Any ideas? Ideally I would also like to use a relative link instead of passing params.navigationId, but as I read that isn't possible yet.

For the record, if I replace  with just an onClick handler:
thumbClick() {
   this.props.history.pushState(null,`/gallery/${this.props.params.navigationId}/detail/${this.props.article.id}`)
}

Then it's working. But I want to use  so I can open the links in a new tab (for usability)

For questions and support, please visit our channel on Reactiflux or Stack Overflow. The issue tracker is exclusively for bug reports and feature requests.

Well, as far as I know this is a bug, since it works once only. Do you need some code to reproduce this?

@jonaswindey yeah that would be great, either failing test case or give me the link to repo so I can check it out. If it's a bug I will reopen.

Ok, after further investigation it seems it's a conflict with https://github.com/FormidableLabs/react-shuffle
If you have a repeat of items and there's a  in it, the props are overwritten sometimes.
This can stay closed. Thanks


fixed get rook moves

Nice try
You did not handle the case of the rook jumping over other pieces.
I'll send you Mr. Satatima's solution to bishop moves, since it's kind of similar to the rook's move

Alright, I will await to receive that.

Add a doc-string to the get_rook_moves method

Alright, will work on that

I did not see the changes. In fact your get_rook_moves method is now blank.

 In fact your get_rook_moves method is now blank.

I actually don"t know what happened but can you see it now

I think you merged my light_team branch into yours

There are issues with your code still

You save the file without resolving merge conflicts

You merged my branch into yours again
When you make changes you don't have to do anything on the pull request for me to send. Just commit the changes, and push them to your online repo. Github does the rest of the job for you


Update Composer dependencies (2021-12-15-00-03)

Visual regression test passed!
View the visual regression test report


Perf & button fixes for Build2019 demo scene

Overview
The Build2019 scene was running at 20fps when disconnected from SpectatorView because its camera settings were set to "ultra" quality, which enabled features like anti-aliasing that caused GPU to spend ~30 ms even when it was rendering no content.
Fix this by using the default HoloLens2 camera profile in the Build2019 demo scene. Also fix several other ux issues I found in the scene such as the buttons not fully compressing.
Changes

Fixed: #285 by using the default HoloLens2 camera profile
Update the button max depth and press distance so buttons press farther
Hook up slider value update script so text field updates

I'd prefer to get the meta files fixed before checking this in, but the review looks good to go after that.

Thanks for the review, Chris! I reverted the meta files.


aggregate_time() issue when uodating to pyam 2.0

Hi, I realized updating from pyam 1.7 to 2.0 a piece of code I used in sub-annual reporting fails
rep_iamSE_sub.aggregate_time(vv, column="subannual", value="year", append=True)
gives the following error, although the input iamDF has values. Also, the code of the function aggregate_time does not seem to have changed, so I wonder what I could do to use the function.
Any suggestions on how to solve this is very welcome. Since we instructed people in a workshop to use these functions. maybe pandas pivot_table has changed behaviour?
for vv in rep_iamSE_sub.variable:
    rep_iamSE_sub.aggregate_time(vv, column="subannual", value="year", append=True)
Traceback (most recent call last):

  Cell In[26], line 2
    rep_iamSE_sub.aggregate_time(vv, column="subannual", value="year", append=True)

  File ~\AppData\Local\anaconda3\envs\message_env\Lib\site-packages\pyam\core.py:1653 in aggregate_time
    _df = _aggregate_time(
![pyamaggregation py, line 185](https://github.com/IAMconsortium/pyam/assets/22025365/33ec2b39-6a44-4826-a138-3f14d3cce179)

  File ~\AppData\Local\anaconda3\envs\message_env\Lib\site-packages\pyam\aggregation.py:186 in _aggregate_time
    .value.rename_axis(None, axis=1)

  File ~\AppData\Local\anaconda3\envs\message_env\Lib\site-packages\pandas\core\generic.py:6202 in __getattr__
    return object.__getattribute
![rep_iamSE_sub timeseries() head()](https://github.com/IAMconsortium/pyam/assets/22025365/31927fc7-255c-461a-906f-2ba2ccc63049)
__(self, name)

AttributeError: 'DataFrame' object has no attribute 'value'

I attach a spyder screenshot from rep_iamSE_sub.timeseries().head()
and an excel screenshot of the df after the pivot table, before running into the error (pyam/aggregation.py, line 185)

I will possibly attach a simple example if I have time, but would be great to get some insights in the meanwhile.
Thanks

So I looked at the test for aggregate_time() and played around a bit, but I did not manage to replicate your error.
Please make sure that you have the latest version of pandas and other dependencies, and try to run the following in a notebook:
DATA = pd.DataFrame(
    [
        ["World", "Primary Energy", "EJ/yr", 12, 15],
        ["reg_a", "Primary Energy", "EJ/yr", 8, 9],
        ["reg_b", "Primary Energy", "EJ/yr", 4, 6],
    ],
    columns=["region", "variable", "unit", 2005, 2010],
)

_df = DATA.copy()

def add_subannual(_data, name, value):
    _data["subannual"] = name
    _data[TEST_YEARS] = _data[TEST_YEARS] * value
    return _data

mapping = [("year", 1), ("summar", 0.7), ("winter", 0.3)]
lst = [add_subannual(_df.copy(), name, value) for name, value in mapping]

df = pyam.IamDataFrame(model="model_a", scenario="scen_a", data=pd.concat(lst))

df.aggregate_time("Primary Energy")

If this also fails, the problem is on your local installation. If my example works and the larger MESSAGE-workflow still fails, please provide a small dataset to illustrate the behavior.

Thanks @danielhuppmann
this work and also in my more complicated case, aggregate_time() works with each single variable.
However, in my case I have a for loop with append and as soon as the values of the first variable gets aggregated, the next aggregation phase fails (like if the variableinput was not considered)
in previous version this used to work. I replicate the issues expanding your examples, which should break
TEST_YEARS = [2005, 2010]

DATA = pd.DataFrame(
    [
        ["World", "Primary Energy", "EJ/yr", 12, 15],
        ["reg_a", "Primary Energy", "EJ/yr", 8, 9],
        ["reg_b", "Primary Energy", "EJ/yr", 4, 6],
        ["reg_a", "Secondary Energy", "EJ/yr", 18, 9],
        ["reg_b", "Secondary Energy", "EJ/yr", 14, 6],
    ],
    columns=["region", "variable", "unit"] + TEST_YEARS,
)

_df = DATA.copy()

def add_subannual(_data, name, value):
    _data["subannual"] = name
    _data[TEST_YEARS] = _data[TEST_YEARS] * value
    return _data

mapping = [("year", 1), ("summar", 0.7), ("winter", 0.3)]
lst = [add_subannual(_df.copy(), name, value) for name, value in mapping]
# remove values with year, otherwise append in aggregate_time gives error
dt = pd.concat(lst)
dt = dt[dt.subannual != "year"]

df = pyam.IamDataFrame(model="model_a", scenario="scen_a", data=dt)

for vv in df.variable:
    print(vv)
    df.aggregate_time("Primary Energy", column="subannual", value="year", append=True)

Of course I can adapt my script and simply append the aggregated values separately. But it is good to be aware of the different behaviour.
Thanks


Unexpected token name «i», expected punc «;»

Running into issues when using babel and uglifyjs together. Getting this error on compilation:
$ npm run build

><EMAIL_ADDRESS>build C:\Projects\uglifyjs-bug-report
> webpack

Hash: 59b8aeb1c10604620688
Version: webpack 3.1.0
Time: 738ms
       Asset    Size  Chunks             Chunk Names
    index.js  138 kB       0  [emitted]  main
index.js.map  164 kB       0  [emitted]  main
   [0] (webpack)/buildin/global.js 509 bytes {0} [built]
   [1] ./src/index.js 197 bytes {0} [built]
   [9] (webpack)/buildin/module.js 517 bytes {0} [built]
    + 7 hidden modules

ERROR in index.js from UglifyJs
Unexpected token name «i», expected punc «;» [./node_modules/isemail/lib/index.js:116,0][index.js:226,13]

Compiling without UglifyJS completes without errors.
The isemail library works standalone, but with the babel trnasformation, UglifyJS throws the error above.
Here is a reproduction repository:
https://github.com/AlesMenzel/uglifyjs-bug-report
Node: 8.1.2
NPM: 5.1.0
Webpack: 3.1.0

What version of UglifyJSPlugin ?

Looking at the webpack repo, the webpack 3.1.0 has "uglifyjs-webpack-plugin": "^0.4.6".

Could you try with npm i -D uglifyjs-webpack-plugin@beta, which uses uglify-es (ES2015+ Minification) to see if that's the cause ?

Yes, running with the "1.0.0-beta.1" seems to fix the issue and there is no error. But the build is twice as long, from ~750ms to ~1250ms.
$ npm run build

><EMAIL_ADDRESS>build C:\Projects\uglifyjs-bug-report
> webpack

Hash: 59b8aeb1c10604620688
Version: webpack 3.1.0
Time: 1255ms
   Asset     Size  Chunks             Chunk Names
index.js  39.3 kB       0  [emitted]  main
   [0] (webpack)/buildin/global.js 509 bytes {0} [built]
   [1] ./src/index.js 197 bytes {0} [built]
   [9] (webpack)/buildin/module.js 517 bytes {0} [built]
    + 7 hidden modules

Yes ES5 <=> ES2015 Minifaction is slower, so Error vs. --perf I guess 😛
We are working on the perf part here #77 atm
⚠  babel transpiled code is especially expensive to minimize, you can alternatively check out babili-webpack-plugin (Babel Minifier) I see if it is faster

For anyone looking at this thread after searching for the error, you're most likely using ES6+ language features in your source without having a compiler configured. Innocent line of code like this for (let i = 0; i... would reproduce the error.

If you're using Preact (don't know if the same applies for React) and experience this error it is probably caused by your imports.
When I imported Component from Preact, I accidentally imported it via import {h, Component} from 'preact/src/preact', which led to this error
It should be import {h, Component} from 'preact' of course

I found this issue today too:
Tracing dependencies for: main
Uglify file: /app/public/assets/main-dist.js
Error: Cannot uglify file: /app/public/assets/main-dist.js. Skipping it. Error is:
SyntaxError: Unexpected token name «of», expected punc «;» (line: 25294, col: 27, pos: 1364286)

Some had introduced for of into our code base. I was only able to find this out by looking at like 25294 in main-dist.js

https://github.com/webpack-contrib/uglifyjs-webpack-plugin/issues/409


Fix: Vic2 ignores the first line of localization files

It expects the first line to be:
KEY;ENGLISH;FRENCH;GERMAN;POLISH;SPANISH;ITALIAN;HUNGARIAN;CZECH;HUNGARIAN;DUTCH;PORTUGUESE;RUSSIAN;FINNISH;X
And ignores it. Thus, for example, if you swap government names around in 1.29.csv, and put democracy first:
democracy;Constitutional Republic;République Constitutionelle;Konstitutionelle Republik;;República Constitucional;;;;;;;;;;;;;x;;;

...it will be ignored and the country will revert to vanilla localization, ie. "Democracy".
It affects only a handful of keys but is trivially fixed, just copy paste the KEY line at the beginning of each file.

Oh, 1.29.csv was actually included in? My bad, it slipped in. Nevertheless, I'll take care of this issue.


fluidd: 1.16.2 -> 1.17.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/cadriel/fluidd/releases.
meta.description for fluidd is: "Klipper web interface"
meta.homepage for fluidd is: "https://docs.fluidd.xyz"
meta.changelog for fluidd is: ""
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 1.17.0 with grep in /nix/store/k9rjfvmbdijzhnbymy1aqm4hvsk4qj9b-fluidd-1.17.0

found 1.17.0 in filename of file in /nix/store/k9rjfvmbdijzhnbymy1aqm4hvsk4qj9b-fluidd-1.17.0

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

1 x86_64-linux rebuild(s)
0 x86_64-darwin rebuild(s)

First fifty rebuilds by attrpath
fluidd

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/k9rjfvmbdijzhnbymy1aqm4hvsk4qj9b-fluidd-1.17.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A fluidd https://github.com/r-ryantm/nixpkgs/archive/f8f3e70263e68698753e71f73b3a466dcc23b003.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/k9rjfvmbdijzhnbymy1aqm4hvsk4qj9b-fluidd-1.17.0
ls -la /nix/store/k9rjfvmbdijzhnbymy1aqm4hvsk4qj9b-fluidd-1.17.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

fluidd

Maintainer pings
cc @zhaofengli for testing.

Version 1.17.1 is now available. Please consider #163001. @zhaofengli can you please review it? Thank you.


Frontend code changes for Azure OCR.

Frontend code changes for Azure OCR.

@Pragya7011 Please fix build errors
And also I will push this once we get the backend piece reviewed and pushed to the main branch. Please add a comment or personally message me when its on the main branch for backend.

@johbaxter  : I have updated code for build issues . Will update once backend code is reviewed .


Add link to Core C++ API

@maxammann Thanks Max, could you also merge it?

@maxammann Thanks Max, could you also merge it?

I don't have merge permissions here :)


An error is reported when moving the pdf to the workspace

How are you running AnythingLLM?
AnythingLLM desktop app
What happened?
I use the ollama api (ollama is deployed locally), I can chat normally in the workspace, and the content returned by the background ollama server also proves that the local ollama server works properly. But when I finished uploading the documents and moved them to the workspace, the system showed an error:
[Failed to add batches to table motor: lance error: LanceError(Arrow): Invalid argument error: Values length 17280 is less than the length (768) multiplied by the value size (768) for FixedSizeList(Field { name: "item", data_type: Float32, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }, 768), C:\Users\runneradmin.cargo\registry\src\index.crates.io-6f17d22bba15001f\lance-datafusion-0.12.1\src\utils.rs:41:28]

Are there known steps to reproduce?
No response

Same

I found a fix. I hope it helps you, too. I downloaded anythingllm before but later deleted it, perhaps because the registry or some other attached files were not completely deleted, which caused this error when installing again. I used the tool HiBit Uninstaller to re-delete anythingllm. It is worth mentioning that after deleting the program with this tool, we must use advanced scanning and cleaning the relevant information of anythingllm, including the registry information. Get your computer ba
---- Replied Message ----
| From | @.> |
| Date | 08/25/2024 16:46 |
| To | Mintplex-Labs/anything-llm @.> |
| Cc | brownplayer @.>,
Author @.> |
| Subject | Re: [Mintplex-Labs/anything-llm] An error is reported when moving the pdf to the workspace (Issue #2172) |
Same
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

What? This is because your embedder model returned something badly or the vector db table was not in alignment with your embedder model dimensionality. If you changed your embedder model at any point you can no longer use your workspaces that have content in them, because the embedder model no longer matches what was used to embed content.
Typically, you can just delete the documents and reset the vector DB on the workspace's settings and reupload and embed and be done.
If you uninstall anythingllm that uninstalls the program, the storage folder, which has all your documents and workspaces can be found here https://docs.useanything.com/installation/desktop/general and you can just delete that. No need to install any third-party or scanning tool, its just a storage folder of files used by the app. You can just delete it


chore(main): release 1.2.0

:robot: I have created a release beep boop
1.2.0 (2024-08-26)
Features

anthropic: :sparkles: #111 #99 add prompt caching (175c901)

This PR was generated with Release Please. See documentation.

:robot: Created releases:

v1.2.0
:sunflower:


HIVE-25726: Upgrade velocity to 2.3 due to CVE-2020-13936

What changes were proposed in this pull request?
Velocity project announced CVE-2020-13936 on 20210309 and through NVD 20210317 to get detected internally.
https://nvd.nist.gov/vuln/detail/CVE-2020-13936
http://velocity.apache.org/news.html
Why are the changes needed?
This patch addresses security vulnerability by upgrading velocity version.
Does this PR introduce any user-facing change?
No
How was this patch tested?
Relying on existing tests.

@kgyrtkirk @nrg4878 : Please review and provide your feedback.

Fix has been merged to master. Please close the PR.

Thank you @nrg4878  for approving and merging the fix.


Percentage symbol addition to custom props values in hsl() parameters

Hello, how you doin
I'm having issues when compiling my css for production, passing an custom-prop as lightness value in an HSL value.
What happens is that it looks like CleanCSS  add the percentage symbol to the 2nd and 3rd value as it expect it to be like this, but I need to pass an custom prop.
IMPORTANT: clean-css is now in a maintenance mode. PRs are still welcome, and I will try do an occasional bugfix relase.
Precheck

Do a quick search and make sure a bug has not yet been reported;
do a quick check if the bug still exists in the latest patch version;
finally, be nice and have fun!

Environment

clean-css version - npm ls clean-css: ^4.3.0 (gulp-clean-css)
node.js version - node -v: 18.12.1
operating system: Windows 11

Configuration options
export const styles = () => {
	return gulp
		.src(PATH.styles.src)
		.pipe(gulpif(!PRODUCTION, sourcemaps.init()))
		.pipe(sass({
			includePaths: [
				'node_modules'
			]
		}).on('error', sass.logError))
		.pipe(gulpif(PRODUCTION, cleanCSS({ compatibility: "ie8" })))
		.pipe(gulpif(!PRODUCTION, sourcemaps.write()))
		.pipe(gulp.dest(PATH.styles.dest))
		.pipe(server.stream());
};

Input CSS
.my-class {
   color: hsl(0,0%,var(--l-95));
}

Actual output CSS
.my-class {
   color: hsl(0,0%,var(--l-95%));
}

Expected output CSS
.my-class {
   color: hsl(0,0%,var(--l-95));
}

Hi @paolosax - sorry for taking a while to respond. This issue is fixed in clean-css 5+. Hope you can upgrade from 4.x.

Hello @jakubpawlowicz - actually I was wrong. Or rather: I'm using "GULP-clean-css", and it looks like that this Gulp plugin is not updated to Clean CSS 5+... mmmh

Hello @paolosax - there's an opened PR in gulp-clean-css updating clean-css to version 5, but it's been unmerged for a long time now. However this comment may help you: https://github.com/scniro/gulp-clean-css/pull/89#issuecomment-1311983060


Added CTS v3 quick start sample code and tests

I work at Google for CTS team. I was tasked with creating sample code for C#. I have completed all sample code for CTS v3 but I am submitting the quick start guide first to make sure everything is formatted properly.

I signed it!

Let me know if I should create a separate GitHub account using my @google email address.

Leave a note here when the code is ready for another look.

@SurferJeffAtGoogle
All suggested fixes have been applied and pushed to the forked repo.
"Applied suggested changes
Removed HowTo from naming
Removed environment variable fixture
Updated readme files
Added verb main method"


Earned % & Traffic %

Totally erroneous percentage since the last update.
Alexandre

Hey, this has already been fixxed redownload the release.


Handle  nodes

After the AST refactor, building ecosystem fails because there are unknown roles causing docutils to emit "problematic" nodes, triggering an assertion in the parser.
This PR adds a test and suppresses the node, but I'm open to other behaviors!

Why didn’t Actions run on this PR?

A dinnae ken! GitHub just says "One or more errors occurred" without the usual option to restart the check


iOS: useWindowDimensions is not returning correct values after rotating when phone is locked.

Description
Hello React Native team. I found a bug that useWindowDimensions hook returns old values after rotating the phone when it is locked.
Steps to reproduce

Create a default app with both portrait and landscape support.
Add into the code const { height, width } = useWindowDimensions();
console.log the values height and width, and proceed with the following.
Rotate the phone to landscape
Lock the phone
Rotate the phone to portrait
Unlock the phone.
Height and Width values are not updated.

Tested on iOS.
React Native Version
0.74.2
Affected Platforms
Runtime - iOS
Output of npx react-native info
System:
  OS: macOS 14.5
  CPU: (10) arm64 Apple M1 Pro
  Memory: 1.27 GB / 16.00 GB
  Shell:
    version: "5.9"
    path: /bin/zsh
Binaries:
  Node:
    version: 22.3.0
    path: /opt/homebrew/bin/node
  Yarn:
    version: 4.2.2
    path: /opt/homebrew/bin/yarn
  npm:
    version: 10.8.1
    path: /opt/homebrew/bin/npm
  Watchman:
    version: 2<IP_ADDRESS>
    path: /opt/homebrew/bin/watchman
Managers:
  CocoaPods:
    version: 1.15.2
    path: /opt/homebrew/bin/pod
SDKs:
  iOS SDK:
    Platforms:
      - DriverKit 23.5
      - iOS 17.5
      - macOS 14.5
      - tvOS 17.5
      - visionOS 1.2
      - watchOS 10.5
  Android SDK: Not Found
IDEs:
  Android Studio: 2024.1 AI-241.15989.150.2411.11948838
  Xcode:
    version: 15.4/15F31d
    path: /usr/bin/xcodebuild
Languages:
  Java:
    version: 17.0.10
    path: /usr/bin/javac
  Ruby:
    version: 2.6.10
    path: /usr/bin/ruby
npmPackages:
  "@react-native-community/cli": Not Found
  react: Not Found
  react-native:
    installed: 0.74.2
    wanted: "*"
  react-native-macos: Not Found
npmGlobalPackages:
  "*react-native*": Not Found
Android:
  hermesEnabled: Not found
  newArchEnabled: Not found
iOS:
  hermesEnabled: true
  newArchEnabled: false

Stacktrace or Logs
Start in portrait mode:
LOG  {"height": 812, "width": 375}
Rotated to landscape:
LOG  {"height": 375, "width": 812}
Rotated to portrait
LOG  {"height": 812, "width": 375}
Rotated to landscape
LOG  {"height": 375, "width": 812}
Locked screen
Rotated to portrait
Unlocked screen

Reproducer
https://github.com/agusvazquez/react-native-bug
Screenshots and Videos
No response

Potentially a duplicate of one of those:
https://github.com/facebook/react-native/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+useWindowDimensions

Hey Mr Bot don't close it we need a fix 👍 😅

Hi I tried and seems it works in the latest master branch, maybe this is fixed in the latest version?

Tried again in react-native 0.76.5 and it seems to work fine.
Thanks for the help.


Demote Inactive Citizens to Resident - they can come back! #113

Finished demote inactive citizen. Added updateactivs and updateactive to proposals, Modified makecitizen to add an active entry in proposals state.

Deployment

[ ] accounts
[ ] proposals


Move all constants to file

Move all the constants at the top of the index to a separate module, and use destructuring to define them as necessary in the index. This will facilitate splitting the code into smaller, more logical modules later on.

+1

+1

You can remove changing the repo when not in production (line 8). That was just a test repo I used for development.

@Smittyvb Okay, will do.

This has merge conflicts, even if we have enough votes, ChaosBot will fail to merge this.

If you would merge this
https://github.com/Chaosthebot/chaos2/pull/14
Would this then be able to merge or do we need a separate pull request for that?


Added handling of rate limiting errors

Thanks for this! I did a tiny bit of cleanup and committed it to master as 0d3e99c0900e44c328cfecdd2aa2a8010b24fc78

@paulhammond Thanks for merging :) Could you do a release please?

@paulhammond +1 for release


Adding text around questions

I could use some guidance on how to add text in the questions area. For example, it would be nice on Step 3 to have "See Resource: Exposure Assessment" replaced with just "Exposure" and then, below that, have some text directing them to the exposure assessment resource (ideally, linking to it as well). I tried just adding a description but it broke that step.

Can put in an empty question and make it radio buttons but then don't add any radio buttons. Then give it a description. That should work.

See step 5, should be some examples there.


Replace BareVersion with version-number crate

Replace BareVersion with https://github.com/foresterre/version-number

I came to open an issue to allow skipping patch version in --min and --max, from a brief look at version-number, it would do that.


Log to a file - Feature Request

I couldn't see this documented or implemented but I needed the colourised output from Evilginx2 to be stored to a file. Best way I found to do was to edit the end of log/log.go with the below (Bash shell redirection wouldn't work for my use case). This did the trick for me:
165     logFile, err := os.OpenFile("/var/log/evilginx2.log", os.O_CREATE | os.O_APPEND | os.O_RDWR, 0666)
166     if err != nil {
167         panic(err)
168     }
169
170     var formatted_msg = "\r[" + time_clr.Sprintf("%02d:%02d:%02d", t.Hour(), t.Minute(), t.Second()) + "] [" + sign.Sprintf("%s", LogLabels[lvl]) + "] " + msg.Sprintf(format, args...)
171     logFile.WriteString(formatted_msg)
172     return formatted_msg

Could make it configurable to either enable or disable logging for the sake of it.

be sure to close that opened file as well
check go docs on this topic: https://pkg.go.dev/os#example-OpenFile-Append
// If the file doesn't exist, create it, or append to the file
logFile, err := os.OpenFile("/var/log/evilginx2.log", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
    log.Fatal(err)
}
if _, err := f.Write([]byte("appended some data\n")); err != nil {
    f.Close() // ignore error; Write error takes precedence
    log.Fatal(err)
}
if err := f.Close(); err != nil {
    log.Fatal(err)
}

But that's really a very basic example provided by go.dev, not especially ideal for real world scenarios. In production, this could potentially cause a lot of disk I/O
I'd enhance this with a bufio.Writer to handle the writing part and the I/O problem

**Hi 👽 ! If you are interested in ⚙Advanced Phishing⚙ and the ✨Evilginx3✨ tool then I offer you Google Gmail and Custom Phishlets with a ✅Free Tutorial✅ which arrives if the 📈Telegram reaches 50 Subscribers📈
⏳ GOOGLE GMAIL PHISHLET 2024 WITH PROOF ⏳
->
⛔https://www.youtube.com/watch?v=nA47UknIE6A⛔
🔓 Ask your questions and a free tutorial from 50 subs on telegram !!!
->
📩https://t.me/Noremorse1011📩**


Downloading both a media and a post pic and they're identical

How can I only download one of them?   They are identical images, one just has "media" in the name and one has "post" in the name

This has been discussed in some issue previously. Essentially this is being done because there isn't a definitive proof that post and media are always identical. So far no one done any tests to prove that it is safe to ignore one or another.


[Improve] - Logo Design

Improvements:

Design a logo for the website, do share the respective design
Change all the occurrences of the logo as well

Note: Logo should be relevant to the website application

I would like to work on this.

@vishwas031 , go ahead!


Set up payment processors for Open Collective

[x] Connect a Stripe account (for contributions).
[ ] Connect a PayPal account (for contributions and payouts).

Depends on:

[x] #14

Stripe is now connected to the Lunar bank account and Open Collective.

I think I need to contact Open Collective support regarding PayPal.

This will be handled by our fiscal host going forward; see #31.


certs not updating. leader-election blocked by lock?

What happened:

Previously working / updating Certificates not updating.
Two instances of Openshift ACME running.

One instance reporting this...
I0103 00:24:46.571147       1 leaderelection.go:352] lock is held by openshift-acme-7f65979ff9-hgsz4_8f58d3f6-9cf7-4745-af7b-476b0505caa9 and has not yet expired
I0103 00:24:46.571381       1 leaderelection.go:247] failed to acquire lease fg/acme-controller-locks
Other instance reporting this...

I0103 00:24:16.217493       1 reflector.go:432] k8s.io/client-go@v0.17.0/tools/cache/reflector.go:108: Watch close - *v1.Route total 0 items received
I0103 00:25:04.539294       1 reflector.go:432] k8s.io/client-go@v0.17.0/tools/cache/reflector.go:108: Watch close - *v1.LimitRange total 0 items received
I0103 00:25:15.362207       1 reflector.go:432] k8s.io/client-go@v0.17.0/tools/cache/reflector.go:108: Watch close - *v1.ReplicaSet total 0 items received
I0103 00:26:15.924614       1 reflector.go:432] k8s.io/client-go@v0.17.0/tools/cache/reflector.go:108: Watch close - *v1.Service total 0 items received
I0103 00:27:04.606876       1 reflector.go:432] k8s.io/client-go@v0.17.0/tools/cache/reflector.go:108: Watch close - *v1.ConfigMap total 2054 items received
I0103 00:27:30.959775       1 reflector.go:432] k8s.io/client-go@v0.17.0/tools/cache/reflector.go:108: Watch close - *v1.LimitRange total 0 items received
I0103 00:27:55.497750       1 reflector.go:432] k8s.io/client-go@v0.17.0/tools/cache/reflector.go:108: Watch close - *v1.Secret total 9 items received

What you expected to happen:
Clean logs and certificates up to date.
How to reproduce it (as minimally and precisely as possible):
Not sure.
Anything else we need to know?:
Environment:

OpenShift/Kubernetes version (use oc/kubectl version):
OKD 4.7.0

Others:

@tnozicka

seeing this when loading the cert...
[I] jkassis@Jeremys-MBP ~ [124]> ws "wss://pubsub.shinetribe.media/connPut?ConnUUID=b3f0b2d8-f5f8-452c-83fc-c476ecb7a3df"                               01.02 16:36
x509: certificate has expired or is not yet valid: current time 2022-01-02T16:36:11-08:00 is after 2022-01-02T01:42:28Z
[I] jkassis@Jeremys-MBP ~ [1]>                                                                                01.02 16:36

brought the pods down and the "leader election blocked" logs reappear. proceeding as if this is normal. looking at the certificate status, it appears that the cert is up for re-issue on 02-01, which seems odd given that the fetched cert has already expired.
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  creationTimestamp: '2021-10-04T02:24:53Z'
  generation: 3
  managedFields:
    - apiVersion: cert-manager.io/v1
      fieldsType: FieldsV1
      fieldsV1:
        'f:spec':
          .: {}
          'f:commonName': {}
          'f:dnsNames': {}
          'f:issuerRef':
            .: {}
            'f:kind': {}
            'f:name': {}
          'f:secretName': {}
      manager: Mozilla
      operation: Update
      time: '2021-10-04T02:38:51Z'
    - apiVersion: cert-manager.io/v1
      fieldsType: FieldsV1
      fieldsV1:
        'f:spec':
          'f:privateKey': {}
        'f:status':
          .: {}
          'f:conditions': {}
          'f:notAfter': {}
          'f:notBefore': {}
          'f:renewalTime': {}
          'f:revision': {}
      manager: controller
      operation: Update
      time: '2021-12-03T01:42:28Z'
  name: pubsub-shinetribe-media
  namespace: fg
  resourceVersion: '307455716'
  selfLink: /apis/cert-manager.io/v1/namespaces/fg/certificates/pubsub-shinetribe-media
  uid: a528dc92-636c-40c8-862e-38dfa6986cc7
spec:
  commonName: pubsub.shinetribe.media
  dnsNames:
    - pubsub.shinetribe.media
  issuerRef:
    kind: Issuer
    name: le-wildcard-issuer
  secretName: cert-pubsub-shinetribe-media
status:
  conditions:
    - lastTransitionTime: '2021-10-04T02:42:30Z'
      message: Certificate is up to date and has not expired
      observedGeneration: 3
      reason: Ready
      status: 'True'
      type: Ready
  notAfter: '2022-03-03T00:44:07Z'
  notBefore: '2021-12-03T00:44:08Z'
  renewalTime: '2022-02-01T00:44:07Z'
  revision: 3

Seems like the algo that determines the renewal time is broken?!? Here's what my browser gets for that cert...

I believe problem has been there all along.
Forced to delete the Pods once in a while to ensure renewal process gets triggered.

encountering this issue as well. have tried force deleting the pods and bringing running pods down to 0 and bringing it back up but lock still held by some ghost


OnLoadEvent not being fired

What version of go-chrome are you using (tag, hash, etc.)?
v1.0.0-rc7
What behavior do you expect? What are you trying to accomplish?
I am trying to print a PDF. I expect it to be printed when the 'page load' event is fired.
What behavior are you experiencing instead?
The chrome browser loads and the program hangs until the timeout.
Sample output:
➜  tester go run main.go
 info Starting process
   ⇢  flags=--addr=localhost --disable-extensions --disable-gpu --headless --hide-scrollbars --no-first-run --no-sandbox --port=9222 --remote-debugging-address=<IP_ADDRESS> --remote-debugging-port=9222 --user-data-dir=/var/folders/bz/84vb3m3d4sd24h07n24_kzq1gx0ldc/T/ path="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
   ⇢  chrome.chromium.go:238 github.com/mkenney/go-chrome/tot.(*Chrome).Launch
   ⇢  2019-06-19T16:06:52.996-06:00
 info New socket connection listening
   ⇢  socketID=1 url="ws://localhost:9222/devtools/page/49291603D1EBC96427F879A4F1581100"
   ⇢  socket.socketer.go:74 github.com/mkenney/go-chrome/tot/socket.New
   ⇢  2019-06-19T16:06:54.007-06:00
 info Websocket connection established
   ⇢  status="101 WebSocket Protocol Handshake" url="ws://localhost:9222/devtools/page/49291603D1EBC96427F879A4F1581100"
   ⇢  socket.web_socketer.go:37 github.com/mkenney/go-chrome/tot/socket.NewWebsocket
   ⇢  2019-06-19T16:06:54.010-06:00

How can this behavior be reproduced?
Run this code with go run main.go:
package main

import (
	"encoding/base64"
	"errors"
	"fmt"
	"io/ioutil"
	"log"
	"os"
	"os/exec"
	"time"

	chrome "github.com/mkenney/go-chrome/tot"
	"github.com/mkenney/go-chrome/tot/page"
)

func main() {

	chromeApp, err := Chrome()
	if err != nil {
		panic(err)
	}

	browser := chrome.New(
		&chrome.Flags{
			"addr":                     "localhost",
			"disable-extensions":       nil,
			"disable-gpu":              nil,
			"headless":                 nil,
			"hide-scrollbars":          nil,
			"no-first-run":             nil,
			"no-sandbox":               nil,
			"port":                     9222,
			"remote-debugging-address": "<IP_ADDRESS>",
			"remote-debugging-port":    9222,
		},
		chromeApp, "/tmp", "/dev/null", "/dev/null",
	)

	if err := browser.Launch(); err != nil {
		panic(err)
	}

	websiteURL := "https://www.google.com"
	tab, err := browser.NewTab(websiteURL)
	if err != nil {
		panic(err)
	}

	if enableResult := <-tab.Page().Enable(); nil != enableResult.Err {
		panic(err)
	}

	results := make(chan *page.PrintToPDFResult)

	tab.Page().OnLoadEventFired(
		func(event *page.LoadEventFiredEvent) {
			log.Printf("loaded the thing")

			pdfResult := <-tab.Page().PrintToPDF(&page.PrintToPDFParams{})
			results <- pdfResult
		},
	)

	timeout := time.After(time.Duration(100) * time.Second)

	select {
	case <-timeout:
		panic("Timeout exceeded")
	case result := <-results:
		if result.Err != nil {
			panic(result.Err)
		}

		data, err := base64.StdEncoding.DecodeString(result.Data)
		if err != nil {
			panic(err)
		}

		err = ioutil.WriteFile(buildTemporaryFilename("c02bfcab-2854-448e-8ce0-a4fe3d8ad7d6", "pdf"), data, 0644)
		if err != nil {
			panic(err)
		}
	}
}

func buildTemporaryFilename(id, ext string) string {
	return fmt.Sprintf("%s%c%s.%s", os.TempDir(), os.PathSeparator, id, ext)
}

// Chrome determines that path to Chrome on the system
func Chrome() (string, error) {
	chromeBinaries := []string{
		"google-chrome-stable",
		"chromium-browser",
		"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
	}

	for _, chrome := range chromeBinaries {
		path, err := exec.LookPath(chrome)
		if err == nil {
			return path, nil
		}
	}

	return "", errors.New("chrome was not found in your path")
}

Hi @Daniel-Houston,
It does return fast, but it looks to me like your script works as intended. I noticed that it creates some zombie processes so I added a Close() call and changed the file path for testing.
*** main.go     2019-06-20 18:53:45.000000000 -0600
--- test.go     2019-06-20 19:10:48.000000000 -0600
***************
*** 37,40 ****
--- 37,41 ----
                chromeApp, "/tmp", "/dev/null", "/dev/null",
        )
+       defer browser.Close()

        if err := browser.Launch(); err != nil {
***************
*** 44,47 ****
--- 45,49 ----
        websiteURL := "https://www.google.com"
        tab, err := browser.NewTab(websiteURL)
+       defer tab.Close()
        if err != nil {
                panic(err)
***************
*** 78,82 ****
                }

!               err = ioutil.WriteFile(buildTemporaryFilename("c02bfcab-2854-448e-8ce0-a4fe3d8ad7d6", "pdf"), data, 0644)
                if err != nil {
                        panic(err)
--- 80,84 ----
                }

!               err = ioutil.WriteFile("/tmp/tst.pdf", data, 0644)
                if err != nil {
                        panic(err)

Adding the Close() calls uncovered that the socket listeners don't shutdown gracefully and throw some errors, I will take a look at that.
But, it created the pdf I expected: tst.pdf

log output:
→ go run test.go
 info 2019-06-20T19:17:38.657-06:00 Starting process
   ⇢  flags={"addr":"localhost","disable-extensions":null,"disable-gpu":null,"headless":null,"hide-scrollbars":null,"no-first-run":null,"no-sandbox":null,"port":9222,"remote-debugging-address":"<IP_ADDRESS>","remote-debugging-port":9222,"user-data-dir":"/var/folders/0x/xys3g4w97hl8_kkgx3kh2zw00000gp/T/"} path="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
   ⇢  chrome.chromium.go:238 github.com/mkenney/tmp/vendor/github.com/mkenney/go-chrome/tot.(*Chrome).Launch
 info 2019-06-20T19:17:39.678-06:00 New socket connection listening
   ⇢  socketID=1 url="ws://localhost:9222/devtools/page/B45B743DCFE4739A319B81778837738F"
   ⇢  socket.socketer.go:74 github.com/mkenney/tmp/vendor/github.com/mkenney/go-chrome/tot/socket.New
 info 2019-06-20T19:17:39.683-06:00 Websocket connection established
   ⇢  status="101 WebSocket Protocol Handshake" url="ws://localhost:9222/devtools/page/B45B743DCFE4739A319B81778837738F"
   ⇢  socket.web_socketer.go:37 github.com/mkenney/tmp/vendor/github.com/mkenney/go-chrome/tot/socket.NewWebsocket
 info 2019-06-20T19:17:40.197-06:00 Executing handler
   ⇢  event="Page.loadEventFired" handler#=0 socketID=1
   ⇢  socket.socketer.go:218 github.com/mkenney/tmp/vendor/github.com/mkenney/go-chrome/tot/socket.(*Socket).handleEvent
2019/06/20 19:17:40 loaded the thing
error 2019-06-20T19:17:41.384-06:00 5001: An unknown error occurred
   ⇢  socketID=1
   ⇢  socket.socketer.go:302 github.com/mkenney/tmp/vendor/github.com/mkenney/go-chrome/tot/socket.(*Socket).listen
error 2019-06-20T19:17:41.385-06:00 nil response from socket
   ⇢  socketID=1
   ⇢  socket.socketer.go:309 github.com/mkenney/tmp/vendor/github.com/mkenney/go-chrome/tot/socket.(*Socket).listen
error 2019-06-20T19:17:41.385-06:00 Unknown response from web socket
   ⇢  data="{\\\"error\\\":null,\\\"id\\\":0,\\\"method\\\":\\\"\\\",\\\"params\\\":null,\\\"result\\\":null}" method="" responseID=0 socketID=1
   ⇢  socket.socketer.go:325 github.com/mkenney/tmp/vendor/github.com/mkenney/go-chrome/tot/socket.(*Socket).listen
 info 2019-06-20T19:17:41.386-06:00 Socket shutting down
   ⇢  socketID=1 url="ws://localhost:9222/devtools/page/B45B743DCFE4739A319B81778837738F"
   ⇢  socket.socketer.go:337 github.com/mkenney/tmp/vendor/github.com/mkenney/go-chrome/tot/socket.(*Socket).listen
 info 2019-06-20T19:17:41.398-06:00 Chromium exited
   ⇢  signal="signal: interrupt"
   ⇢  chrome.chromium.go:132 github.com/mkenney/tmp/vendor/github.com/mkenney/go-chrome/tot.(*Chrome).Close

Interesting that it works for you. When I run it on my computer, I get the first three info messages, ending in
info Websocket connection established
   ⇢  status="101 WebSocket Protocol Handshake" url="ws://localhost:9222/devtools/page/465162C4B03BF774D1A696D59D3C930D"
   ⇢  socket.web_socketer.go:37 github.com/mkenney/go-chrome/tot/socket.NewWebsocket
   ⇢  2019-06-21T09:52:21.872-06:00

But I don't get the message saying
 info 2019-06-20T19:17:40.197-06:00 Executing handler
   ⇢  event="Page.loadEventFired" handler#=0 socketID=1

Do you have any idea what would be causing the event not to fire on my computer? Is there anything I can do to debug it?

You could try starting Chrome in dev mode from the command line, something like
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --addr=localhost --port=9222 --remote-debugging-port=9222 --remote-debugging-address=<IP_ADDRESS> --disable-extensions --disable-gpu --headless --hide-scrollbars --no-first-run --no-sandbox"

and then try your script and see if that makes any difference. Dev mode isn't the default, I'm not sure why it's on for me. I've messed with my Chrome settings a lot over the years though so who knows 🤷♂
Alternatively, if you use Docker there's a docker-comopse.yml that will start a headless chrome service that you can access at localhost:9222. It might complain about the port if you have any zombie background chrome processes lurking around though, you can just use a different port in the docker-compose.yml file.
docker-compose up -d chrome

Let me know if that helps at all.

The graceful socket shutdown updates are included in v1.0.0-rc8.


accuracy = sum(correct)/n_ground_truth : ZeroDivision Error: Division by Zero

I got this error while trying to evaluate top-1 video accuracy of a recognition result.I used the same script provided
python -m util_scripts.eval_accuracy ~/data/kinetics.json ~/data/results/val.json --subset val -k 1 --ignore
does anyone have any idea how to solve this problem ( from my understanding it's unable to load the groundtruth) ?
ps: for the training I used the UCF101.

hello there,i yet meet the same questions,please tell me how do you solve?

@9bxlue just in the --subset change it to validation not val


Added ability to use SQL "table.field" notation as FieldConfig name

Example: If i use field name "u.id" now first will be tried to read "id" property and then "u". If something went wrong, now exceptions are thrown.

Hi!
Thanks for contribution. Very interesting improvement. It's definently needful.
Now I afraid that it can broke backward compatibility & this package has no tests in opposition to https://github.com/view-components/grids so merging will be not easy.
Can you modify pull request or create another for merging it into 'dev' branch?
Regarding this part:
$res = is_object($res) ? $res->{$part} : $res[$part];

I think it requires isset check.
todo: check that it works with DbalDataProvider

One more question: why you don't use view-components/grids ? Contributing there would be much easier.

It also should work when filtering (currently: ambiguous column)


Fix root pins, attempt 3.

Following failed attemps in #10 and #11, this is a third attempt to fix the hashes for the cds-crtools metapackage such that concurrently buildings against different versions of root_base actually results in unique packages.

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.


spark: provided-hadoop -> nixpkgs hadoop

Motivation for this change
spark should use nixpkgs provided hadoop to make it consistent
for bigger Nix hadoop + spark setup
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

conflict resolved


Use Nodejs vars in Python

Today, I can use store to save Nodejs vars for use in Python cells, but I can't do the opposite. That is, I can't access Python vars in %%node cells.

So the syntax for "store" is like this:
%%node
var x = [ { a:1, b:2, c:3}, {a:3, b:4, c:2}];
store(x, 'y');

and a data frame 'y' is made available in Python.
If you need to move Python data to Node, then presumably you are doing Python code - so the mechanism needs to be Python:
store(df, 'z')

would be the nearest equivalent syntax? Or I could do node.store(df,'z') ?

@glynnbird we should implement like the scala bridge and automatically import all the variables that we can bridge from the Notebook global scope. That way you don't need to use store on Python, just declare the variable in Python and start using it in Node


Add private scanning to VirusTotal service

Describe the solution you'd like
Add private file scanning as a submission parameter option to the service
Additional context
Note: This requires a specific license/API key that allows use of the Private Scanning APIs.
Reference: https://developers.virustotal.com/reference/post_files
API Endpoint: /api/v3/private/files

@cccs-rs - any updates on the feature?

Based on the parameters, we'd have to add a series of submission params to fully enable Private Scanning of files (unless we want to take all the defaults if we can):
https://docs.virustotal.com/reference/post_files
Also since the VT client doesn't seem to support Private Scanning natively, we'll also have to do that implementation as well (which I don't imagine should be hard)
This can be added if there's still interest in the feature.


Crash immediately after launch

Black screen and it stops working. At first it happened just after the 4gb popup, so first thing I tried was do the 4gb patch manually, and it kept crashing just the same way except for the popup not showing anymore. Then I replaced the modded BIO4 folder with the original, and it launched just fine! The tweeks GUI works and I was able to test some settings. Apparently it has something to do with the modded BIO4 folder (I checked its size and everything should be fine).
Specs:
AMD A10 4C + 8G
AMD Radeon RX 560 Series 4gb VRAM
16.0 GB RAM
Windows 10 64bit
Clean Steam Install
Of course the game ran 10/10 before
bio4.exe.20220203151435.log

Did you delete the original BIO4 folder before placing the new one from the HD project?

Yes

Seems to be a crash in XCOMPRESS::fast_decode_verbatim_block, maybe one of the .lfs files is corrupt in some way.
If you install something like QuickSFV I can make a file you can use to check your install with, will upload in a sec.

I'm guessing my safest bet is to download everything again, but in any case let's try that while I do

BIO4-HDProject-SFV.zip
If you install QuickSFV you can extract that next to your BIO4 folder and run it, and that should verify the install for you, might take 5-10 minutes to go through it though.

Yep, I already installed it. Let's check that out. Ty for the trouble :)

@emoose I sent you an email, just a heads up in case it gets lost in your spam folder.

So yeah, it found a good deal of hash mismatches. The console just closed with no report file left behind tho, so I'll just download the files again.

Hmm, I wonder if albert should've included an automatic installer with hash checking and all.
Not sure if he's open to re-release an updated package with such changes. I'll have to ask him later.

@emoose I sent you an email, just a heads up in case it gets lost in your spam folder.

Replied 👍

So I downloaded the entire mod again and this time it launched, but now I'm experiencing some other consistent crashes on numerous different circumstances. I could start documenting them all along with their crashdumps, but running the SFV again it looks like there are still some hash mismatches, so I'll have to assume that's the problem again. Idk, I'm downloading it from the torrent file so idk idk idk

In any case, thank you guys a ton for the patience.


ctest: portably running two mpi applications concurrently

For testing rendezvous and an application coordinating with it ctest execute_process is being called with two COMMANDs here:
https://github.com/SCOREC/wdmapp_coupling/blob/31b75236b382765dd73d445f6933bad525d75270/cmake/runTwoMpiJobs.cmake#L6-L17
The call stack is listed below for reference.
This usage of execute_process was intended to be a mechanism to concurrently run multiple applications with the output of application i piped via stdout into the input of application i+1.  In our usage, the applications ignore stdin so the piping mechanism is not needed.
The problem being hit is described here:
https://stackoverflow.com/questions/61007499/cmake-ctest-execute-processes-in-parallel-as-a-test
if the first application exits before the second then the second test fails with SIGPIPE.

This is the high-level developer call to add a test:
https://github.com/SCOREC/wdmapp_coupling/blob/31b75236b382765dd73d445f6933bad525d75270/test/CMakeLists.txt#L50-L52
that calls
https://github.com/SCOREC/wdmapp_coupling/blob/31b75236b382765dd73d445f6933bad525d75270/test/CMakeLists.txt#L33
which calls the following macro when ctest is called to run the two requested mpi applications:
https://github.com/SCOREC/wdmapp_coupling/blob/31b75236b382765dd73d445f6933bad525d75270/cmake/runTwoMpiJobs.cmake#L2

Using a bash script now https://github.com/SCOREC/wdmapp_coupling/blob/16eb43d30f8b7c39bb98bf6afbead537a68f48f8/ctest/runMultipleMpiJobs.sh ... gave up on portability.