Any walkthrough, examples for connecting to Panono?

Cannot get connected. Do you have any walkthroughs or examples on getting started? Thanks.

Hey!
In https://github.com/florianl/panonoctl#documentation all steps are written down.
>>> from panonoctl import panono
>>> cam = panono()
>>> cam.connect()
>>> cam.capture()
>>> cam.disconnect()

Best regards,
Florian

Hi @florianl,
Sorry for the confusion, I meant more along the lines of how do I connect. I am having trouble identifying the IP of the Panono, the port to talk on, and which path to connect to; also, do I hard code these values into panono.py?
Thanks

Hi @neuhargj ,
the Panono periodically sends Simple Service Discovery Protocol (SSDP) messages.
These messages are used by the provided library, to extract the necessary information to communicate with the camera. Thus, you do not need to enter such information.
You simply need to be connected to the WiFi from the Panono.

If there are further questions, feel free to ask @neuhargj .
Otherwise, I will close this issue.

Hi @neuhargj
I pushed a new version. Please let me know, if you've got still problems.
In my last walkthrough I missed to do the auth().
>>> from panonoctl import panono
>>> cam = panono()
>>> cam.connect()
>>> cam.auth()
>>> cam.capture()
>>> cam.disconnect()

Kind regards,
Florian

I close this issue. If there are further problems, feel free to open an issue.
Thanks for your feedback!

hi @florianl, sorry for the late response, ive had a few other projects going. i am still unable to connect, on cam.connect() i get "timed out" and "false", i was trying to connect through a virtual machine and i didnt know if the virtualization was causing an issue, so i was trying to run on a raspberry pi instead.

Hej @neuhargj
if you are connected to the WLAN of the panon, run as root
tcpdump -vv -A -s 0 -i $YOUR_WLAN_INTERFACE udp port 1900 and host <IP_ADDRESS>
do you get SSDP messages? If there are no messages, then it is a network problem.
Otherwise, if there is an output, please paste it here.
Kind regards,
Florian

@florianl,
i was able to get past the timed out, false problem, but hten i ran into an issue with the websocket module not having a create_connection method; i researched this previously and tried to manually switch over to a different module, but this did not work.

Did you install panonoctl from github.com or using pip?

@florianl, ive tried from both the github project and pip from my macbook pro, an ubuntu vm, and a raspberry pi, no luck

hej @neuhargj
the missing attribute create_connection looks like a missing dependency.
Is websocket installed as required in https://github.com/florianl/panonoctl/blob/master/setup.py#L33 ?
For an easy update, you can use: pip install -U panonoctl
Do you get SSDP messages? And how do they look like?

@florianl, do you have a link to the websocket module? all i can find is websocket-connection module that has a create_connection function
https://pypi.python.org/pypi/websocket-client/
i do not get any ssdp messages

hej @neuhargj
If there are no ssdp messages, you are not connected to the WLAN of the panono properly.

@florianl,
i am getting the ssdp packets, but still getting the same error, please see below:

hej @neuhargj
I fixed the issue with the requirement by using websocket-client in 'setup.py'. I'm sorry for the circumstances.
For an easy update, you can use: pip install -U panonoctl
Looking forward for your feedback

@florianl,
thanks for your updates, everything is working fine now. you've built a great tool

thanks @neuhargj
now - I will close this issue.


Let user upload GeoJSON file of nogos

Hi,
This is a very first draft of the ability to create nogos from a GeoJSON file or URL.

This is definitely not yet ready to get merged, I'm just opening this to get feedback, especially on the UI part.
This can be tested on Paris using this GeoJSON export of impacting roadworks for instance: https://opendata.paris.fr/explore/dataset/chantiers-perturbants/download/?format=geojson&timezone=Europe/Berlin.
TODO:

[x] read a nogoWeight feature property from the GeoJSON if present and only use the weight input from the dialog as a default if not.
[x] read nogo circles
[ ] Add a buffer geometry feature?

Best,

Just as a feedback, I'm asking whether the input file can accept also line features in addition to areas. The use case might be simply to pre-select some roads that for some reasons are not usable, and upload portions of roads to be used as no-go "areas".
Another clarification is the nogo weight: would it be in your idea a single weight to "calibrate" the no-go?
It would be also great to provide a way to assign different weights to the no go areas/lines in the GeoJSON and to make this understandable by the routing algorithm...

I'm asking whether the input file can accept also line features in addition to areas.

It is currently accepting line and polygon features from GeoJSON file. The use case I have in mind is precisely to upload opendata features about road works to avoid them :)

Another clarification is the nogo weight: would it be in your idea a single weight to "calibrate" the no-go?

This is a penalty used by BRouter under the hood to compute routing. This is basically a cost proportional to the distance travelled inside the nogo area.

It would be also great to provide a way to assign different weights to the no go areas/lines in the GeoJSON and to make this understandable by the routing algorithm...

This is already possible in the routing algorithm but not in the frontend yet. See #144 for some discussion about it.

It would be also great to provide a way to assign different weights to the no go areas/lines in the GeoJSON and to make this understandable by the routing algorithm...

This is already possible in the routing algorithm but not in the frontend yet. See #144 for some discussion about it.

Although there is no UI for it (yet), the client-side code does support individual weights, you can already pass them with a URL?
So I guess it might make sense to read a nogoWeight feature property from the GeoJSON if present and only use the weight input from the dialog as a default if not.
This would probably also enable to add an export option for nogo areas to GeoJSON later and importing them with this feature again (in part related to #16 and abrensch/brouter#91).

I pushed a new commit to improve the handling of loading the nogos from geojson. Nogo weight should be used as a fallback value now and points can be imported as circles.
The main remaining issue is that the URL hash is not updated when loading the geojson file.

The main remaining issue is that the URL hash is not updated when loading the geojson file.

Updating the route and hash can be done by adding the following calls after nogos.setOptions:
    updateRoute({
        options: nogos.getOptions()
    });
    urlHash.onMapMove();

By the way, @nrenner do you know any easy way to buffer a geometry in meters in Brouter-web?

For example with Turf: http://turfjs.org/docs/#buffer
To work with Leaflet layers I think you would need to convert with toGeoJSON() methods and back with GeoJSON layer.
What is your use case?

Some additional thoughts:

UI

As loading additional things is planned (e.g. tracks #30), it might be better to already create a "Load" dropdown menu (like "Download") with "Nogos" as single item for now
Not sure, but I personally would prefer "Load" instead of "Upload", as it's not really uploading to and storing on the server, just loading into the browser, what do you think?
I would prefer About being the last nav item

Why is the nogo weight mandatory now? Wouldn't it take the default prohibitive value on the server if not set?
i18n keys are missing (translatable since #159), but we also can do that afterwards
We haven't used Promise and fetch yet. I have looked into browserslist tools like browserslist-generator to check if we have visiting Browsers that don't support them and found a few. But I would need to check further and with current logs and will merge as is for now.

What is your use case?

Oh, great, I missed that Turf could do buffering! Typical use case is when using raw OpenData files for road works for instance, sometimes the lines / polygons are very narrow (in the sense that they cover only the very particular portion of the street which is impacted) but not well positioned (with a slight offset or not covering the whole highway from OSM data). Using a bit of buffering is a very easy solution for this. Another reason is when the polygon for the nogo only cover half of the road, but you expect the road works to actually have impact on a much larger area (traffic jam, etc).
Thanks for all the feedback, I'll have a look at this during the weekend and update the PR!

Hi,
I should have reworked this code a bit to include the remarks (i18n, UX etc) as well as the buffering. Everything is squashed and rebased against the current master branch.
The UX part is probably sub-optimal. I think a progress bar could be shown during loading and processing of GeoJSON file for instance, but this is probably fine for a first draft.
This should be ready for review and (hopefully) mergeable. Let me know if you have any question or doubt on this.
Best,

Thanks for the update and your contribution!

The UX part is probably sub-optimal.

There is always room for improvement, it's fine as it is.

Thanks!


Where create sample projects can be viewed?

Hi,
I requested to create new sample project and I received an e-mail the project and the iModel were created.
Where I can see (to list) the project (or more projects in future)? I expected it will be in my ProjectWise projects list, but I see only the empty project, not iModel. Maybe I did wrong that I created the project with the requested name upfront (before the request was sent).
With regards,
Jan

Hi Jan,
The project has been created for you in our test drive organization. There is a possibility that the project you created is your organization is “shadowing” the one we created for you. Can you search the project name using the Search tool? If you are still unable to find it, we can investigate further.

Hi,
sorry for the delay, I was outside home for a few days. Yes, using search (and also it seems there was some delay, so the created project was not displayed in the list immediately) I was able to identify the project, marked as External.

Is it correct I do not see any file inside the project?
I plan to play with examples and to study more how authentication and access to data work in API.
With regards,
Jan

Hi Jan,
Great. You won't see a file but you should be able to see the iModel listed when you click on the iModel link on the project page.

Thanks,
Roop

Thanks for your quick response!

you should be able to see the iModel listed when you click on the iModel link on the project page.

Unfortunately iModel information is missing in my project details:

Regards,
Jan

Hi Jan,
We just checked the logs, and there seems to have been an error during registration. We just fixed it and you should see it now. We apologize for the inconvenience.

Hi,

We just fixed it and you should see it now.

Yes, it works now, I see the project in my list:

We apologize for the inconvenience.

No problem, thanks for your help! I treat it as an evaluation / testing period, when issues are expected. And when there are no issues, the testing has not been good enough :smirk:
Because I have the access to example data and I see it in the list and can access it using viewing demo application, I close this issue as solved.
With regards,
Jan


How best to handle extending req.log?

Context:
We want to attach the pino-http middleware first so that req.log is always defined and can be used in all Express middleware / all route handlers. However, we also always want to include information about which tenant/customer a request is for, but our authorization middleware gets invoked later.

So we hack it by reassigning req.log. However, this middleware is defined and exported from an internal library.
This has caused some friction for the internal consumers of our library when they've setup AsyncLocalStorage to capture req.log, following this article. https://blog.logrocket.com/logging-with-pino-and-asynclocalstorage-in-node-js/
The issue was that this only captured the original req.log, and not the new child logger from our middleware, and required a less than ideal workaround to lazily dereference req.log.

We could extend our internal library to be aware of AsyncLocalStorage or move the AsyncLocalStorage logger into the library itself, but is there a better/sanctioned solution or something in the API I've missed that simplifies this?

@mcollina, thanks for the quick response. I understand there's no sanctioned solution for AsyncLocalStorage, but is there a better solution for middleware than reassigning req.log if all downstream middleware should have extra bindings?
I don't think I was very clear in asking that question, but I understand the answer is probably the same.

@mcollina would the team be open to merging something like this in (of course with documentation updates and tests) https://github.com/ramblingenzyme/pino-http/pull/1 to provide an official API which lets you extend the bindings and not reassign req.log/res.log?

No, not really. Proxy is very slow.

@mcollina, I've just noticed that Pino has a setBindings method, which is in the TS types but not in the API docs. Should I actually be using that something like logger.setBindings({ ...logger.bindings(), newBindingProp: "string" })


feat: replace grpc-js with grpc-web for clients

Based on some experimentation, it seems like hubs can support both grpc and grpc-web if we host a proxy to translate between the grpc-web HTTP and grpc HTTP/2 calls. That would allow other hubs or backends to communicate with the hubs over grpc and also let devs build client-only apps that use grpc-web. So this issue is not actually replacing grpc-js with grpc-web, we just want to add support for grpc-web. Going to close this issue in favor of the other linked issue https://github.com/farcasterxyz/hubble/issues/573.


ES Module support?

Hello,
I am having difficulty setting up function app as modules is this something that can be configured to support it?
I've modified the tsconfig.json and tsconfig.build.json to have:
"module": "ES2022",
"target": "ES2022",
"moduleResolution": "node",

and included type: "module" in package.json
When I run it this is the error I get:
 Worker was unable to load function my-new-func: 'require is not defined in ES module scope, you can use import instead
Regards

Sorry for being quiet @penleychan, I was kind of busy lately.
Out of curiosity, why do you need ES Module support?

No problem, majority of our library were written in ES Module. I was kinda able to get it working with modifying your nxazure/func project with combination of https://github.com/LeDDGroup/typescript-transform-paths and ts-patch Only caveat is that I need to provide the custom @transform-path for typescript-transform-paths plugin to output correct path. Bit unintuitive but works for now. Hopefully better solution in the future.

Nice! Thanks for sharing!
If you ever get the time and the will, a PR with that solution would be very welcome!

If and when you get the time @penleychan , feel free to reopen the issue to discuss this again.


Remove deprecated methods from tests

Thanks @MatthewMckee4 for pointing this out
Summary by Sourcery
Remove deprecated methods and update test assertions to improve code quality and maintainability.
Bug Fixes:

Correct the test assertions to use 'insts' instead of 'references' for component instances.

Enhancements:

Remove deprecated method calls and related warnings from the codebase.

Nice work @MatthewMckee4
I think we got all the rid of all the warnings now :tada:


Quickbooks Online Linked Service - Need more information on how refresh tokens are handled

We are using the Quickbooks Online Linked Service, described here: https://learn.microsoft.com/en-us/azure/data-factory/connector-quickbooks?tabs=data-factory
The copy data service doesn't ask for Access Tokens. It must therefore be using the refresh token to request a new access token.
When you refresh the access token, QBO will periodically update the refresh token, invalidating the previous refresh token. See: https://developer.intuit.com/app/developer/qbo/docs/develop/authentication-and-authorization/faq

It is unclear from the documentation how the Linked Service handles the new refresh token. If you are using a secret from the Azure Key Vault, does it update the value to the latest refresh token?
If it does nothing with the new refresh token -- not even returning it as an output, this is a bug.
If it updates the refresh token value, the documentation needs to be updated to explain this.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 998a28b0-5278-73e2-7e27-b01997807797
Version Independent ID: c7a902d0-b098-0e3d-ef13-3f5f3082382d
Content: Copy data from QuickBooks Online (Preview) - Azure Data Factory & Azure Synapse
Content Source: articles/data-factory/connector-quickbooks.md
Service: data-factory
Sub-service: data-movement
GitHub Login: @jianleishen
Microsoft Alias: jianleishen

Information on the page:
Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 998a28b0-5278-73e2-7e27-b01997807797
Version Independent ID: c7a902d0-b098-0e3d-ef13-3f5f3082382d
Content: Copy data from QuickBooks Online (Preview) - Azure Data Factory & Azure Synapse
Content Source: articles/data-factory/connector-quickbooks.md
Service: data-factory
Sub-service: data-movement
GitHub Login: @jianleishen
Microsoft Alias: jianleishen

@rgknight
Thanks for your feedback! We will investigate and update as appropriate.

@rgknight
Thank you for bringing this to our attention.
I've delegated this to content author, who will review it and offer their insightful opinions.

Any update on this?

Hi @rgknight, thanks for your dedication to our documentation. Unfortunately, at this time we have been unable to review your issue in a timely manner and we sincerely apologize for the delayed response. The requested updates have not been made since the creation of this issue, and the timeline for resolution may vary based on resourcing, so we've created an internal work item to incorporate your suggestions. We are closing this issue for now, but feel free to comment here as necessary. #please-close


[FLINK-23209] Introduce HeartbeatListener.notifyTargetUnreachable

With this PR Flink's HeartbeatServices listen to the result of the heartbeat rpcs. If a rpc fails with RecipientUnreachableException, then it will fail the heartbeat and call HeartbeatListener.notifyTargetUnreachable. All Flink components (ResourceManager, JobMaster and TaskExecutor) will treat this signal similar to a heartbeat timeout. This means that connection failures on the TCP layer will now speed up the detection of lost components.
The basic change happens in the HeartbeatMangerImpl and HeartbeatManagerSenderImpl where we listen on the result future of HeartbeatTarget.receiveHeartbeat and .requestHeartbeat. This is necessary in order to any RPC related failures to the HeartbeatManager so that we can react on a RecipientUnreachableException and trigger HeartbeatListener.notifyTargetUnreachable.
The PR adds tests for the main components in JobMasterTest, TaskExecutorTest and ResourceManagerTest. Moreover, there are tests for the HeartbeatManager behaviour itself in HeartbeatManagerTest.
This PR is based on #16342.

One big question is how we want to treat the unreachable signal. The problem is that on unstable network environments where the network connection often breaks down, listening to the unreachable signal will lead to more cancellation and re-deployments of tasks. On the other hand, it speeds up the detection of dead TaskExecutors significantly.
One idea could be to make it configurable and to also define a threshold for when to call HeartbeatListener.notifyTargetUnreachable. The main question imo is how long these intermittent networking problems exist and at what point one rather wants to restart the job than to wait on the existing TaskManager's to reconnect.

I'm not sure if we really want to introduce another option that does similar things as the akka.ask.timeout; it will make it harder to understand in which circumstances some event will cause a failure.
Currently, if a TM goes down the heartbeat mechanism needs between heartbeatTimeout to heartbeatTimeout + heartbeatInterval time to detect it; because it might go down right after a heartbeat was acknowledged.
How about we use this new mechanism to reduce this delta instead? We start the timeout clock not only when the heartbeat request is sent, but also if an RPC message cannot be delivered. Similarly, we can also stop the timeout clock if an RPC message got through, independent of what the heartbeats do.
IOW, any RPC message would be treated like a heartbeat request, and heartbeats are just a way to ensure periodic communication.

By default the heartbeat timeout is 50s while interval is 10s, so there may not be a significant improvement only to reduce the interval.
But I find that the default akka.ask.timeout is 10s. Considering that the heartbeat is also sent via akka, I'm not sure whether the heartbeat.timeout is working.

By default the heartbeat timeout is 50s while interval is 10s, so there may not be a significant improvement only to reduce the interval.

Well one needs to ask themselves why it is that the timeout is multiple times the interval.
If the timeout is that large because a target should truly only be considered unreachable if nothing got through during this entire period, then in any case both mechanism will work the same way (<= because users configure it that way).
Beyond that, it is mostly for reliability isn't it; wouldn't want to treat a TM as unreachable on the off-chance that these exact 2 messages got lost in the network. But if other messages also function as heartbeats, then a user has more leeway to reduce the timeout, increasing detection speed without compromising reliability.

Considering that the heartbeat is also sent via akka, I'm not sure whether the heartbeat.timeout is working.

The heartbeat timeout isn't applied per-message, essentially when you send a heartbeat request you start a timer for the heartbeat timeout. If it comes back in time you stop the timer. If another heartbeat request is triggered and a timer is already running, then you don't reset it.
IOW; the heartbeat timeout describes how much time must pass during which none of the heartbeats are acknowledged for the target to be considered unreachable.

Well one needs to ask themselves why it is that the timeout is multiple times the interval.
If the timeout is that large because a target should truly only be considered unreachable if nothing got through during this entire period, then in any case both mechanism will work the same way (<= because users configure it that way).

In fact I saw that in HeartbeatManagerOptions, defined as defaultValue though the method is deprecated, am I at the right place?

IOW, any RPC message would be treated like a heartbeat request, and heartbeats are just a way to ensure periodic communication.

There's a counter example that, assuming the heartbeat interval is 10s and timeout is 5s, when JM send a RPC request(not heartbeat) that costs more 5s to process in the remote TM, and there's no response of other RPC requests or heartbeat requests received during the 5s, the request will result in heartbeat timeout, though there's nothing wrong.
The main difference is that a heartbeat request is expected to be responded as soon as possible, while a RPC request may take some time before responded.

Till and I have discussed a little in the mail list on the topic, and I'd like to share a point I came up there.

I was just considering that, since the environment is shared by JM and TMs, and the connections among TMs (using netty) are flaky in unstable environments, which will also cause the job failure, is it necessary to build a strongly guaranteed connection between JM and TMs, or it could be as flaky as the connections among TMs? As far as I know, connections among TMs will just fail on their first connection loss, so behaving like this in JM just means "as flaky as connections among TMs".

And here's the reply from Till in the mail list.

One simple approach could be to make the number of failed heartbeat RPCs until a target is marked as unreachable configurable because what represents a good enough criterion in one user's environment might produce too many false-positives in somebody else's environment. Or even simpler, one could say that one can disable reacting to a failed heartbeat RPC as it is currently the case.

I think it's ok to introduce a new option to determine when JM can ensure a connection loss really means a TM loss. The option is in fact a part of heartbeat configuration, rather than akka configuration, and heartbeat options are all "expert" options, indicated by the document annotation. I suppose it's not a confusing option for experts. And of course we need to provide a proper default value for normal users.
We may need another interval option if we use number of failed heartbeat RPCs as the new option. Current heartbeat interval is too long for it.
Or maybe we can use another timeout option that we can keep retrying (with a non-configurable short delay) to connect to TM until the timeout. The main difference between it and current heartbeat timeout is that, current heartbeat timeout is for liveness probe, when we suppose the TM to be living, while the new timeout is for death probe, when we suppose the TM to be already dead, and we already have some evidences about this. So it can be shorter than heartbeat timeout, which means faster failure.

Thanks for sharing your thoughts @zentol and @pltbkd. I think this is a very important discussion to have. In general we do agree that we should try to speed up Flink's detection of failed/dead TaskManagers.
Currently, we only have the heartbeat mechanism for that. The current idea for the heartbeat mechanism is that the components send a heartbeat every heartbeat.interval ms. If one side does not receive a heartbeat for heartbeat.timeout ms, then it marks the other side as dead.
The way it is currently implemented is that every heartbeat signal schedules a new timeout event for in heartbeat.timeout ms and cancels the old one.
One idea for speeding the failure detection up is to decrease the default values for interval and timeout. Note that interval < timeout. However, it is currently unclear how low we can go because it strongly depends on the network environment stability of our users.
Another idea would be to use additional signals. One example could be a signal from the underlying resource manager to tell us that a TaskExecutor is dead (https://issues.apache.org/jira/browse/FLINK-23216). Another signal could be whether we could actually deliver our heartbeat to the target (information from the TCP stack whether there is a connection and whether the message was lost).
The connection/lost message signal is the one this PR tries to make use of. Note that we cannot solely rely on this signal because a remote TaskExecutor might block its main thread which makes it unresponsive but we could still deliver heartbeat requests to it (from the TCP perspective). The question is whether this signal is more predictive than others.
The connection/lost message signal can also generate false-positives because there could be a transient network split. This strongly depends on the actual user environment. Therefore, my suggestion would be to introduce a threshold for lost heartbeat messages before triggering a timeout/mark the TaskExecutor as dead.
Note that TaskExecutors will fail their tasks if there is a connection loss between TaskExecutors. Moreover, there is no retry for the lost connections. Hence, if the user runs Flink in an unstable environment, then she will sooner or later experience restarts because of the unstable network environment. Hence, one could argue that heartbeating could mark their targets as dead after a single lost heartbeat message because it won't produce more false-positives as Flink's data-layer will already produce.
Of course, we can also think about more complex models where all different signals are integrated to produce a confidence value and then for some value we say that the target is now "dead". Also we can think about different retry methods (e.g. changing the request interval based on the signal), but I think that simply having a threshold for lost heartbeat messages could already improve the situation for some expert users.

Therefore, my suggestion would be to introduce a threshold for lost heartbeat messages before triggering a timeout/mark the TaskExecutor as dead.

I'm curious as to how this threshold would be implemented considering that there is no reliable pattern for how many/often RPC messages are sent.
It seems like any approach where X failed messages in Y time would behave undesirable in the extreme cases, such as 1 RPC per Y, or hundreds of RPCs within Y/1000. I guess it would have to be something like "If all (but at least LOWER_THRESHOLD) messages in Y time could not be delivered then the target is considered unreachable."

Of course, we can also think about more complex models where all different signals are integrated to produce a confidence value and then for some value we say that the target is now "dead".

I think that would become too complex and difficult to understand to the user. (buy maybe I imagine something overly complicated)
However, I do like the idea of combining the information of both heartbeats and other mechanisms; something along the lines of "if 1 heartbeat was not answered within akka.ask.timeout and no RPC (with at least LOWER_THRESHOLD attempts) went through within the timeframe (heartbeat_start -> heartbeat_timeout), then we consider the target unreachable."; essentially we shortcut the heartbeat timeout, instead of having an entirely separate mechanism.

Hence, one could argue that heartbeating could mark their targets as dead after a single lost heartbeat message because it won't produce more false-positives as Flink's data-layer will already produce.

That only applies if there are actual data connections between task executors; it does not apply to jobs without shuffles, jobs that are currently being deployed (potentially crashing the TM in the process), nor cases where the JM/all TMs go down.

I prefer to keep the mechanism simple and easy to understand, though I also agree that some expert configurations may be needed in specific scenarios. In the first version, a simple mechanism is enough for most scenarios, and current behavior can be reserved as default or alternative.

I guess it would have to be something like "If all (but at least LOWER_THRESHOLD) messages in Y time could not be delivered then the target is considered unreachable."

I think this is a simple and clear rule, with a proper LOWER_THRESHOLD default value which users possibly don't need to modify. Time is the main concern and more environment-related, which is also approved by Liu in the mail list.

That only applies if there are actual data connections between task executors; it does not apply to jobs without shuffles, jobs that are currently being deployed (potentially crashing the TM in the process), nor cases where the JM/all TMs go down.

It indeed increases the instability if we make connections between JM and TM as weak as that between TMs, especially when no connection exists between TMs. But I suppose it's acceptable to have the same experience in a certain cluster with different type of jobs, no matter whether data connections exist.
Moreover, in a job without data connections, tasks in other TMs won't be blocked when one TM is lost, and the failover cost is lower. So faster detecting of TM lost is not as necessary as those jobs with data connections. Maybe we don't need to worry a lot for such jobs when we build the new mechanism since current one is possibly enough.
By the way, similar to k8s' liveness probe, maybe it's better to respond heartbeat requests asap rather than queue it with other RPC requests, which I prefer to use as readiness probe. In this way, responding of heartbeat requests in a certain time can be more predictable, therefore the result of timeout is more reliable, and timeout can possibly be decreased. Though the connection loss is still not a reliable signal.
But since I'm not experienced in akka, I can't tell if it's possible or easy to implement.

The idea would be to establish a reconnection mechanism between the TMs. This hasn't been done yet, though. The non-trivial part is probably that we need to introduce an ack-protocol and keep a window of sent events on the sender side in order to resend events in case of lost packages.

@tillrohrmann
This is quoted from your reply in the mail list. I'd like to make sure if this is a long term plan or is going to take actions right now. If it's in recent plan, the heartbeat mechanism may also need to be more reliable.

I think in the first version the threshold would solely be defined for the heartbeat rpcs (e.g. if the HeartbeatManager fails to send x heartbeat rpcs, then mark the target as dead). This entails that the time until marking someone as dead is threshold * heartbeat interval. I hope that this definition is easy enough to understand for our users.

That only applies if there are actual data connections between task executors; it does not apply to jobs without shuffles, jobs that are currently being deployed (potentially crashing the TM in the process), nor cases where the JM/all TMs go down.

Agreed that in some cases this might make the cluster a tad bit less stable but given that Flink is able to run jobs with shuffles, it seems that Flink runs often enough in a stable-enough environment these days. Hence, I would suggest to go with a default of 1 heartbeat rpc loss until marking a target as dead. If users start to complain, we can still disable it.
I would leave more complex models for future improvements.

By the way, similar to k8s' liveness probe, maybe it's better to respond heartbeat requests asap rather than queue it with other RPC requests, which I prefer to use as readiness probe. In this way, responding of heartbeat requests in a certain time can be more predictable, therefore the result of timeout is more reliable, and timeout can possibly be decreased. Though the connection loss is still not a reliable signal. But since I'm not experienced in akka, I can't tell if it's possible or easy to implement.

The problem is that the heartbeat is used to transport status information about the components. Since we require a certain order of rpcs, we cannot send the heartbeat signals easily from outside the main thread because it could lead to race conditions and outdated status information. One could try to use some logical clocks to synchronize the messages again, but this hasn't been tried yet.

This is quoted from your reply in the mail list. I'd like to make sure if this is a long term plan or is going to take actions right now. If it's in recent plan, the heartbeat mechanism may also need to be more reliable.

There are no immediate plans for adding this feature but it will eventually be required.
So long story short, I would be in favour of adding a threshold for lost heartbeat rpcs until a target is marked as dead. The default should be 1 and it is configurable by the user if it is too aggressive on their environment.  What do you think?

This entails that the time until marking someone as dead is threshold * heartbeat interval. I hope that this definition is easy enough to understand for our users.

yes, I think that is easy to understand. Restricting it to heartbeat RPCs also ties the whole story together; you have heartbeats as the one mechanism to detect dead machines, with the timeouts handling unresponsive TMs (i.e., those blocked by some long-running operation or running exceedingly slow), while the threshold handles TMs that are already dead.

Hence, I would suggest to go with a default of 1 heartbeat rpc loss until marking a target as dead.

We can try that out, but I am apprehensive about the default of 1 (purely because expecting every message to go through is like the thing everyone drills you not to do).

The problem is that the heartbeat is used to transport status information about the components. Since we require a certain order of rpcs, we cannot send the heartbeat signals easily from outside the main thread because it could lead to race conditions and outdated status information. One could try to use some logical clocks to synchronize the messages again, but this hasn't been tried yet.

We could also think of decoupling heartbeats and periodic status updates again. It would result in more RPCs, but the total amount of data may become less (depending on how low the previous heartbeat interval was).

This entails that the time until marking someone as dead is threshold * heartbeat interval. I hope that this definition is easy enough to understand for our users.

I overall agree it is a good enough plan for the first version. Maybe we should suggest user not to set the threshold to more than timeout/interval, which by default is 50s/10s=5, a relative small value. We could add the suggestion in the configuration description, if you think it's necessary as well.

We can try that out, but I am apprehensive about the default of 1 (purely because expecting every message to go through is like the thing everyone drills you not to do).

I agree with zentol and I'd like to suggest the default value to be 2, standing for discovering and confirming.

The problem is that the heartbeat is used to transport status information about the components. Since we require a certain order of rpcs, we cannot send the heartbeat signals easily from outside the main thread because it could lead to race conditions and outdated status information. One could try to use some logical clocks to synchronize the messages again, but this hasn't been tried yet.

I now understand why the heartbeats also need to be in order. IMO, there are some advantages to have a more reliable and frequent heartbeat mechanism, with which I think both interval and timeout could be decreased. But since I don't know many details here, I would leave this issue to experts.

Alright, then let's go with a default of 2. I will update this PR to include the threshold.
Maybe as a potential follow up we could think about decreasing the heartbeat timeout to 15s and the interval to 3s.

I've added another commit that adds the threshold functionality. Please take a look @zentol and @pltbkd.

Thanks a lot for the review @zentol. I've addressed your comments.


Outline buttons background remains blue after clicking on mobile

On a mobile page, click on an outline button. It gets a hover state applied to it and doesn't change.
I can reproduce this in chrome with mobile tools turned on also. I'm unsure if this is a browser bug, so I'm filing an issue.

Just wanted to follow up here with what we discussed last week. This hover behavior is standard browser behavior on mobile where hover effects stay until another item is clicked or the page is scrolled.
But for btn-outline elements with the selected class, this makes it difficult to know whether it is hovered or selected since they have the same styles.

.btn-outline
.btn-outline:hover
.btn-outline.selected
.btn-outline.selected:hover

Regular buttons have a different behavior, where the .selected vs. :hover styling is different:

.btn
.btn:hover
.btn.selected
.btn.selected:hover


fix: Claims Retrieval Issue in SqlIdentityHubStore Due to Missing pay…

fix: Claims Retrieval Issue in SqlIdentityHubStore Due to Missing payloadFormat #107
Closes #107

thanks for the fix, but please familiarize yourself with our pr etiquette, which states that raising PRs without associated issues is not allowed. After having raised an issue, it should go through triage before any work can happen.

LGTM, I'll approve when unit-test is added.

@konrad1adamczyk postgres tests are failing. can you see what's going on there?

yes I see it here, on my local machine it was passing, I will fix

checkstyle is failing due to a missing license header.

handled through #112


Create percolate_deltas experiment notebook

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
see 3 files with indirect coverage changes


Add vue-plyr to plugins section

I have just spent a bunch of time writing a VueJS plugin for plyr, and I would greatly appreciate it if you would include it in the plugins section of the README.
I have already made a PR for it at #769 so you could just merge that and it would be all fine.
Thanks very much :)

If you want to take a look at it before you add it you can find it here:
https://github.com/redxtech/vue-plyr

It's also on NPM as vue-plyr
https://www.npmjs.com/package/vue-plyr

Sure. Will do. Bear in mind that v3 is coming very soon (it's on NPM already) so will need to update when that goes to master?

Yeah, I'll make sure to do that. The reason why I'm not on the V3 beta already was because when I used import plyr from 'plyr I received a message saying that plyr wasn't found, and when using it in a Vue component it told me that setup() was not a function.

Ah that should be fixed now but setup has been removed in favour of a proper constructor...

Ah ok.
Would you say that v3 is ready for a plugin yet? I can easily update mine to support it.
The constructor has changed from let player = plyr.contructor(selector, options) to let player = new Plyr(selector, options) right ?

It's still in testing and the feedback has been a little slow really. I'll badger people about it. It's pretty close though I think.
Yeah that's the change for the player. Previously the setup function would always return an array and now it returns a single instance. Also some of the config options have changed but not sure if that would effect your wrapper.

Yeah, the config things shouldn't change anything at all. However, when I do this.player = new Plyr(document.getElementById('js-player')) I get no errors, this.player is set to the proper value, but the actual video/audio/etc element remains unchanged, even though the correct selector is being passe through.


Change name of "metaport" package to "zopen"

The "metaport" package has a name too similar to the "meta" repo, which serves a different purpose.  Perhaps the this should be renamed to "zopen" since that's the main purpose of the repo.

The zopen code is actually held inside the meta repo - with the testing for it inside this (metaport) repo.  It would probably make sense for zopen to sit within it's own repo these days [with the current infrastructure, that would be "zopenport"!], with meta being more for the project "stuff" like docs, website etc.  Changes to the doc shouldn't necessarily require an update to zopen but that still happens.  Might also make it easier to test.


Update static_files.asciidoc

Minor update clarifying the need of the priv directory on project's root.

Merged locally with tweaks. Thanks!


visio online plan 2 with ms office 2016 standard (msi)

will this combination work? what ms office suite will work with Visio online plan 2? or conversely, what Visio plan will work with MS Office 2016 standard (msi)? also, which skype for business version will work with Visio online plan 2?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d40c96f3-1c97-b538-ef82-ffbf0eef204d
Version Independent ID: 2d6883b5-1b5c-dc16-9999-17d0652bfa48
Content: Supported scenarios for installing different versions of Office, Project, and Visio on the same computer
Content Source: DeployOffice/install-different-office-visio-and-project-versions-on-the-same-computer.md
Service: o365-proplus-itpro
GitHub Login: @DHB-MSFT
Microsoft Alias: danbrown

Visio Online Plan 2 can be combined with Microsoft 365 Apps, Office 2019 or Office 2016 retail (no VL, no MSI).


PHP not recognized as a language anymore since 1.51.0 - *.php files are detected as plain text.

VSCode Version: 1.51.1
OS Version: MacOS 10.13.6

This happened since 1.51.0
Steps to Reproduce:

Start with extensions disabled (or don't, it doesn't matter)
*.php files are detected as plain text files and thus have no syntax coloring or intellisense.

This is my settings.json.
Adding {"files.associations": { "*.php": "PHP" }} doesn't help.
Adding PHP extensions like felixfbecker.php-intellisense doesn't help either.
The Select language mode menu doesn't display PHP so I guess it's not even recognized as a language anymore.

Does this issue occur when all extensions are disabled?: Yes
Full VSCode version:
Version: 1.51.1
Commit: e5a624b788d92b8d34d1392e4c4d9789406efe8f
Date: 2020-11-11T01:11:34.018Z (1 day ago)
Electron: 9.3.3
Chrome: 83.0.4103.122
Node.js: 12.14.1
V8: <IP_ADDRESS>-electron.0
OS: Darwin x64 17.7.0

I'm having the same issue with .tsx and .jsx files being displayed as plain text files.
Files that are still working properly with colour highlighting are
.js
.json
.svg
.env
.ts
.yaml
.md
.mdx
.ejs

Version: 1.51.1
Commit: e5a624b788d92b8d34d1392e4c4d9789406efe8f
Date: 2020-11-11T01:11:34.018Z
Electron: 9.3.3
Chrome: 83.0.4103.122
Node.js: 12.14.1
V8: <IP_ADDRESS>-electron.0
OS: Darwin x64 18.7.0

I'm after tracking my problem down to the TabNine Autocomplete extension

We've released a new version with a fix: https://github.com/codota/tabnine-vscode/pull/206
Sorry for anyone who is affected by this.

Ok, I reinstalled VSCode, after removing all extensions and just kept the entries at settings.json that I was still making use of, and the problem was solved.
PHP back again detected as a language.
PS: Hadn't installed or used tabnine ever.


More Badges: Activity, Code coverage, report card

Codecov Report

Merging #15 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #15   +/-   ##
=======================================
  Coverage   70.59%   70.59%
=======================================
  Files          51       51
  Lines        1826     1826
=======================================
  Hits         1289     1289
  Misses        401      401
  Partials      136      136

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0ad1a1a...40c73cf. Read the comment docs.

+1


read_csv does not support gziped csv's

It is common for large csv's to be compressed. Polars supports compressed csv's and autodecompresses them. Feature request maybe or maybe it was just turn off by mistake?

Can you please investigate? Could it also be a version issue?

Okay, on investigation it is a optional feature, well in polaris allot of things are optional it seems.
Just adding "decompress", to the the depencies section of [dependencies.polars] in explorer/native/explorer/Cargo.toml fixes it.  I made a local working version!

@ricpruss that's fantastic! Would you mind opening a PR with that addition? I'm sure others would benefit. :)


ADR seems to be bugged on one side

As seen in the screenshot, the ADR for the CTs aren't displayed, while for the Ts they are.
Only tested in a CS2 demo using the newest version of the hud.

could you share the demo file you're using?

demotestv1.zip
Here you go - this is the one where I've tested it with the screenshot, though I am certain that I've encountered the problem in a different demo too.

@29j2003 looks like the file you uploaded is empty, could you try again please? :D

https://leetify.com/app/match-details/844af22a-deaa-45ce-ba30-f49b8f8acdcd/overview It appears that GitHub removed the DEM-File, so I've just posted the leetify match where you can download it directly from if that is okay!


[BUG] The image list packing part of the build script should be architecture-aware

Describe the bug

We will soon publish ARM64 artifacts for the v1.3.0 release. The build pipeline includes an image list packing stage, which packs the image lists for the current and previous versions (ref). It isn't arch-aware yet, so the results (the packed image lists) will be the same for both AMD64 and ARM64 built artifacts.
The packed image lists inside the built ISO will be referenced in the final stage of Harvester upgrades to purge unneeded container images. If the image lists contain wrong image names, the container images cannot be removed.
In addition, future releases will publish the image lists with ARCH as the name suffix. We need to adapt this to prevent build failure.
To Reproduce
Steps to reproduce the behavior:

Build the v1.3.0-rc4 ISO image
Examine the built image, check the packed image list

Expected behavior

Fetch the correct image list according to the target architecture and pack them into the ISO image.
Support bundle

N/A
Environment

Harvester ISO version: v1.3.0-rc4
Underlying Infrastructure (e.g. Baremetal with Dell PowerEdge R630): not relevant

Additional context
N/A

milestone required.

thanks @lanfon72 , milestone added,  this one should be included in v1.4.0

Test PASS, both AMD64 and ARM64 arch has image_list_all.txt for v1.3.0, v1.3.1 and v1.3.2, close as fixed.
Environment

Harvester Image

harvester-v1.4.0-rc2-amd64.iso
harvester-v1.4.0-rc2-arm64.iso

Test Steps

 Mount Harvester AMD64 and ARM64 ISOs

 Check AMD64 v1.3.0, v1.3.1 and v1.3.2 image_list_all.txt exists

 Check ARM v1.3.0, v1.3.1 and v1.3.2 image_list_all.txt exists


Detox Build for Android Failing

Describe the bug
When I run detox build -c android, it is failing.
Environment (please complete the following information):

Detox: 14.3.4
React Native: 0.60.3
Node: 10.15.3
Device: 5.1 WVGA

Device and Verbose Detox Logs
error.txt

What is the project that you are building?

getting the same error

React Native 0.60 is still not 100% supported unfortunately, this includes compilation issues. We hope this will be resolved soon.

It is a private repo. Android worked before with RN 0.60.3 so I think something in a newer Detox is breaking or some dependencies changed. If I have the time I can try to make a basic reproducing repo but I assumed it was just something that needed to be changed/added in the gradle file.

I tried running it again and the build passed but the tests showed this error. When building it again after this, it fails with the same error in the original comment in the issue.
error.txt

What have you tried? Can you please be specific?
Did you try two different versions of detox on the same project (which?), did it compile on one and failed on the other?

I ran it with the same version of Detox, 14.3.4. I ran detox build, detox test, then detox build in that order with 2 minutes between each command after it finished running.

Closing in favor of #1689. Please refer to the versions support table for latest info on this.

@ReshmiCode you simply have a discrepancy among the required kotlin standard lib version through various deps. This can be resolved by either excluding kotlin from Detox's deps (example) or by forcing a specific kotlin version globally (stackoverflow).


NTR: venom hypersensitivty

New term: venom allergy
Synonym: IgE- mediated venom hypersensitivity, venom hypersensitivity
Parent: MONDO_0005271 'allergic disease'
Definition: A allergic disease involving venom.
Source: PMID:26154789
See https://github.com/obophenotype/human-phenotype-ontology/issues/3428
add chebi venom:
https://www.ebi.ac.uk/ols/ontologies/chebi/terms?iri=http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FCHEBI_78505

@nicolevasilevsky - could you put a difficulty label on this?

I agree with 'medium' effort- should take less than an hour

Thanks Nicole,
Kallia

On 5 May 2021, at 16:57, Nicole Vasilevsky @.***> wrote:
I agree with 'medium' effort- should take less than an hour
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub https://github.com/monarch-initiative/mondo/issues/158#issuecomment-832809982, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASPFECPSL3M2TQ3F4N5KO6DTMFTGBANCNFSM4E5UAFGQ.

Nicole is looking into importing 'venom' term from CHEBI by adding it to a text file, then we need to add Equivalent class definition.


Fix channel selection bug in Python 2 and Windows

Ready to merge @nippoo this one should not take too long to review


How to encode spaces?

Incredible tool, however, how to encode spaces when replacing strings?

@street-grease-coder If you check out the release version I have up, there's a charset.xml included already which includes spaces if that's what you're talking about. Other than that, it should not be any different than just replacing words. The only thing I would make sure of is to enclose the string in quotation marks when you pass it on the command line. So, instead of:
romtool.exe replace rom.gbc rom2.gbc Hello, world! This is a test
It should look like this:
romtool.exe replace rom.gbc rom2.gbc "Hello, world!" "This is a test"

Thanks for the response and for this awesome tool. I did check out the latest charset.xml, and saw the encoding for the whitespace, however it didn't work for me properly. I was trying to replace two words separated with whitespace in a Pokémon yellow rom. If it's extremely useful to you I can get around to create a screenshot =)


We were not handling the user option on create

With this patch we can get run_passwd_test.go to pass
Signed-off-by: Daniel J Walsh<EMAIL_ADDRESS>

@jwhonce @mheon @baude @vrothberg @giuseppe @TomSweeneyRedHat @QiWang19 PTAL

LGTM

LGTM

/lgtm


Integration Test - Release - 2019/03/18 01:57:05

Auto create for integration test. AnotherAuthorUpdatedArticleFlow

@PRMergerTest3 : Thanks for your contribution! The author, @v-caxian, has been notified to review your proposed change.


t3

x

From all of us on the Cloud Advocacy team at Microsoft, thank you @johnpapa for creating this issue!
Have a Happy Holiday season!
Don't forget to share your solution on Twitter with the tag #25DaysOfServerless

@all-contributors please add @johnpapa for code

@all-contributors please add @johnpapa for code

@all-contributors please add @johnpapa for code


Test CI

Here's the diff between the packages installed in CI here (where CI is failing) and the last time CI passed on master:
6c6
<   - aiobotocore=1.2.0=pyhd8ed1ab_0
---
>   - aiobotocore=1.2.0=pyhd3eb1b0_0
34,35c34,35
<   - boto3=1.16.52=pyhd8ed1ab_0
<   - botocore=1.19.52=pyhd8ed1ab_0
---
>   - boto3=1.16.54=pyhd8ed1ab_0
>   - botocore=1.19.55=pyhd8ed1ab_0
37c37
<   - brotlipy=0.7.0=py38h94c058a_1001
---
>   - brotlipy=0.7.0=py38h5406a74_1001
58c58
<   - cytoolz=0.11.0=py38h1130bcf_1
---
>   - cytoolz=0.11.0=py38h5406a74_2
68c68
<   - freetype=2.10.4=h3f75d11_0
---
>   - freetype=2.10.4=h4cff582_1
75c75
<   - grpc-cpp=1.34.0=h1671c54_0
---
>   - grpc-cpp=1.34.1=h1671c54_0
140c140
<   - matplotlib-base=3.3.3=py38h9bfd75d_0
---
>   - matplotlib-base=3.3.3=py38hdcc317e_0
241c241,243
<   - zstd=1.4.8=hf387650_1
\ No newline at end of file
---
>   - zstd=1.4.8=hf387650_1
>   - pip:
>     - pytest-timeout==1.4.2
\ No newline at end of file

Based on the tests that are failing, the package that stood out the most to me as potentially being related is cytoolz. Locally when I revert cytoolz to the conda package build where we last saw CI pass I was able to get some of the erroneous tests to pass (🎉).
Just pushed a commit up which removes cytoolz from CI (it's an optional dependency anyways) to see what happens.

Here's the diff between the packages installed in CI here (where CI is failing) and the last time CI passed on master:
6c6
<   - aiobotocore=1.2.0=pyhd8ed1ab_0
---
>   - aiobotocore=1.2.0=pyhd3eb1b0_0
34,35c34,35
<   - boto3=1.16.52=pyhd8ed1ab_0
<   - botocore=1.19.52=pyhd8ed1ab_0
---
>   - boto3=1.16.54=pyhd8ed1ab_0
>   - botocore=1.19.55=pyhd8ed1ab_0
37c37
<   - brotlipy=0.7.0=py38h94c058a_1001
---
>   - brotlipy=0.7.0=py38h5406a74_1001
58c58
<   - cytoolz=0.11.0=py38h1130bcf_1
---
>   - cytoolz=0.11.0=py38h5406a74_2
68c68
<   - freetype=2.10.4=h3f75d11_0
---
>   - freetype=2.10.4=h4cff582_1
75c75
<   - grpc-cpp=1.34.0=h1671c54_0
---
>   - grpc-cpp=1.34.1=h1671c54_0
140c140
<   - matplotlib-base=3.3.3=py38h9bfd75d_0
---
>   - matplotlib-base=3.3.3=py38hdcc317e_0
241c241,243
<   - zstd=1.4.8=hf387650_1
\ No newline at end of file
---
>   - zstd=1.4.8=hf387650_1
>   - pip:
>     - pytest-timeout==1.4.2
\ No newline at end of file

Based on the tests that are failing, the package that stood out the most to me as potentially being related is cytoolz. Locally when I revert cytoolz to the conda package build where we last saw CI pass I was able to get some of the erroneous tests to pass (🎉).
Just pushed a commit up which removes cytoolz from CI (it's an optional dependency anyways) to see what happens.

Uninstalling cytoolz in the CI environment did the trick. I've opened up https://github.com/conda-forge/cytoolz-feedstock/issues/36 to report the issue. For now I think it's safe to just not include cytoolz in the CI environment until the issue with the conda-forge build has been fixed.

Uninstalling cytoolz in the CI environment did the trick. I've opened up https://github.com/conda-forge/cytoolz-feedstock/issues/36 to report the issue. For now I think it's safe to just not include cytoolz in the CI environment until the issue with the conda-forge build has been fixed.


Crystal testimonial

Better to have two different testimonials on the free trial and demo page.
Preview: https://deploy-preview-794--roadie.netlify.app/request-demo/

https://app.shortcut.com/larder/story/9752/marketing-site-improvements

@jorgelainfiesta I tried a white logo too and it didn't look any better unfortunately.


All embedded apps submitted for app review required to adopt session tokens

This notification showed up in the Shopify Partner Dashboard a couple of days back. I love this project and I'm already building my next app using it as boilerplate but wonder if this new policy will be an issue? As far as I can see cookies are being used in at least one place.

It'll be a issue, they are deprecating the old cookie session method.
I'll update the repo in the coming weeks with the new update, if I don't receive by then a pr from a person ^^'
Most of the changement will be done in this repo: https://github.com/bluebeel/nextjs-shopify-auth

That's great to hear! I'm unsure if I'll have time to look into it before you. I will notify you if I do however!

Hi guys, are you currently working on it? I may also help if needed :)

Hello,
First of all, thank you for the enthusiasm you have.
As a first track I was thinking of starting again from the example offered by Shopify with Koa and "reverse engineer" the lib like the first time.
It has been updated and contains the new token authentication and the lib uses the new shopify node api.
After the question would be to know if we are still obliged to use SSR app or with the new mode of authentication, we could move towards full static app.
BTW there is an open discussion for an official example for nextjs with the new token https://github.com/Shopify/shopify-app-bridge/issues/13
Otherwise @ctrlaltdylan did a great job creating a boilerplate example. You can start from this one.

Thanks for the reply!
I gave it a go - it' been quite frustrating - especially as using ngrok takes 5 minute to see a change 😅
I took some inspiration from both this and the boilerplate above - the issue with that is that it's a basic implementation (for example, the nonce is not checked), but it's definitely a start

Thanks for the reply!
I gave it a go - it' been quite frustrating - especially as using ngrok takes 5 minute to see a change 😅
I took some inspiration from both this and the boilerplate above - the issue with that is that it's a basic implementation (for example, the nonce is not checked), but it's definitely a start

You can make a pr so we can see your work and maybe help you?

Thanks for the mention @bluebeel, right it's just a basic prototype. But nonce checking & tests are upcoming. I've been using the package in some form since October on a few production apps. Others as well.
In a perfect world, Shopify would release another version of their official tutorial & auth repos for Next without Koa.js. I've been talking with them on and off about making that switch. It's possible they might do something in the coming months, but not optimistic it will be anytime soon.

Is this still an issue? Looking at this boilerplate as a starting point. Thanks!

Would also love to know if this has been resolved


ebiten: An API to skip Draw when Update is not called

From the discussion at #1520 and  #1595, we might want an API to skip Draw calls when Update is not called (especially when vsync is off). In most games, the state should be updated only in Update and then Draw doesn't have to be called when Update is not called. In this case, the last offscreen result can be reused.
As the side effect of skipping, FPS will no longer represent the number of Draw calls per second.

https://github.com/hajimehoshi/ebiten/discussions/2061
There are some confusions in Update and Draw. I'd like to confirm the current implementation of Update/Draw is good or not...


Tests fail to compile

FAILED: examples/mpi/CMakeFiles/test_mpi.dir/mpi.cpp.o
/usr/bin/c++  -I/usr/ports/devel/doctest/work/doctest-2.4.11/scripts/development_only -I/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/parts -I/usr/ports/devel/doctest/work/doctest-2.4.11 -isystem /usr/local/include -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -Werror -fstrict-aliasing -pedantic -pedantic-errors -fvisibility=hidden -Weverything -Wno-c++98-compat -Wno-c++98-compat-pedantic -Wno-c++98-compat-bind-to-temporary-copy -Wno-c++98-compat-local-type-template-args -Qunused-arguments -fcolor-diagnostics -O2 -pipe -fstack-protector-strong -fno-strict-aliasing   -DNDEBUG -std=gnu++11 -MD -MT examples/mpi/CMakeFiles/test_mpi.dir/mpi.cpp.o -MF examples/mpi/CMakeFiles/test_mpi.dir/mpi.cpp.o.d -o examples/mpi/CMakeFiles/test_mpi.dir/mpi.cpp.o -c /usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/mpi.cpp
In file included from /usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/mpi.cpp:1:
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:113:4: error: empty expression statement has no effect; remove unnecessary ';' to silence this warning [-Werror,-Wextra-semi-stmt]
  };
   ^
/usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/mpi.cpp:23:22: error: 'test_rank' was marked unused but was used [-Werror,-Wused-but-marked-unused]
  int x = f_for_test(test_rank);
                     ^
/usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/mpi.cpp:25:3: error: 'test_nb_procs_as_int_constant' was marked unused but was used [-Werror,-Wused-but-marked-unused]
  MPI_CHECK( 0,  x==10 ); // CHECK for rank 0, that x==10
  ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:157:28: note: expanded from macro 'MPI_CHECK'
#define MPI_CHECK          DOCTEST_MPI_CHECK
                           ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:134:47: note: expanded from macro 'DOCTEST_MPI_CHECK'
#define DOCTEST_MPI_CHECK(rank_to_test, ...)  DOCTEST_MPI_GEN_ASSERTION(rank_to_test,DOCTEST_CHECK,__VA_ARGS__)
                                              ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:130:30: note: expanded from macro 'DOCTEST_MPI_GEN_ASSERTION'
  static_assert(rank_to_test<test_nb_procs_as_int_constant.value,"Trying to assert on a rank greater than the number of procs of the test!"); \
                             ^
/usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/mpi.cpp:25:3: error: 'test_rank' was marked unused but was used [-Werror,-Wused-but-marked-unused]
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:157:28: note: expanded from macro 'MPI_CHECK'
#define MPI_CHECK          DOCTEST_MPI_CHECK
                           ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:134:47: note: expanded from macro 'DOCTEST_MPI_CHECK'
#define DOCTEST_MPI_CHECK(rank_to_test, ...)  DOCTEST_MPI_GEN_ASSERTION(rank_to_test,DOCTEST_CHECK,__VA_ARGS__)
                                              ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:131:22: note: expanded from macro 'DOCTEST_MPI_GEN_ASSERTION'
  if(rank_to_test == test_rank) assertion(__VA_ARGS__)
                     ^
/usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/mpi.cpp:26:3: error: 'test_nb_procs_as_int_constant' was marked unused but was used [-Werror,-Wused-but-marked-unused]
  MPI_CHECK( 1,  x==11 ); // CHECK for rank 1, that x==11
  ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:157:28: note: expanded from macro 'MPI_CHECK'
#define MPI_CHECK          DOCTEST_MPI_CHECK
                           ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:134:47: note: expanded from macro 'DOCTEST_MPI_CHECK'
#define DOCTEST_MPI_CHECK(rank_to_test, ...)  DOCTEST_MPI_GEN_ASSERTION(rank_to_test,DOCTEST_CHECK,__VA_ARGS__)
                                              ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:130:30: note: expanded from macro 'DOCTEST_MPI_GEN_ASSERTION'
  static_assert(rank_to_test<test_nb_procs_as_int_constant.value,"Trying to assert on a rank greater than the number of procs of the test!"); \
                             ^
/usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/mpi.cpp:26:3: error: 'test_rank' was marked unused but was used [-Werror,-Wused-but-marked-unused]
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:157:28: note: expanded from macro 'MPI_CHECK'
#define MPI_CHECK          DOCTEST_MPI_CHECK
                           ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:134:47: note: expanded from macro 'DOCTEST_MPI_CHECK'
#define DOCTEST_MPI_CHECK(rank_to_test, ...)  DOCTEST_MPI_GEN_ASSERTION(rank_to_test,DOCTEST_CHECK,__VA_ARGS__)
                                              ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:131:22: note: expanded from macro 'DOCTEST_MPI_GEN_ASSERTION'
  if(rank_to_test == test_rank) assertion(__VA_ARGS__)
                     ^
/usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/mpi.cpp:31:22: error: 'test_rank' was marked unused but was used [-Werror,-Wused-but-marked-unused]
  int x = f_for_test(test_rank);
                     ^
/usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/mpi.cpp:33:3: error: 'test_nb_procs_as_int_constant' was marked unused but was used [-Werror,-Wused-but-marked-unused]
  MPI_CHECK( 0,  x==10 ); // CHECK for rank 0, that x==10
  ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:157:28: note: expanded from macro 'MPI_CHECK'
#define MPI_CHECK          DOCTEST_MPI_CHECK
                           ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:134:47: note: expanded from macro 'DOCTEST_MPI_CHECK'
#define DOCTEST_MPI_CHECK(rank_to_test, ...)  DOCTEST_MPI_GEN_ASSERTION(rank_to_test,DOCTEST_CHECK,__VA_ARGS__)
                                              ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:130:30: note: expanded from macro 'DOCTEST_MPI_GEN_ASSERTION'
  static_assert(rank_to_test<test_nb_procs_as_int_constant.value,"Trying to assert on a rank greater than the number of procs of the test!"); \
                             ^
/usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/mpi.cpp:33:3: error: 'test_rank' was marked unused but was used [-Werror,-Wused-but-marked-unused]
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:157:28: note: expanded from macro 'MPI_CHECK'
#define MPI_CHECK          DOCTEST_MPI_CHECK
                           ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:134:47: note: expanded from macro 'DOCTEST_MPI_CHECK'
#define DOCTEST_MPI_CHECK(rank_to_test, ...)  DOCTEST_MPI_GEN_ASSERTION(rank_to_test,DOCTEST_CHECK,__VA_ARGS__)
                                              ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:131:22: note: expanded from macro 'DOCTEST_MPI_GEN_ASSERTION'
  if(rank_to_test == test_rank) assertion(__VA_ARGS__)
                     ^
/usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/mpi.cpp:34:3: error: 'test_nb_procs_as_int_constant' was marked unused but was used [-Werror,-Wused-but-marked-unused]
  MPI_CHECK( 1,  x==11 ); // CHECK for rank 1, that x==11
  ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:157:28: note: expanded from macro 'MPI_CHECK'
#define MPI_CHECK          DOCTEST_MPI_CHECK
                           ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:134:47: note: expanded from macro 'DOCTEST_MPI_CHECK'
#define DOCTEST_MPI_CHECK(rank_to_test, ...)  DOCTEST_MPI_GEN_ASSERTION(rank_to_test,DOCTEST_CHECK,__VA_ARGS__)
                                              ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:130:30: note: expanded from macro 'DOCTEST_MPI_GEN_ASSERTION'
  static_assert(rank_to_test<test_nb_procs_as_int_constant.value,"Trying to assert on a rank greater than the number of procs of the test!"); \
                             ^
/usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/mpi.cpp:34:3: error: 'test_rank' was marked unused but was used [-Werror,-Wused-but-marked-unused]
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:157:28: note: expanded from macro 'MPI_CHECK'
#define MPI_CHECK          DOCTEST_MPI_CHECK
                           ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:134:47: note: expanded from macro 'DOCTEST_MPI_CHECK'
#define DOCTEST_MPI_CHECK(rank_to_test, ...)  DOCTEST_MPI_GEN_ASSERTION(rank_to_test,DOCTEST_CHECK,__VA_ARGS__)
                                              ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:131:22: note: expanded from macro 'DOCTEST_MPI_GEN_ASSERTION'
  if(rank_to_test == test_rank) assertion(__VA_ARGS__)
                     ^
/usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/mpi.cpp:35:3: error: 'test_nb_procs_as_int_constant' was marked unused but was used [-Werror,-Wused-but-marked-unused]
  MPI_CHECK( 2,  x==-1 ); // CHECK for rank 2, that x==-1 (which is not the case -> will trigger a failure report)
  ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:157:28: note: expanded from macro 'MPI_CHECK'
#define MPI_CHECK          DOCTEST_MPI_CHECK
                           ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:134:47: note: expanded from macro 'DOCTEST_MPI_CHECK'
#define DOCTEST_MPI_CHECK(rank_to_test, ...)  DOCTEST_MPI_GEN_ASSERTION(rank_to_test,DOCTEST_CHECK,__VA_ARGS__)
                                              ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:130:30: note: expanded from macro 'DOCTEST_MPI_GEN_ASSERTION'
  static_assert(rank_to_test<test_nb_procs_as_int_constant.value,"Trying to assert on a rank greater than the number of procs of the test!"); \
                             ^
/usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/mpi.cpp:35:3: error: 'test_rank' was marked unused but was used [-Werror,-Wused-but-marked-unused]
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:157:28: note: expanded from macro 'MPI_CHECK'
#define MPI_CHECK          DOCTEST_MPI_CHECK
                           ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:134:47: note: expanded from macro 'DOCTEST_MPI_CHECK'
#define DOCTEST_MPI_CHECK(rank_to_test, ...)  DOCTEST_MPI_GEN_ASSERTION(rank_to_test,DOCTEST_CHECK,__VA_ARGS__)
                                              ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:131:22: note: expanded from macro 'DOCTEST_MPI_GEN_ASSERTION'
  if(rank_to_test == test_rank) assertion(__VA_ARGS__)
                     ^
/usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/mpi.cpp:41:12: error: 'test_nb_procs' was marked unused but was used [-Werror,-Wused-but-marked-unused]
    CHECK( test_nb_procs == 2 );
           ^
/usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/mpi.cpp:44:12: error: 'test_rank' was marked unused but was used [-Werror,-Wused-but-marked-unused]
    CHECK( test_rank == 0 ); // should fail on proc 1
           ^
15 errors generated.
[ 84% 101/113] /usr/bin/c++  -I/usr/ports/devel/doctest/work/doctest-2.4.11/scripts/development_only -I/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/parts -I/usr/ports/devel/doctest/work/doctest-2.4.11 -isystem /usr/local/include -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -Werror -fstrict-aliasing -pedantic -pedantic-errors -fvisibility=hidden -Weverything -Wno-c++98-compat -Wno-c++98-compat-pedantic -Wno-c++98-compat-bind-to-temporary-copy -Wno-c++98-compat-local-type-template-args -Qunused-arguments -fcolor-diagnostics -O2 -pipe -fstack-protector-strong -fno-strict-aliasing   -DNDEBUG -std=gnu++11 -MD -MT examples/mpi/CMakeFiles/test_mpi.dir/main.cpp.o -MF examples/mpi/CMakeFiles/test_mpi.dir/main.cpp.o.d -o examples/mpi/CMakeFiles/test_mpi.dir/main.cpp.o -c /usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/main.cpp
FAILED: examples/mpi/CMakeFiles/test_mpi.dir/main.cpp.o
/usr/bin/c++  -I/usr/ports/devel/doctest/work/doctest-2.4.11/scripts/development_only -I/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/parts -I/usr/ports/devel/doctest/work/doctest-2.4.11 -isystem /usr/local/include -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -Werror -fstrict-aliasing -pedantic -pedantic-errors -fvisibility=hidden -Weverything -Wno-c++98-compat -Wno-c++98-compat-pedantic -Wno-c++98-compat-bind-to-temporary-copy -Wno-c++98-compat-local-type-template-args -Qunused-arguments -fcolor-diagnostics -O2 -pipe -fstack-protector-strong -fno-strict-aliasing   -DNDEBUG -std=gnu++11 -MD -MT examples/mpi/CMakeFiles/test_mpi.dir/main.cpp.o -MF examples/mpi/CMakeFiles/test_mpi.dir/main.cpp.o.d -o examples/mpi/CMakeFiles/test_mpi.dir/main.cpp.o -c /usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/main.cpp
In file included from /usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/main.cpp:3:
In file included from /usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:7:
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/mpi_reporter.h:22:7: error: 'overflow' overrides a member function but is not marked 'override' [-Werror,-Wsuggest-override]
  int overflow(int c) { return c; }
      ^
/usr/include/c++/v1/streambuf:294:22: note: overridden virtual function is here
    virtual int_type overflow(int_type __c = traits_type::eof());
                     ^
In file included from /usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/main.cpp:3:
In file included from /usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:7:
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/mpi_reporter.h:24:7: error: padding size of 'doctest::(anonymous namespace)::NullStream' with 4 bytes to alignment boundary [-Werror,-Wpadded]
class NullStream : public std::ostream {
      ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/mpi_reporter.h:32:19: error: declaration requires an exit-time destructor [-Werror,-Wexit-time-destructors]
static NullStream nullStream;
                  ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/mpi_reporter.h:32:19: error: declaration requires a global destructor [-Werror,-Wglobal-constructors]
In file included from /usr/ports/devel/doctest/work/doctest-2.4.11/examples/mpi/main.cpp:3:
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:16:38: error: no previous extern declaration for non-static variable 'sub_comms_by_size' [-Werror,-Wmissing-variable-declarations]
std::unordered_map<int,mpi_sub_comm> sub_comms_by_size;
                                     ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:16:1: note: declare 'static' if the variable is not intended to be used outside of this translation unit
std::unordered_map<int,mpi_sub_comm> sub_comms_by_size;
^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:16:38: error: declaration requires an exit-time destructor [-Werror,-Wexit-time-destructors]
std::unordered_map<int,mpi_sub_comm> sub_comms_by_size;
                                     ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:16:38: error: declaration requires a global destructor [-Werror,-Wglobal-constructors]
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:45:5: error: no previous extern declaration for non-static variable 'world_size_before_init' [-Werror,-Wmissing-variable-declarations]
int world_size_before_init = mpi_comm_world_size();
    ^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:45:1: note: declare 'static' if the variable is not intended to be used outside of this translation unit
int world_size_before_init = mpi_comm_world_size();
^
/usr/ports/devel/doctest/work/doctest-2.4.11/doctest/extensions/doctest_mpi.h:45:5: error: declaration requires a global constructor [-Werror,-Wglobal-constructors]
int world_size_before_init = mpi_comm_world_size();
    ^                        ~~~~~~~~~~~~~~~~~~~~~
9 errors generated.

Version: 2.4.11
clang-14
FreeBSD 13.1

Please specify command line instructions used for repro. Thanks!


Embedded structs unable to resolve ambiguous names

looks similar to issue #162. If they're super similar, sorry for the noise.
Consider this example:
type A struct {
  Id int64 `db:"id"`
  Name string `db:"name"`
}

type B struct {
 OwnId int64 `db:"id"`
 AId int64 `db:"a_id"`
 Val string `db:"val"`
}
type C struct {
 OwnId int64 `db:"id"`
 AId int64 `db:"a_id"`
 OtherVal string `db:"other_val"`
}

embedded := struct{
 A, B, C
}{}
dbmap.SelectOne(&embedded, "select * from A a inner join B b on a.id = b.a_id inner join C c on a.id = c.a_id where c.id = ? limit 1", someAid); err != nil {
....

this will result in the following error:
gorp: No fields [ a_id a_id] in type

I'm guessing it's hard (impossible?) to resolve these names without explicitly stating them in the select query? I suppose this functionality is not a deal breaker for most people, but it would be nice to be able to avoid being overly verbose in my select (probably the correct thing to do) or having to do 3 explicit selects. I don't know if my gripe will actually get resolved, but the  in type _____  part of the error message should at least get fixed :)

I'd recommend writing a little helper to read the column names using dbMap.TableFor and parsing the columns, as a replacement for select *.  We should maybe add functionality for exactly that purpose.
As to the error: yeah, that should definitely be fixed.


Need filter when choosing what to download

Describe the feature you'd like
Add a file filter when add a torrent and choose what to download
Describe a visual implementation

Additional context
It cost me tons of time to just unselect those .zip file in my torrent.....That is painful

HI, thank you for your suggestion. I'll try to add it in the next version, this could be useful.

Feature added in v1.8.0.


[epub] Unable to load most of elements

only a few elements will be displayed
I want to render a novel, but i found that only menu and the first picture of each character will be displayed
To Reproduce
Steps to reproduce the behavior:

use example code to open epub file

Expected behavior
file:
test0.zip
expected:
there should be full color cover, menu, each chapter starts with a picture, and body follows

but body of each chapter won't be rendered:

device
i've tested on win11 and  android, both of them won't work correctly

maybe i found where the problem is
these data is in   bookContent.Html instead of book.Chapters, maybe you only render the data in book.Chapters


Basic File Browser

For opening local HDF5 / RAW files

Could additionally accept Drag&Drop events - https://developer.mozilla.org/en-US/docs/Web/API/HTML_Drag_and_Drop_API/File_drag_and_drop

Drag and drop sounds awesome! We should make sure that the entire backend is local and display a helpful message if it is not. I'm quite sure someone will try this with a remote cluster...

Yeah, otherwise it could offer to upload & ingest the dropped file, though I don't know how efficient it is to do the upload via the browser. Could actually work...

Good idea to offer upload! With a confirmation step because it might not be what the user wants to do, for example if the same network file system or cloud folder is mounted on both client and server.On Jul 9, 2018 16:48, Alexander Clausen<EMAIL_ADDRESS>wrote:Yeah, otherwise it could offer to upload & ingest the dropped file, though I don't know how efficient it is to do the upload via the browser. Could actually work...
—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or mute the thread.

Yeah, also the binary formats may need some parameters to be read...
So the flow would be:

Drop file
Get a parameters dialog
If the file is remote, start uploading (with an indicator somewhere on the screen - normal work can continue)
If the file is not remote, or on completion of upload, open file as Dataset

The "normal" (non-DnD) flow should be the same actually - you first select a file (from local file system or from HDFS), then you get the parameter dialog, and then it is opened. This enables us to make some things more user friendly, for example we could pre-fill some form fields etc. For EMPAD we can at least guess the scan dimensions from the filesize, and provide information for form input validation, which is a bit awkward if the file name is also part of the same form.
I'll create a separate ticket for uploading via DnD - I think it fits for the 0.2 milestone, do you agree?

Actually DnD for direct reading (without 'uploading') will be limited, as the fullPath property is not universally supported.

For the client, we should use react-window to render the list of files, to support directories with many files.
Need to think about how to properly open MIB files - idea: instead of specifying a glob pattern, just select the *.hdr file, use its 'basename' to construct a glob: default.hdr -> default*.mib

There are some security considerations concerning the file browser (or even without the file browser: loading from arbitraty paths) and multi-user environments - LiberTEM always has access to all files of the user it is running as - including sensitive information like passwords or SSH keys, and other users can access that LiberTEM instance.
We should think about this a bit more, and at least introduce a cmdline switch to whitelist access to certain paths, where the user expects to load data from, or generate a token which the user has to include when he first accesses the UI.

Hm, how does, for example, Nextcloud handle authentication? Is there a "best practice" for a system like LiberTEM that gives a web-based interface for local files? Can we assume that LiberTEM is always running as the Unix user that belongs to the "web user" or might there be web users that don't have a local Unix account?I'm just a bit worried because in particular on a cluster it gets messy IMO if we can't rely on the operating system to manage access. We should probably not roll our own solution but rely on good third party code. Is there a code base we could use?

I'm new to these LiberTEM discussions, but want to suggest checking out Globus authentication.  It provides access to SSO for academic and lab users as well as OpenID (Google authentication), and InCommon.  It would also make it easy to tap into a growing materials data infrastructure where Globus is the de facto standard for large file transfer and offers storage solution.  There are lots of resources to learn more, but one place to start would be the Globus docs: https://docs.globus.org/authorization-authentication-guide/

@davidelbert thank you for joining the discussion! Globus looks interesting, from a quick glance it looks similar to the B2ACCESS/B2SHARE/B2DROP services provided by EUDAT, right? I'd love to learn more about that, do you have first-hand experience using this system?

"disappearing files" bug filed upstream: https://github.com/bvaughn/react-window/issues/45

4ca7310 fixed the scrolling issue for me. Thx @sk1p !


Don't use findTreeItem in AzExtTreeFileSystem

We haven't gotten findTreeItem to work for v2 yet since tree item IDs haven't been ironed out. However, I'm proposing that we move away from using findTreeItem in AzExtTreeFileSystem.  AzExtTreeFileSystem is one of the main places where findTreeItem is used.
vscode.FileSystemProvider relies on handling Uris within VS Code. Within the file system provider methods we need access to the tree item in order to read/write the file content. Currently, we're adding the tree item ID to the Uri query params, and then using findTreeItem to retrieve the tree item within the handler.
Although that works great, I found it sorta wasteful to use findTreeItem to retrieve a reference to an object that we already have when forming the Uri. Instead, I just store the tree item in a Map with a random key and pass the key in the Uri, eliminating the need for findTreeItem. This method is probably a bit faster too.
Drawbacks of this method include:

File system provider will not be able to handle URIs that have originated from this method and have tree items in the Map. This is fine since we currently don't support this anyway.

Oh, this is only for the FileSystem? So findTreeItem will still be used in normal TreeDataProviders?

Oh, this is only for the FileSystem? So findTreeItem will still be used in normal TreeDataProviders?

Yes this PR is only removing the usage of findTreeItem (and anything AzExtTree* related for that matter) from the file system. Side note: maybe we should rename this class...
However, findTreeItem is still used in other places in the v1.5 client extensions. But we are not exposing a find API in v2. At least not yet. There's a chance we absolutely must add it when upgrading v1.5 extensions to v2. However we want to avoid this as much as we can because the whole find tree item thing is a bit problematic.


The "bugs to triage" mail contains bugs of type task

I received a "bugs to triage" mail from autonag today which listed not only defects, but also tasks. Please let me know if you need additional information on the scope.

@emceeaich wdyt ?

Right now we're assuming that tasks should be triaged. What model is your team working from for managing tasks?


orientDb stores java.util.Date without time

Bug Description
As described in the subject, orientDb stores java.util.Date without time. For example, once stored org.carlspring.strongbox.domain.ArtifactEntry#created is always at midnight.
Steps To Reproduce

Run org.carlspring.strongbox.services.impl.ArtifactEntryServiceTest and watch the logs

Expected Behavior
org.carlspring.strongbox.domain.ArtifactEntry#created and other java.util.Date fields should be stored in database with proper time

Please assign this on me.

@evsikov ,
Please, feel free to dig into this task and to join our chat channel.

Can I try this?

I am currently working on this issue.


Install error.  TrueNAS-SCALE-<IP_ADDRESS>

Describe the problem
When I try to install docker from truecharts it gives me the error. I have last OS version

Container definition
truechats
Container logs
Ошибка: Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/middlewared/job.py", line 427, in run
await self.future
File "/usr/lib/python3/dist-packages/middlewared/job.py", line 465, in __run_body
rv = await self.method(*([self] + args))
File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1247, in nf
res = await f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1379, in nf
return await func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/plugins/chart_releases_linux/chart_release.py", line 486, in do_create
await self.middleware.call(
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1368, in call
return await self._call(
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1328, in _call
return await self.run_in_executor(prepared_call.executor, methodobj, *prepared_call.args)
File "/usr/lib/python3/dist-packages/middlewared/main.py", line 1231, in run_in_executor
return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
File "/usr/lib/python3.9/concurrent/futures/thread.py", line 52, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/lib/python3/dist-packages/middlewared/plugins/chart_releases_linux/helm.py", line 44, in helm_action
raise CallError(f'Failed to {tn_action} chart release: {stderr.decode()}')
middlewared.service_exception.CallError: [EFAULT] Failed to install chart release: Error: INSTALLATION FAILED: execution error at (minecraft-java/templates/common.yaml:2:4): Expected helm version [>=3.12], but found [v3.9.4]. Upgrade TrueNAS SCALE OS

I don't maintain the Truechart definition. You need to contact the author of that.

You most likely still doesn't have the latest version. You may have to manually update your Truenas SCALE. Is your version by chance 22 and not 23?

You most likely still don't have the latest version. You may have to manually update your Truenas SCALE. The latest version is 23 not 22.

Thanks!


Backtick in .zshrc file causing real issues

I have the following line in my ~/.zshrc file:

I've specifically done a screen grab so you can see the Name insanity. I have to have those backticks in the string to make the aws call work properly. If i try to :wq that as is and source ~/.zshrc I get: command not found: Name
What am I missing?
sorry for the tiny screen grab

Alias
#aws cli
alias list-aws-public="aws ec2 describe-instances --output table --query 'Reservations[].Instances[].[Tags[?Key==`Name`] | [0].Value, PublicIpAddress]'"

Do you have a function called Name that you're trying to execute and return the value of?
Backticks in ZSH (and also in bash) are used for command substitution. You you want the backticks without this behavior--which is the expected behavior--you will need to escape them or place the enclosing string inside of single quotes.

This is an issue with how zsh (and bash and other shells) handle quoting. Double quotes will substitute variables and other things, like inline commands ($(command) or command). Single quotes don't. More info about quoting here.
Thus, a possible fix for your alias is using single quotes everywhere, with a little help of backslashes to quote inner single quotes:
-alias list-aws-public="aws ec2 describe-instances --output table --query 'Reservations[].Instances[].[Tags[?Key==`Name`] | [0].Value, PublicIpAddress]'"
+alias list-aws-public='aws ec2 describe-instances --output table --query '\''Reservations[].Instances[].[Tags[?Key==`Name`] | [0].Value, PublicIpAddress]'\'''

That looks a bit confusing because you can't quote a single quote inside single quotes.
A better fix, which is the first one proposed by @varCharlie, is escaping the backticks with a backslash, like so:
-alias list-aws-public="aws ec2 describe-instances --output table --query 'Reservations[].Instances[].[Tags[?Key==`Name`] | [0].Value, PublicIpAddress]'"
+alias list-aws-public="aws ec2 describe-instances --output table --query 'Reservations[].Instances[].[Tags[?Key==\`Name\`] | [0].Value, PublicIpAddress]'"

This just uses a couple of characters more, and you don't end up with the ugly syntax of quoting single characters.
I hope that explains a bit more about how zsh works. Let me know if you have trouble with something else!


[apostrophe-areas] Port and implement areas plaintext helpers

This has all been ported, yeah @boutell ?


add peer dependency

Add required dependency according to https://eslint.org/docs/developer-guide/working-with-plugins#peer-dependency

Thanks for the work, I'll review these over the weekend.


Fix comments and add more operators

This pull request fixes:

The comments issue
Operators boundaries for non alfabetic operators like ==, <|, <=...

Adds:

Operator recognision of &&, | and =

Reviewed the commits. Really great job. I hadn't spent much time on the regexes. These look great


CartoDB + PG Strom

Hello,
Has anyone already successfully integrated cartoDB with PG Strom?
To try it, I have a small server with a Nvidia Cuda, and I'd like to know if anyone has tried the experience... I work on large set of relational data (> 3GB) to viz public transport.
Thank you

There has been no activity on this issue for more several months. We are closing it. If you think this still needs to be addressed please open a new issue.


[Bug]-[16]:[SSO] [Disconnect] - Superuser unable to login

Is there an existing issue for this?

[X] I have searched the existing issues

Description
Scenario- The superuser used form login to create an account with Appsmith before configuring SSO. A superuser then uses Admin settings to configure SSO with a service provider. On successful configuration, the logins are driven by the SSO provider. The superuser shares the same email for form login as well as SSO. On disconnecting the SSO, the form logins are disabled, and thus, a superuser is also unable to connect to Appsmith. This could be a valid scenario where a company might have moved to another SSO provider and would want to disconnect and re-configure a new SSO provider. So, a superuser should be able to use form login as well as SSO login.
Steps To Reproduce
Go to Admin settings --> select SAML --> configure SAML for SSO --> SSO successfully configured.
Login using SSO --> Login successful.
Go to Admin settings --> Edit SAML --> Click Disconnect --> SSO disconnected.
Login using Form login -- > A login attempt is unsuccessful and the user is prompted with an error message
"Please use Keycloak authentication to login to Appsmith". (Screenshot attached below)

Public Sample App
No response
Version
Self hosted

@jnikhila great catch!
@ankitakinger @trishaanand @vuiets we should think about this scenario to prevent users from shooting themselves in the foot

Stats

Stat
Values

Reach
4

Effort (months)
0.5

People using SSO/SAML to login: 24
Assuming that 20% of them will transition to a different provider: 2
I've parked the effort for this to be 2 person-weeks, please correct me if I'm wrong here.

@Nikhil-Nandagopal, AFAIK, once you have configured SSO, you will have to have at least one form of login available to turn it off. This case was discussed during SSO weekly. We will check how we missed this scenario.

@hiteshjoshi I think the problem here is that once you turn it off, you can have form login still enabled but because your ID is now linked to the SSO login, you can't use form login to login

Linked to #34651.

any update?


Could you provide example code for AdaLoRA finetuning decoder-only model?

Feature request
The current example of AdaLoRA is on facebook/bart-base.  Since AdaLoRA requires hand-crafted calculations on loss, would it be possible to provide me some hints on how can this be done when it comes to decoder-only (e.g., Llama-Instruct) LM?
Specificially, I would like to mask out the loss calculation on the instruction part or system prompt, focusing only on the assistant response.
Motivation
AdaLoRA requires hand-crafted calculations on loss, which becomes complex when desired to mask out some system/instructino tokens.

As there is response yet, I am trying to answer your question how I would approach this:

Specificially, I would like to mask out the loss calculation on the instruction part or system prompt, focusing only on the assistant response.

My idea would be to work with the special tokens in the prompt. Depending on the model you are using (the prompt template varies across different model architectures), I would look for the <|assistant|> special token. As the special tokens <|user|> and <|system|> are typically before the <|assistant|> special token, my approach would be to write custom trainer class that masks the input starting with the <|assistant|> prompt and excludes everything before that special token.

Thank you for your response. Yes, I have successfully written a custom code to set the labels of tokens before <|assistant|>  to -100. Then following the example code provided, a custom training loop is conducted. The results are reasonable.

I'm glad to hear it worked, thanks for your suggestion @d-kleine.
If there are no further questions, feel free to close the issue @SpeeeedLee.


Change unstable to unstable-kv

Deno deprecated unstable to more granular control over api deprecation. Please change the --unstable to --unstable-kv in deno.json.

Thanks for the reminder!
I will hopefully make the update tomorrow :)

I updated that, and bumped deno and other deps' versions. Enjoy<EMAIL_ADDRESS>


Update for current YouTube

Not sure if anyone is still using... or maybe i should publish my own on greasyfork instead?

Thank you for works.  I've checked that they will work properly.  Updates will come soon.