Add "Exposed Ports" to Add container page

When adding a container one can set Port mapping or "Publish all exposed ports" but there is no option to expose a port. Sometimes the Dockerfile doesn't specify any exposed ports but it is useful to be able to expose a port at container creation time. In docker-cli this maps to "--expose".

You can already, you just click the plus for a new entry and then type in a port number manually on the container side (right hand side)
Rgds,
Neil Cresswell
On 19/03/2017, at 7:15 AM, Stefan Dimitrov <notifications<EMAIL_ADDRESS>wrote:
When adding a container one can set Port mapping or "Publish all exposed ports" but there is no option to expose a port. Sometimes the Dockerfile doesn't specify any exposed ports but it is useful to be able to expose a port at container creation time. In docker-cli this maps to "--expose".
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_portainer_portainer_issues_698&d=DQMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=AAorBM4TlIe2nScEmz6SshinTiZLbYZPTOhPJg7QZpw&s=te7sn-JbCi8y90nmhgERadyVr_qT2s1GInrWcbiWeDw&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AWGrlWx3eHlfIVu9wSha7gekZr48gIEbks5rnB9FgaJpZM4MhfvK&d=DQMCaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=AAorBM4TlIe2nScEmz6SshinTiZLbYZPTOhPJg7QZpw&s=R6E--t-UwYma5KxceBojUvRSw7CWV4uTS6swjzMdYaQ&e=.

@ncresswell that's not the same, it maps a random port to the exposed port. If the container is on the bridge network it will open it up on the public IP.

@ncresswell is correct I think, here is a blurb from a docker networking blog post by CenturyLink:
"There are several ways to do this (Expose ports): you can expose a port via the --expose flag at runtime, or include an EXPOSE instruction in the Dockerfile. You can also publish ports by using the -p or -P flags in the Docker run string."
The post explains the differences between the individual types of network setups involving ports: https://www.ctl.io/developers/blog/post/docker-networking-rules/
I don't believe a random port is mapped aswell, but the port that is specified. See the attached images.
Mapping 9000 -> 9122 and 9002 -> 9123

Exposed on those ports

Available on specified host port 9122

Correct.
You can use portainer to expose ports as defined in the docker file, or to expose new ones using the UI
Rgds,
Neil Cresswell
On 19/03/2017, at 1:10 PM, Steven Gantz <notifications<EMAIL_ADDRESS>wrote:
@ncresswellhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_ncresswell&d=DQMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=tsxcnJTSnDELteI7bZvIa2pmMqQR2XqrzWtY99RLbqc&s=ETOO3N6dfsUn8DTINtDOJqMoekzr6jMkEnATVt8HlKY&e= is correct I think, here is a blurb from a docker networking blog post by CenturyLink:
"There are several ways to do this (Expose ports): you can expose a port via the --expose flag at runtime, or include an EXPOSE instruction in the Dockerfile. You can also publish ports by using the -p or -P flags in the Docker run string."
The post explains the differences between the individual types of network setups involving ports: https://www.ctl.io/developers/blog/post/docker-networking-rules/https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ctl.io_developers_blog_post_docker-2Dnetworking-2Drules_&d=DQMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=tsxcnJTSnDELteI7bZvIa2pmMqQR2XqrzWtY99RLbqc&s=rr-KGIi0o7iOAB5KkSWUxMKPL_jM2Ui8suMtacdm378&e=
I don't believe a random port is mapped aswell, but the port that is specified. See the attached images.
Mapping 9000 -> 9122 and 9002 -> 9123
[image]https://urldefense.proofpoint.com/v2/url?u=https-3A__cloud.githubusercontent.com_assets_7944136_24077038_8802c6a8-2D0c16-2D11e7-2D90a2-2D26882d4e8abc.png&d=DQMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=tsxcnJTSnDELteI7bZvIa2pmMqQR2XqrzWtY99RLbqc&s=dNs8oEJD6Mncc4iL17vUc_3wlFdt6jpE9jdpUMUhhro&e=
Exposed on those ports
[image]https://urldefense.proofpoint.com/v2/url?u=https-3A__cloud.githubusercontent.com_assets_7944136_24077042_a69ed2fa-2D0c16-2D11e7-2D8c9f-2D5454321f8bd0.png&d=DQMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=tsxcnJTSnDELteI7bZvIa2pmMqQR2XqrzWtY99RLbqc&s=iynHHp8GqCBEO8_RDxdJnda9pUaV1AWxghWnp_5wxY8&e=
Available on specified host port 9122
[image]https://urldefense.proofpoint.com/v2/url?u=https-3A__cloud.githubusercontent.com_assets_7944136_24077048_c4d92c0c-2D0c16-2D11e7-2D986f-2D334840fb7be1.png&d=DQMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=tsxcnJTSnDELteI7bZvIa2pmMqQR2XqrzWtY99RLbqc&s=YR8YMsZXydkVaUtfTtUpnoW1Atw8zPiNeKDi0WMSIIg&e=
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_portainer_portainer_issues_698-23issuecomment-2D287584461&d=DQMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=tsxcnJTSnDELteI7bZvIa2pmMqQR2XqrzWtY99RLbqc&s=K43BG-IHuu1I3LUncehqcCk1rmUaigQfXdjh4aZiPXk&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AWGrlcDgRX4GFksvXXKNZLo0obC-2DzHrAks5rnHJlgaJpZM4MhfvK&d=DQMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=0fx0h4vB56iTLpw2McH1ZD6TqG_QGpbggVOB-PfMJpM&m=tsxcnJTSnDELteI7bZvIa2pmMqQR2XqrzWtY99RLbqc&s=8fxqHpSGdTfXdTKaq8My9NiMdlC7VypcbIrs9lKSscg&e=.

You guys are confusing "EXPOSING" and "PUBLISHING" ports. What the current UI does is allow you to "PUBLISH" ports (i.e. the docker -p flag). If you specify two ports it will open the left-hand one on the host (i.e. on all interfaces by default) and will forward it to the right-hand one on the container. If you don't specify the left-hand one then it will open a random port on the host and will forward you to the right-hand one on the container. That's PUBLISHING ports, i.e. OPENING ports on the HOST.
What I asked for is EXPOSING ports (i.e. docker --expose or EXPOSE in the Dockerfile). When you expose a port there is no mapping involved, no port forwarding involved, it just allows linked (old style links) containers to talk over the exposed port or service discovery.

I think @dimitrovs is talking about this :
https://docs.docker.com/engine/reference/run/#overriding-dockerfile-image-defaults
and
https://docs.docker.com/engine/reference/run/#expose-incoming-ports
It's the possibility to override some Dockerfile instructions
It's not about really expose a port on host, it's just add an exposed port to the container configuration
Some tools (like traefik) read those exposed port (which are not really exposed on host) for doing some tasks on
Regarding on https://docs.docker.com/engine/api/v1.26/#operation/ContainerCreate
Portainer allow the "-p" and "-P" of docker run, with the
containerConfig.HostConfig.PortBindings
, but I think no option for "--expose=[]" yet
containerConfig.ExposedPorts

@WTFKr0 yes, that's exactly it. Exposed ports are meta information about the container which is used for service discovery and auto-configuration, as mentioned.

Hi! is this implemented or not yet?

@panderine not implemented yet!

@surtic86 no ETA yet, I'd accept a PR for this

Okey i will look into it if i can help with this.
How should we Implement this Feature? Should i add another Section under "Port mapping" labled like "Port exposing" just with on Numberfield for the Container?

@surtic86 I think that it should be added in the advanced container settings panel (network tab) instead of under the ports configuration section of the main panel.
Reason is I believe that it will confuse most of the users if we allow them to add ports mapping and expose ports in the same place.
UX behavior should be similar to port mapping, with one number input and protocol selector.

Waiting eagerly for this, otherwise it's not possible to setup nginx-proxy as it requires -e for service discovery. https://hub.docker.com/r/jrcs/letsencrypt-nginx-proxy-companion/. Has anyone found a workaround? Thanks.

Any news on that?

@11111000000 no ETA yet, contributions are welcome.

So I started researching into how to go about doing this, and this is just a UI  implementation issue as the functionality for just exposing ports is for the most part already implemented.
The function preparePortBindings in the createContainerController actually configures exposed ports for the purpose of binding:
    function preparePortBindings(config) {
      const bindings = ContainerHelper.preparePortBindings(config.HostConfig.PortBindings);
      _.forEach(bindings, (_, key) => (config.ExposedPorts[key] = {}));
      config.HostConfig.PortBindings = bindings;
    }

changing the last line to config.HostConfig.PortBindings = {}; prevents the binding but leaves the port exposed.
I tested this on a fork alongside nginx-proxy and it does successfully expose ports.
Obviously this is a UI hack, and if you left it at this you could no longer port bind but only expose.
But this is a starting point for implementing exposed ports.

Is this by any chance already fixed? Or do I need to try to create a MR for it?

Still interested in this in 2021.... Any published fixes?

This needs added. Unifi is example where changing the https guest portal port number to support cloudflare (443/2083) within the container does not expose the port.
I'm needing to run this container via command line for now. This is a home lab so no major enterprise orchestration tools being used atm.

@klaki892 did you perform a pull request at all if it's that simple? I would love this added for my home lab. Also smaller orgs not using orchestration tools / config management may benefit from this simple missing feature.

Bumping once again as many containers in my workflow rely on exposing ports (nginx). Any progress on this since 2017?

Bumping once again as many containers in my workflow rely on exposing ports (nginx). Any progress on this since 2017?

yes,we need  to configure port mapping when add a container ,but this is not implemented,
so it seems to be useless

Bumping! Is it hard to implement ?

Also bumping, it would be very useful to be able to change the exposed ports through the Portainer UI

Bumping as well - this would be great for service discovery with nginx-proxy and unifi.

I'm also still waiting for exposed-port to finally be available in portainer to use nginx-proxy. Is there still no implementation for this in mid 2023? Without this feature I can't seriously use portainer in production use case.


Split out small objects (candles)

As raised by Farid on Icevision forum, small objects are detected with significantly lower accuracy. The Fire dataset does include many small objects in the form of candle flames, which are not really the kind of significant fires I wish to detect. Overall model accuracy could be improved by moving the candle images to a separate dataset.
https://arxiv.org/pdf/1902.07296.pdf

Actually better to optimise model to detect small fires, since this is how all fires start!


Home Page (Completion)

A great job well-done on implementing the HTML for the Home page.
@rodgersxy @Dprof-code @chykB @Maryanemwende @flexywork327 @TechDanny
Thank you for your active contribution.
However, we need to round up the page and make it come to life, by adding the CSS, JS, and the other rest.
@Mubarak1A will be your team lead, and @dohoudaniel will be your team's moderator.
@Dprof-code, we are waiting for your solution to #39. This will help speed the process of development.
This Google Doc contains everything we need to implement in the home page.
Visual Aid:

Home Page (Figma)
Website Details
Home Page (Google Doc)
YouTube Video

Yes I would work on it today.


gps issue

[ ] I have read the troubleshooting section of the README
What type of issue is this?
[ ] Error or problem compiling the library
[ ] Question or request for help
[ ] Hardware issue
[ ] Network issue
[ 1] GPS issue
What are you using?
Modem:  i am using a7608-e esp32-s3-wroom
AT+SIMCOMATI command returns
String: whenever i try to start the gps i get restartgps! and i see the messega every now and then
Scenario, steps to reproduce
 active gps info
Expected Results
 like i said gps info
Actual Results
 OK
AT+CGNSSPWR?
+CGNSSPWR: 1,0,1
OK
AT+CGNSSPWR?
+CGNSSPWR: 1,0,1
gps restart!
Debug and AT Command Logs

Are you running the GPS_NMEA_Output example?
OK
AT+CGNSSPWR?

+CGNSSPWR: 1,0,1

OK
AT+CGNSSPWR?

+CGNSSPWR: 1,0,1   #  The return should be turned on here and it should not jump to restart GPS.
gps restart!


[Request] Adding vipbitcoin exchange

Hi guys,
Can anyone help to add vipbitcoin  exchange? It's an Indonesian-based crypto exchange and provides 0% fee for market maker. (Should be interesting for those who like to set up frequent trades in their strategies).
Here's the APIdocumentation provided by the exchange.
Really appreciate if it could be done shortly. Thanks!

Waiting for vipbitcoin to get back to me on the test account.

Great! Hopefully you can get it soon.
Otherwise, you might register a regular trade account. As far as I know, the registration only use phone verification (sms) unless we want to connect the account into our local Bank account. First time I registered, I was using non-Indonesian phone number and it was enough to register an account.
Thanks for your effort so far @danKHUU.

O., I registered for an account.
Working on it, should be ready for testing in couple of days.
I will let you know soon.

Waiting for good news from @danKHUU . Have you been able to start testing?

Hi,
Yes i am running live bot testing now. wont be too long...

Good...
Happily waiting for it.

Hey,
Sorry for taking a bit, have been busy with works. I am about to put in a pull request.

Wow... great news then. :)

You can try this out by downloading the develop branch of gekko here: https://github.com/askmike/gekko/tree/develop
It will be part of the next stable release. As such I am closing this issue :)


Support for prefixed tables

Hey! Thank you a lot for this great project, I made a ton of google searches to finally find you...
I'm wondering if you support prefixed tables? My project run on multiple environments and my tables should be prefixed (some_prefix_my_table). An option in the .skeema file would be a perfect fit for me.
I stumbled upon https://github.com/skeema/skeema/issues/76 but this seems to be about prefixing databases names.
Do you have or plan any support for prefixing tables?
Thank you very much for your answers and have a nice day

Skeema doesn't support table name prefixes currently, but we can keep this open as a feature request.
I'd love to learn a bit more about the use-case please, as I'm not really familiar with it. Does each database schema use a different table name prefix for its tables, or do multiple prefixes exist within each database schema? Is there a particular motivation for using table name prefixing, vs using the same table names in different schemas? Thanks!
On a separate topic, re: tons of google searches, I'd like to find ways to make Skeema more discoverable. Are there any particular search keywords you tried first unsuccessfully? Thanks again.

Hey, thank you for your answer! If no one ever asked you something like that and I'm the first, it might not be worth it to even keep this open.
The use case is mainly with legacy apps where databases where used by multiple services. Thinking of Wordpress, Prestashop kind of services.
My personal use case that lead me to open this issue is also for a legacy app that use the same schema for multiple instances of the same service. Think of it as Wordpress A with prefix wp1, Wordpress 2 with wp2. In our "deployment process", we need to upgrade those wp1_articles when code is updated etc, wp2_articles some time later when user2 wants to upgrade etc...
Anyways, the more I think about it, the more I think it's bad practice, so we might just focus on migrating to different schemas rather than asking for skeema to do something about it.
For the google search question, here are some keywords I can remind of where I can't find you on the front page:

mysql migration tool
schema diff mysql (this search made me use some npm package named dbdiff, but it was not as cool as skeema)
CI mysql schema strategies (the github article mentioned below actually shows now with this search)

I finally found you on this github article: https://github.blog/2020-02-14-automating-mysql-schema-migrations-with-github-actions-and-more/
This article must have gave you some reasonable juice because I actually can find you on most of the keywords that come to my mind.
The thing I was really searching for was to get my database schema as versionable code. I come from ORMs such as Doctrine and Spring Data that allowed me to sync my code with my schema. Switching to pure SQL made me loose such tools.. So skeema is great! I just had to enable WSL on my windows to make it work, and hell yeah it works.

I see, that use-case makes more sense now, thanks! Prefixed tables in the same schema would be tricky to support in the near-term, since currently Skeema assumes the *.sql files in one subdir fully define a single logical schema. To support prefixes within the same schema, this would need to change to allow for the definitions to map to only a subset of the objects in a database schema. It's definitely possible this will get more flexible in the future, but no clear plans yet.
A related future feature idea is the notion of templating: providing the ability to use placeholder variables and conditional logic in the *.sql files. This could help with a few other feature requests (e.g. #87 and #21) as well. On its own though it wouldn't fully handle the ability to have different table name prefixes within the same schema, so it still would require changes in the dir-to-schema mapping flexibility to handle this use-case.
Anyway, we can either keep this open to see if other users have the same requirement, or close if you plan to migrate to separate schemas anyway -- up to you?
re: search terms, that's helpful, much appreciated! Good to know for future SEO efforts. And thank you for the kind words, always glad to hear from a happy user :)

Closing this for now, but will reopen if other users request similar functionality. Thanks!


fixes issue loading locations due to pagination

Substitute this line for a meaningful title for your changes
Overview

This is something to discuss with @agnessa: regardless we are trying to override the limit of pagination in some endpoints, setting pagination[size]: -1 will, actually, NOT disable the pagination but disable the default one (25) but it will never override the hard limit set in the API controllers. I don't think there's a way to override that default only for a single or specific endpoints, so we might need to come up with something like a custom endpoint to skip this pagination limit.

Removed the locationAtom: it was being used to store cached data.

Fixed logic to get location bounds (main reason of the crashes)

Designs
Link to the related design prototypes (if applicable)
Testing instructions
Please explain how to test the PR: ID of a dataset, steps to reach the feature, etc.
Feature relevant tickets

Checklist before submitting

[ ] Meaningful commits and code rebased on develop.

If the problem can be solved by increasing the hard limit, we can just do that - however, not to some crazy number, because next thing that happens is requests are going to time out and instances become unavailable. Otherwise, we can use Tiago's approach from orcasa: https://github.com/Orcasa-Platform/orcasa/commit/bd6f6f32943310ccd3f2f940be1fc49886563a18

I think increasing the limit to ~300 should be fine. This is mainly up to the location list: we need that list fully available at once no matter what. Right now, there are around 17X locations. I don't think this number will change too much in the future, hopefully.
So yeah, to keep things I would advocate to increase the general cap to ~300. I am not sure if this can suppose a problem to the stability of the platform, in general we are trying to skip pagination for the tabular data and locations, AFAIK.


Arrows not showing

When I am trying to do something like
<Slider>
                {items.map((item)=>{
                    return (
                        <h1 key={item.id}>
                            {item.name}
                        </h1>
                    )
                })}
</Slider>

I don't get arrows on carousel sides. Moreover it shows only one item per dot as opposed to showing all items per slide. If I use the same code but without items.map function it works as expected.

The problem was because when componentDidMount hook happens my items list is empty and new items are added inside that hook (for server-side rendering). Fixed the problem by fetching data before first rendering, but I think numberOfChildren should be counted on every rerendering, not only on component mount.
P.S. Thanks for the package!


[FEATURE] Support local deployment of BGE Re-Ranker

Support local deployment of BGE Re-Ranker and invocation through URL

Related: https://github.com/langchain4j/langchain4j/issues/1549

Hi @fmx0717, where is BGE Re-Ranker supposed to be deployed? Who will host it?

Hi @fmx0717, where is BGE Re-Ranker supposed to be deployed? Who will host it?

@langchain4j
Previously, colleagues have already been using bge-large-zh-v1.5 in their own Python services, deploying it together with their Python service and hosting it through the Python service. Now, I want to reuse bge-large-zh-v1.5 in Java via the interface provided by the Python service.  For example, I need something similar to the following approach: ScoringModel scoringModel = BgeLargeZh15ScoringModel.builder().baseUrl("http://<IP_ADDRESS>:8080/xx").build();

So that is this python service and what API does it have?


Logout when not active for a set of time.

Version: 0.1.8
Can anyone tell how can I change the timeout time for inactive? I was trying to edit a product page, but I took longer than usual. When I finish editing. The system log me out. How can I change it to longer time?

Hi @lYesterdaYl,
1st Way: There is a file in the project root directory named '.env'. In that file search for the key 'SESSION_LIFETIME'. Update that value.
OR
2nd Way: If somehow key is not present then just add the key and add value.
OR
3rd Way: Go to 'config/session.php' and search for the key 'lifetime' and value theirs.
Important Note: Value should be filled in respect of minutes.

I hope the above reply fixed your issue. I am closing this issue, feel free to reopen.


fix Custom Scalar should not be edited. output

fix: https://github.com/apollographql/apollo-ios/issues/3323
cast to TemplateRenderer
https://github.com/apollographql/apollo-ios-dev/blob/cce0d079611173f78f7cac6feeb56fc158ae0820/apollo-ios-codegen/Sources/ApolloCodegenLib/FileGenerators/CustomScalarFileGenerator.swift#L11-L13
Optional is different.
https://github.com/apollographql/apollo-ios-dev/blob/cce0d079611173f78f7cac6feeb56fc158ae0820/apollo-ios-codegen/Sources/ApolloCodegenLib/Templates/CustomScalarTemplate.swift#L23-L25
The implementation of the extension is used.
https://github.com/apollographql/apollo-ios-dev/blob/cce0d079611173f78f7cac6feeb56fc158ae0820/apollo-ios-codegen/Sources/ApolloCodegenLib/Templates/TemplateRenderer.swift#L76-L80
relevance
https://github.com/apollographql/apollo-ios-dev/pull/152

@matsudamper: Thank you for submitting a pull request! Before we can merge it, you'll need to sign the Apollo Contributor License Agreement here: https://contribute.apollographql.com/

Do we need to be worried that this was more than just a comment-related bug? I'd be far more concerned if we were actually overwriting pre-existing custom scalar files; do we need an additional test for that?

No, this wouldn't have affected that. It was just a change in the function signature for the functions:
func renderHeaderTemplate(
     nonFatalErrorRecorder: ApolloCodegen.NonFatalError.Recorder
   ) -> TemplateString? {

func renderDetachedTemplate(
    nonFatalErrorRecorder: ApolloCodegen.NonFatalError.Recorder
  ) -> TemplateString? {
    nil
  }

and

func renderHeaderTemplate(
     nonFatalErrorRecorder: ApolloCodegen.NonFatalError.Recorder
   ) -> TemplateString? {

Do we need to be worried that this was more than just a comment-related bug? I'd be far more concerned if we were actually overwriting pre-existing custom scalar files; do we need an additional test for that?

Executing the run-codegen.sh script on the 1.8.0 tagged release doesn't change any of the generated files and we have generated custom scalars in both AnimalKingdomAPI and GitHubAPI.
I've had a look through FileGeneratorTests and ApolloCodegenTests but I don't think we have a test that actually checks written file output to ensure nothing is written to pre-existing files that should not be overwritten.

No, this wouldn't have affected that. It was just a change in the function signature for the functions:

Sorry. I had my reply in draft for a while and didn't see your reply before posting it.


Improve Job Lineage edges

Edges for job lineage graph have mostly reflected input validity, and limits, but ignored ES query valves and de-duping.
These have been added to job lineage JSON, but it's becoming busy in the edge text:

Most egregious, the counts, which do not reflect a final count.  Might be worth calculating a total count, if possible, and including at the end?  or, continue to have counts for each filter applied?

This is will be tricky, if not impossible.
This is because input filters are applied across all input Jobs equally, and this occurs in Spark after Records have been "mixed" when retrieved from Mongo.
There are workarounds for validity and limits, but filteres like de-duping and es queries really can't be known for each Job.
One option, would be counting records from each input Job after all filtering has been done and writing to JobInput link.  This more precise count might be nice to have on the edge, and would be available elsewhere if needed.

Example pyspark query that would show counts for input Jobs, after all filtering:
In [10]: records.groupBy('job_id').count().show()
+------+-----+
|job_id|count|
+------+-----+
|   421|  323|
|   420|  438|
+------+-----+

But what would be the performance hit of this group_by()?

Confirmed this counting will work, but would need to address re-running of spark stages.
Or, could write input_job_id to Record rows, and then Job post-completion, get counts from Mongo based on input_job_id?

Done.


When Changing Months, the days do not update properly

(Start Month)

(End Month)
When I navigate from month to month, the days do not update properly. For example, I started in August of 2016, and the Dates/Days are correct. However, when I use the navigation arrow to go to the month of July, the days are not correct. July 13th for example, should be on a Wednesday but it is not.

Thanks for reporting the issue. I can reproduce this problem in Windows UWP in release mode. Did you see this problem on any other platform?
I looked into it and somehow the ExportRenderer is not working.
Try adding this to your UWP projects AssemblyInfo.cs file:
[assembly:Xamarin.Forms.Platform.UWP.ExportRenderer(typeof(XamForms.Controls.CalendarButton), typeof(XamForms.Controls.Windows.CalendarButtonRenderer))]

@rebeccaXam I emailed you back before reading the assembly code. What will I have to do to make this work for iOS and Android? I'm guessing...
[assembly:Xamarin.Forms.Platform.iOS.ExportRenderer(typeof(XamForms.Controls.CalendarButton), typeof(XamForms.Controls.iOS.CalendarButtonRenderer))]

[assembly:Xamarin.Forms.Platform.Android.ExportRenderer(typeof(XamForms.Controls.CalendarButton), typeof(XamForms.Controls.Android.CalendarButtonRenderer))]

After re-testing, the issue does not exist on android but does exist on iOS.

@rebeccaXam Just wanted to confirm that the issue still exists on iOS, even after updating to 1.0.4

In 1.0.7, this issue still exists on iOS release build. When changing month, the days do not update properly. But it is fine in iOS debug build.
Although I can still enable debugging option in the iOS release build, to deploy a larger app and dismiss this issue.
Hope someone has a solution to fix this issue.

https://github.com/rebeccaXam/XamForms.Controls.Calendar/issues/10

The fix for this shown in the README did not work for me, I had to do this instead:
[assembly: Xamarin.Forms.ExportRenderer(typeof(XamForms.Controls.CalendarButton), typeof(XamForms.Controls.<Platform>.CalendarButtonRenderer))]
However, I only experienced this issue on iOS.

The fix that was shown in the README did not work for me, I had to do this instead in platform project's AssemblyInfo.cs:
[assembly: Xamarin.Forms.ExportRenderer(typeof(XamForms.Controls.CalendarButton), typeof(XamForms.Controls.<Platform>.CalendarButtonRenderer))]
ExportRenderer is no longer in the platform-specific namespaces it seems.


[Feature Request]: OIDC Validation

Description
For security purposes and in context of specification OIDC should get an validation:
LINK

I really don't see a reason to do this? Here's a snippet from Google's OIDC docs:

You need to validate all ID tokens on your server unless you know that they came directly from Google. For example, your server must verify as authentic any ID tokens it receives from your client apps.

Since the ID token is single use and we can assume it's directly from the provider's API, I don't see any benefit on validating it.


Hover Animation

I want to add some hover animation to the webpage kindly provide me the acess

@akhil11sharma Can you elaborate more

The red background creates a high contrast, making the images less noticeable. However, I can implement a hover effect on the images to make them stand out when the mouse moves over them. This approach will enhance the user experience, making it easier for users to distinguish and interact with the images more effectively.Kindly grant me acess to showcase my work

@akhil11sharma Go ahead.

Hi @akhil11sharma  You can start working on it now.
I got my account back today. Sorry for the inconvinience.

@akhil11sharma Will wait until tomorrow.
Please do let me know if any updates.


Kore Based Rewriter API's Search Functionality Related Code

@grosu @dwightguth .Support for search in the Kore based Rewriter API. WIP, do not review yet.

I merged my pull request so if you rebase your own on top of master, I will review this now.

Couple small things but overall it seems aright for now, although I note the search command isn't working yet. But that's not that critical for me. Can you merge your latest changes into this pull request for me to review though?

Jenkins:test this please

Jenkins: test this please

Jenkins: test this please

@dwightguth Please review.

Jenkins: test this please

looks okay. Feel free to merge now or wait until you have the patterns and the watches.

Thanks Dwight, I'm merging this PR now. It gives a good indication of what the new debugger is going to be like, and also gets rid of the old debugger frontend, which was unnecessarily inconvenient to use. Merging this now will also help keep the size of the next diff reasonably small.


Is the backup of ardb consistent?

we use command "BGSAVE backup" , and i want to konw whether the backup is consistent?
thx

all write operations would be blocked until backup finished.


How to change the number of columns depending on the number of items?

How to implement this?

4 columns for 3 or more items
2 columns for 2 items
1 column for 1 item

That is, if I add or delete items from the list, the number of columns should change.
As far as I understand now, the reInit() method does not solve this problem? Can you please tell me how can I implement this?

In this condition, you can edit the source code. While reIniting, assign new options to the old options of macy instance. Just update the options is ok.


fix: support duckdb 1.1.0, update globals() for scanning

duckdbs scanning must have changed in 1.1.0 such that we need to augment the default globals() with the notebook globals(). this feels more correct, but I am unsure why this worked previously.
Fixes #2277 #2272

Updated to eval with the correct globals instead of mutating the module globals.


focal loss

Hi,
I don't know why the targets are set to -1, which will cause the targets in some positions to always be -1. In focal loss, shouldn't the value of targets be 0 or 1?
https://github.com/yhenon/pytorch-retinanet/blob/ef2c16f4ad66b652b836be36465ee63c978f3194/retinanet/losses.py#L90
@yhenon Looking forward to your reply

For example, in the position of 0.4<IoU<0.5, targets is -1

I'm sorry, I made a mistake


tslint errors

Hi @jkuri ,
Thank you very much for this module.
There is a lint error in ng2-datepicker.component.ts:
ng2-datepicker/src/components/ng2-datepicker.component.ts:1:90
'Provider' is declared but never used.
Would you please fix it? :)
There are also a few in slimscroll.directive.ts:
ng2-slimscroll/src/directives/slimscroll.directive.ts:147:9
'el' is declared but never used.
ng2-slimscroll/src/directives/slimscroll.directive.ts:182:9
'target' is declared but never used.
ng2-slimscroll/src/directives/slimscroll.directive.ts:196:9
'barTop' is declared but never used.
ng2-slimscroll/src/directives/slimscroll.directive.ts:214:9
'body' is declared but never used.
ng2-slimscroll/src/directives/slimscroll.directive.ts:215:9
'el' is declared but never used.
Thank you again and I appreciate your effort!
John

To get around this, i had commented out the lines of offending code directly in ng2-slimscroll/src/directives/slimscroll.directive.ts
so comment out lines 147, 182, 196, 214 and 215. It'll at least get things working again until slimscroll gets fixed.

added PR for ng2-slimscroll, one could use the fork that fixes those issues until it's merged: npm i --save odolha/ng2-slimscroll


Performance Discrepancies On Similar Hardware

Currently comparing 2 similar hardware platforms both containing same software builds. Results from this comparison have yielded significantly different performance results.  According to the lscpu dump, eve's dom0 on the E3845 is running as a "para" and eve's dom0 on the E3940 is "full".  Additionally, we a running a Windows 10 VM as a guest and noticed the more vCPUs allocated on the E3940, the lower the performance.  2 allocated vCPUs render the Windows 10 VM unusable.  The performance on the E3845 has remained consistent, which leads me to believe this issue has to do with Xen being in a "para" versus "full" state.
Does anyone know how/when/where the Virtualization Type is set?
Is there a way to manually set the Virtualization Type?
Any other suggestions based on this issue?
Models and Versions:
Architecture: AMD64
Processor: Intel Atom E3845 and Intel Atom 3940
EVE Version: 8.11.0
Linux Kernel: 5.10.121 with SMP
Hypervisor: Xen
Xen Version: 4.15
Other Notes:

VT-X enabled for both systems
conf/grub.cfg was updated to use 4 CPUS for hv_ctrd, hv_eve, and hv_dom0
lscpu man page says Xen dom0 can report wrong data (https://www.man7.org/linux/man-pages/man1/lscpu.1.html)

Thank You

@geoffreymoyerweatherford I see that E3940 has VT-d while E3845 does not support VT-d, which explains why the latter needs to use paravirtualized device drivers. That might make some difference in I/O performance, but I don't know how I/O intensive your application is.
I don't understand why you did:
conf/grub.cfg was updated to use 4 CPUS for hv_ctrd, hv_eve, and hv_dom0
That will make less CPU cores available to the guest VMs, since you are assigning more cores to the EVE orchestration services. Was there an issue which make you change the default of 1?
Note that the CPU allocation for the guest VMs are done using the EVE API from the controller.

Thank you for the response @eriknordmark
Some Rationale:

The reason I tweaked the conf/gub.cfg hv_dom0, was the 3 remaining CPUs never seemed to be released on bootup.  The dmesg's, when using the defaults of 1, says we are booting in SMP but only 1 CPU is available.  This was the case on both the E3940 and E3845.  I thought dom0 would need to release these to make them available to the guests, please correct me if this is wrong.
The reason I tweaked the conf/gub.cfg hv_ctrd, was to increase the max vCPUs available to guest VMs and Containers.  I assumed these ran as containerd-shims, please correct me if this is wrong.
conf/grub.cfg hv_eve can be returned back to its default.

There was an issue on the E3940 when we allocated 2 vCPUs via the EVE API and used the default values of 1 in the grub.cfg,  the Windows OS never properly booted.  It would halt in its initial load and freeze on the "spinning wheel".  When we allocated 1 vCPUs via the EVE API and used the default values of 1 in the grub.cfg,  the Windows OS did boot.  However, we ideally need more cores for this VM.  We were able to allocate 2 vCPUs when we bumped the default values from 1 to 4 in the grub.cfg but performance suffered significantly in the VM.  Thats makes more sense now with your description above regarding EVE services.  using more cycles.
We did not see the same issue when performing this same test on the E3845 using the same constraints.  We were able to allocate 2 vCPUs via the EVE API and use the default values of 1 in the grub.cfg.
Are you aware of a way to disable the VT-d capabilities on the E3940 processor?
Thanks Again.

The reason I tweaked the conf/gub.cfg hv_dom0, was the 3 remaining CPUs never seemed to be released on bootup.  The dmesg's, when using the defaults of 1, says we are booting in SMP but only 1 CPU is available.  This was the case on both the E3940 and E3845.  I thought dom0 would need to release these to make them available to the guests, please correct me if this is wrong.

With Xen the output from Linux running in dom0 only refers to the resources (in this case CPUs) allocated to dom0.
The Xen hypervisor keeps the other CPUs and then will be used by guest domains.
(Note that KVM works completely differently in this respect.)

The reason I tweaked the conf/gub.cfg hv_ctrd, was to increase the max vCPUs available to guest VMs and Containers.  I assumed these ran as containerd-shims, please correct me if this is wrong.

No, they are not. The application instances you deploy on EVE-OS are always deployed on the hypervisor, whether the application instances are VM images or OCI containers. The use of containered inside EVE-OS is an implementation detail (we use linuxkit as a way to build and run the different components inside EVE-OS).

conf/grub.cfg hv_eve can be returned back to its default.

There was an issue on the E3940 when we allocated 2 vCPUs via the EVE API and used the default values of 1 in the grub.cfg, the Windows OS never properly booted. It would halt in its initial load and freeze on the "spinning wheel". When we allocated 1 vCPUs via the EVE API and used the default values of 1 in the grub.cfg, the Windows OS did boot. However, we ideally need more cores for this VM. We were able to allocate 2 vCPUs when we bumped the default values from 1 to 4 in the grub.cfg but performance suffered significantly in the VM. Thats makes more sense now with your description above regarding EVE services. using more cycles.

Is there a particular reason you are using Xen and not KVM? I think KVM is what is commonly used today.
My understanding is that if you use Xen without VT-d you'll need to have the paravirtualized device drivers in your Windows image; I don't think they are present by default. (And it is possible that the pv drivers are not needed with KVM on the same hardware since the Qemu setup is different.)

We did not see the same issue when performing this same test on the E3845 using the same constraints. We were able to allocate 2 vCPUs via the EVE API and use the default values of 1 in the grub.cfg.

That is very odd since I'd expect the absence of VT-d being more problematic; perhaps Xen+qemu does emulation on the fly?

Are you aware of a way to disable the VT-d capabilities on the E3940 processor (without BIOS disabling it)? This option is not available in our BIOS version.

I haven't seen any BIOS where one can enable/disable VT-d; some older BIOS required enabling VT-x since it wasn't enabled by default.

Is there a particular reason you are using Xen and not KVM? I think KVM is what is commonly used today.
We are using Xen since it was the only hypervisor that booted Windows for us.  When tried using KVM (and ACRN) but we saw the Windows VM enter a repair loop then entered the Advanced Settings/Recovery Wizard.  It would never get beyond either of these states.  I was also able to notice the BSOD message SYSTEM_THREAD_EXCEPTION_NOT_HANDLED.


Nested Column Groups

Are there any provisions for nested column groups? For example, a layout like this:
 _______ _____________ ______________ ______________
|       |_____________|______________|______________| // First grouping
|_______|____|____|___|____|____|____|____|____|____| // Second grouping
|_______|____|____|___|____|____|____|____|____|____| // First data row
|_______|____|____|___|____|____|____|____|____|____|
|_______|____|____|___|____|____|____|____|____|____|
|_______|____|____|___|____|____|____|____|____|____|
|_______|____|____|___|____|____|____|____|____|____|
|_______|____|____|___|____|____|____|____|____|____| // Totals

Nevermind, for some reason I was wrapping the ColumnGroups I actually needed inside of a ColumnGroup with no header, which was superfluous. Disregard.


Update limits-specifications-teams.md

Specify max size of DL/SG/O365 group that can be imported in to a team.

@JohanFreelancer9 Copy editing is needed for this PR. Thanks.

@SerdarSoysal PR has been copyedited and is ready for final review, could you please check and merge? Thanks!


Fix: Remove localheinz from condition for merge job

This PR

[x] removes @localheinz from the condition for the merge job

Hmm, looks like it needs approval before auto-merge can be enabled.


Allow multiple instances of FastestServerLB globally

Currently, in Solrs 3, when you build a FastestServerLB instance, it will call initJmx().
Calling initJmx() twice will cause an exception, because it will try to register the class MBean twice, so you can't create two instances of FastestServerLB globally.

The latest version is 2.11.0, therefore I'd release the fix as solrs 2.11.1.

The fix is released with version 2.11.1 now.


add channel slug to SaleorState

Task linked: SALEOR-614 SDK should keep track of chosen channel

I'm not sure, but probably we should allow to change channel more dynamically during app usage. Now, when channel in config changes, the SaleorProvider doesn't reflect its change and queries and mutations will be made with previous channel slug passed as variable.

Usually different channels will have different storefront versions (with different env configurations) so this is not a deal breaker right now, but I agree we should look into making this dynamic in future.


Added docu links between Char and Keyboard modules

Also, I wonder whether one of the two modules should import the other, so that the KeyCode type alias would be defined only in one place. (Imagine you change what type to use for KeyCode at some point for some reason. If you forget about one of the two places, the types of some functions suddenly get out of sync (so that library users cannot really use them together), without the compiler being able to warn you as the implementer of the library.)

Closing this, replaced by cleaner PRs https://github.com/elm-lang/core/pull/382 and https://github.com/elm-lang/core/pull/383 (that can be merged independently from each other).


pre-allocation of disk space

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm requesting a feature unrelated to a specific site
[X] I've looked through the README
[X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Provide a description that is worded well enough to be understood
Take youtube for example, yt-dlp --list-formats, it shows the size for a specific format, so it's possible to pre-allocate disk space.
BTW, though the size might not be accurate, a workaround is possible.
Provide verbose output that clearly demonstrates the problem

[ ] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[ ] If using API, add 'verbose': True to YoutubeDL params instead
[ ] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
No response

Can you explain your usecase?

Can you explain your usecase?

The download speed for me is very slow, allocate disk space KB by KB is not so efficient, and while other applications are allocating disk space at the same time, the physical allocation of file on disk may not be so continuous. If the pre-allocation doesn't write disk, it'll get some benifits by pre-allocation.
BTW many downloader supports pre-allocated disk space.

The download speed for me is very slow

I doubt this will be solved by pre-allocation. Any reason to believe otherwise? Any example we can test? (with verbose logs)

The download speed for me is very slow

I doubt this will be solved by pre-allocation. Any reason to believe otherwise? Any example we can test? (with verbose logs)

I don't mean pre-allocation will speed up downloading, I mean slow downloading speed has chances to make the downloading file on the disk not so continuous. e.g. when download several big files simultaneously, it may like this: file A writes some bytes on disk, then file B, then file C, ..., as a result the files on the disk might interweave each other on the disk.

@playgithub: So your concern is fragmentation? I don't think that will matters much for everyday usage at all, even if using HDDs. Please give an example where fragmentation would be detrimental.
As a workaround you may use aria2c with allows to preallocate disk space afaik.

@pukkandan: Note that downloaders usually provide the preallocation option to not run out of space while downloading (has happened to me before as well). This I think is a valid concern, however this would be hard to implement for the reason that no bigger disk writes should occur after preallocation (state file would be fine). I think at least the fragmented downloader will have problems flushing directly to file offset.

@playgithub: So your concern is fragmentation? I don't think that will matters much for everyday usage at all, even if using HDDs. Please give an example where fragmentation would be detrimental.

Yes, fragmentation.
With pre-allocation, there's only good and no bad.
However if it is complex to implement (especially for cross-platform), it might be better not to implement it.

As a workaround you may use aria2c with allows to preallocate disk space afaik.

That's fine.

I was not implying pre-allocation is useless; but want to understand OP's use-case and whether it'll actually be solved by pre-allocation. This issue looks like an XY Problem to me since there may be various symptoms that a user unfamiliar with yt-dlp's internals assume will be solved by pre-allocation, but actually won't.
It's also not trivial to implement for us, with the various interactions with resume options, fragmentation, postprocessing and needing to be cross-platform. So I want to ensure the benefits are actually worth the effort.
Correct me if I am wrong in any of these - these are my initial concerns, not tested conclusions:
First off, we cannot pre-allocate for fragmented files since the final file size is not known in advance. There are low level strategies to deal with such situation, but they are much more complex that a pre-allocation.
Most post-processing - especially the ones that need a lot of writes - are done by ffmpeg. I don't believe ffmpeg has a pre-allocation option.
So, what's left is direct downloads with no postprocessing; for which fragmentation is not a significant issue unless performing a lot of downloads simultaneously (not recommended for network/disk contention reasons anyway) - a very narrow use-case imo.
The easiest workaround for fragmentation concerns is just to download to a different disk using -P temp:. If you can use a ramdisk like @voidpointertonull, all the better.

I think buffering to write large chunks is good enough.
Does yt-dlp support it already by option --http-chunk-size?


[HUDI-1383] Fixing sorting of partition vals for hive sync computation

What is the purpose of the pull request
Fix hive partition synchronization where partition values (each value) are sorted for this multi part for new partitions computation.
This is a redo of https://github.com/apache/hudi/pull/2262. 2262 has lot of rebase commits, couldn't fix it myself. Hence created a new one.
Brief change log
(for example:)
Fix hive partition synchronization where partition values (each value) are sorted for this multi part for new partitions computation.
Verify this pull request
(Please pick either of the following options)
This pull request is already covered by existing tests, such as TestHiveSyncTool#testMultiPartitionKeySync.
Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

Codecov Report

Merging #2402 (baebb71) into master (ff8313c) will decrease coverage by 41.97%.
The diff coverage is n/a.

@@              Coverage Diff              @@
##             master    #2402       +/-   ##
=============================================
- Coverage     52.02%   10.04%   -41.98%
+ Complexity     2952       48     -2904
=============================================
  Files           395       52      -343
  Lines         17652     1852    -15800
  Branches       1809      223     -1586
=============================================
- Hits           9183      186     -8997
+ Misses         7710     1653     -6057
+ Partials        759       13      -746

Flag
Coverage Δ
Complexity Δ

hudicli
?
?

hudiclient
?
?

hudicommon
?
?

hudiflink
?
?

hudihadoopmr
?
?

hudisparkdatasource
?
?

hudisync
?
?

huditimelineservice
?
?

hudiutilities
10.04% <ø> (-59.62%)
0.00 <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

The diff coverage is n/a.

@@              Coverage Diff              @@
##             master    #2402       +/-   ##
=============================================
- Coverage     52.02%   10.04%   -41.98%
+ Complexity     2952       48     -2904
=============================================
  Files           395       52      -343
  Lines         17652     1852    -15800
  Branches       1809      223     -1586
=============================================
- Hits           9183      186     -8997
+ Misses         7710     1653     -6057
+ Partials        759       13      -746

Flag
Coverage Δ
Complexity Δ

hudicli
?
?

hudiclient
?
?

hudicommon
?
?

hudiflink
?
?

hudihadoopmr
?
?

hudisparkdatasource
?
?

hudisync
?
?

huditimelineservice
?
?

hudiutilities
10.04% <ø> (-59.62%)
0.00 <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


Access Violation, in GC if the camera is busy

Hello!
Seems i found some bug.
Envieronment:
Windows 10 21H2 19044.1826;
ffmpeg version n5.0.1-7-g7389a49fd3-20220706 Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 12.1.0 (crosstool-NG <IP_ADDRESS>_3defb7b);
Camera USB 2.0, ELP Sony IMX179 (3264x2448);
.NET 6.0
SeeShark 3.1.0 (from nuget package)
First make camera busy, for example open https://webcamtests.com
Next run code:
// See https://aka.ms/new-console-template for more information
using FFmpeg.AutoGen;
using SeeShark;
using SeeShark.Device;
using SeeShark.FFmpeg;

Console.WriteLine("Hello, World!");

FFmpegManager.SetupFFmpeg(@"C:\ffmpeg\v5.0_x64\", "/usr/lib");

(int, int)? videoSize = (640,480);
try
{
    Camera camera;
    using (CameraManager manager = new CameraManager())
    {
        camera = manager.GetDevice(0,
            new VideoInputOptions
            {
                InputFormat = null,
                Framerate = new AVRational
                {
                    num = (int)30,
                    den = 1,
                },
                VideoSize = videoSize,
            }
        );
        Console.WriteLine($"Using camera {camera.Info}");
    }
}catch(Exception ex)
{
    Console.WriteLine($"cannot initialize camera {ex.Message}");
}
GC.Collect();
GC.WaitForPendingFinalizers(); //TODO if camera is busy, i crash here in  AccessViolation;

And now we crash on Access Violation in GC:
output:
Hello, World!
cannot initialize camera I/O error
Fatal error. System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at SeeShark.Decode.VideoStreamDecoder.DisposeUnmanaged()
   at SeeShark.Disposable.Finalize()

C:\Users\lubag\OneDrive\Documents\TestApp\TestSeeShark\TestSeeShark\bin\Debug\net6.0\TestSeeShark.exe (process 5872) exited with code -1073741819.
To automatically close the console when debugging stops, enable Tools->Options->Debugging->Automatically close the console when debugging stops.
Press any key to close this window . . .

I notice that you're not manually disposing the camera. Camera is an IDisposable object.
What happens if you call camera.Dispose() before your GC calls?

I understand that the camera is busy, and it returned null. it couldn't open it.

I tested this on Linux and couldn't reproduce. It might be specific to Windows.

I tested this on Linux and couldn't reproduce. It might be specific to Windows.

I doubt this is platform specific considering our samples should also fail in Windows. This might be a user-induced issue and not our fault, but I could be wrong.

SeeShark\Decode\VideoStreamDecoder.cs:170 AccessViolation.
in
ffmpeg.avformat_close_input(&formatContext);
Here as I understand, that IO / Error was on line 55:
ffmpeg.avformat_open_input(&formatContext, url, inputFormat, &dict).ThrowExceptionIfError();

Accordingly, an attempt to close something that has not been opened leads to an error.

Oh, I see. I'll fix that on the PR then.
Also don't hesitate to comment on the PR instead, since you can review code directly which makes it easier to point out where the bug actually lies. :)

Accordingly, an attempt to close something that has not been opened leads to an error.

I now made it so that it doesn't close something that has not been opened (#28). Does it still do an AccessViolation?

I have no clue how I ended up putting the isFormatContextOpen line there... Man I'm tired
Anyways, thanks, will fix this.
Also, it occurred to me that your I/O Error might not be caused directly by the camera being busy.
Because when I try your exact code with my inactive virtual camera, I get this error message:
Cannot initialize camera: Device or resource busy

This tells me that FFmpeg can indeed return such an error message, and that for some reason that's not what it returned when you made your camera busy. It might be a bug inside of FFmpeg though...

Are you testing the example under Windows? or used Linux? In general, I have the impression that the AccessViolation error in case the input is not open is a FFMPEG bug. In the FFMPEG code there are a lot of checks for NULL, in case the Context or something else NULL methods simply return and nothing bad happens. And it shouldn't be like that.

Are you testing the example under Windows? or used Linux? In general, I have the impression that the AccessViolation error in case the input is not open is a FFMPEG bug. In the FFMPEG code there are a lot of checks for NULL, in case the Context or something else NULL methods simply return and nothing bad happens. And it shouldn't be like that.

I only use Linux, no dual boot, and I haven't done that in a while... I would gladly have Windows on a secondary PC, but I want to personally avoid it on my main unit.
And yes, it might be an FFmpeg bug, in which case I can't do much tbh. :/
At least the AccessViolation will be gone by the next release of SeeShark.


Handshakes over websocket

Does this library support init via websocket?
In JS library there is stompClient.over(websocketConnection), how can I do the same with StompKit? Thanks,

I have an [outstanding pull request] (https://github.com/mobile-web-messaging/StompKit/pull/27) to integrate this functionality using RocketSockets. I'm sure there still may be some issues (I've been testing with ActiveMQ with a WS connector).
I haven't received any comments on it though so I'm not sure how active the maintainers are here.

Any chance this will get merged?

@jmesnil @matzew pls merge pull request


Home screen

The code runs just fine and the home screen looks nice, we might need another button for the list of receivers screen. To be implemented in the future


Flatpack package

What would you like to be added:
Flatpack package.
Why is this needed:
It is pretty standard way of distribution of GUI applications this day.
I was able to create it:
apt install flatpak flatpak-builder
cd open-lens
flatpak remote-add --if-not-exists --system flathub https://flathub.org/repo/flathub.flatpakrepo
flatpak install -y org.freedesktop.Platform/x86_64/20.08
flatpak install -y org.electronjs.Electron2.BaseApp/x86_64/20.08
flatpak install -y  org.freedesktop.Sdk/x86_64/20.0
rm -f<EMAIL_ADDRESS>env DEBUG="@malept/flatpak-bundler" npx electron-builder --publish onTag --linux flatpak

More tests are needed, ie. if we really must to remove 1024x1024 icon (flatpak builder breaks with error if icon is bigger than 512x512).

I think it is worth of trying with the nearest release 0.1.1

It looks like Flatpak for current version of OS is supported only for x86_64

Ok, so I think Flatpaks make sense only when they are distributed via https://flathub.org/ then are built and hosted separately. Then can be built from the DEB package (the same for Snap packages).
In that case, I'm going to remove Flatpak from our releases as we would need only DEB package now.


No sync after some time stopped

Describe the issue
After leaving my server down for a couple of weeks (waiting for the new RAM) I accessed everything again and found some problems in the log. PS: Copying the blockchain continued to another machine and then I replaced the files on the server disk
Screenshots

Environment

Hardware platform: [Custom server]
Operating system: [Debian 11]
Version: [Debian GNU/Linux 11 (bullseye)]

Additional context
I've already checked the permissions on the .bitcoin folder and they seem to be fine
lrwxrwxrwx 1 bitcoin bitcoin 13 Feb 26 03:21 /home/bitcoin/.bitcoin -> /data/bitcoin

In addition, I noticed from the log that it keeps trying that series of operations and then gets to 'shutdown: done' and starts again

fixed by reconfiguring read and write permissions to directories


optional _defaultOptions

_defaultOptions now is "TypeScript-optional", which makes Angular 2 AoT work with this library

Could you create a new release and publish it to npm? Thanks!


Process.spawn fails if project folder isn't a real folder

I believe I may have found the cause of ionide/ionide-fsi#4 and ionide/ionide-paket#20. In my testing of ionide/ionide-fsi#4, I found that if Atom's current project folder is somewhere that isn't a real on-disk folder, like atom://about, then I can't spawn Fsi.exe. But if I delete the project folder, or make sure that it's a real on-disk folder instead of a location like atom://about, then I no longer get the "(Path) could not be spawned" error.
(To get atom://about as a project folder, BTW, simply remove all your existing project folders and then go to Help->About. A new "folder" labeled about will show up in the project folders pane; see screenshot below. If you right-click on it and choose "Copy Full Path", then paste that into a text editor, you'll see the path is atom://about.)

It looks like there are several lines of code in https://github.com/ionide/ionide-helpers/blob/master/src/ProcessHelpers.fs where Globals.atom.project.getPaths() is used to set the current working directory for spawning processes. The code appears to assume that these will always be real directories, but that is not always the case. If the project folder is atom://about, or any other value that isn't a real directory (I could also get the name of a file into there by closing all project folders, then opening an individual file), then that assumption will fail and the Process.spawn() function will fail.
Here are the locations in https://github.com/ionide/ionide-helpers/blob/master/src/ProcessHelpers.fs that make this incorrect assumption:
https://github.com/ionide/ionide-helpers/blob/master/src/ProcessHelpers.fs#L73
https://github.com/ionide/ionide-helpers/blob/master/src/ProcessHelpers.fs#L94
https://github.com/ionide/ionide-helpers/blob/master/src/ProcessHelpers.fs#L112
https://github.com/ionide/ionide-helpers/blob/master/src/ProcessHelpers.fs#L118
Also, https://github.com/ionide/ionide-helpers/blob/master/src/SettingsHelpers.fs#L45 should probably be looked at as well.

This also appears to be the cause of ionide/ionide-fsharp#180 -- note the screenshot with an "about" folder in his project-folders pane on the left, and how the Atom window's titlebar contains atom://about. And in ionide/ionide-paket#20, note the "config" folder in his project-folders pane, and the Atom window's titlebar that contains atom://config.
I suspect this is also the real cause behind ionide/ionide-fsharp#178.

atom/atom#10297 is tracking this issue from the Atom side, so the fake atom://foo folders might go away once that bug is fixed. But until it is fixed, ProcessHelpers.fs probably should check for this case.

Great find, we really appreciate the thorough detail of this issue report. I encourage you to make a PR with these changes to record credit for the fix. Otherwise I'll try to get to it in the next day or two, unless @Krzysztof-Cieslak  beats me to it ;)

I don't have a good dev environment set up to test the fix, and I don't really want to make an untested PR. If I compile a new version of ionide-helpers, how would I link that into my running copy of Ionide in Atom? Or would I need to build the complete Ionide project? I've seen https://github.com/ionide/ionide-fsharp/blob/develop/README.md, but I don't know how that would apply to ionide-helpers -- which isn't an apm package per se, but a set of DLLs used by the other packages at some point.
Basically, I'd be happy to do it but I need a bit more explanation about the build process: at what point does the F# become Javascript, and how do I get a change in ionide-helpers into my installed Atom package?

And while I'm at it, another change I think should be made: https://github.com/ionide/ionide-helpers/blob/master/src/ProcessHelpers.fs#L85 looks wrong to me. I've verified that if you pass a Javascript null as the options parameter of Node's child_process.spawn(), it will throw a TypeError. I think that ProcessHelpers.fs line 85:
            | _ -> null |> unbox<AnonymousType598>

should be changed to:
            | _ -> { } |> unbox<AnonymousType598>

As I said in my earlier comment, I haven't been able to test this change in F#. But I did test passing an empty Javascript object to child_process.spawn() as its options parameter, and it's perfectly happy with that. So in theory, the change above should work.

Currently Ionide Helpers is manually updated across the various projects and stored in the /lib/ directory. Eventually we'd like to use github releases as a dependency through paket rather than creating a nuget package for this dl, but that functionality hasn't been added to paket yet.
Currently the way I test out updates is using this FAKE Build Target. You'll need to copy your updated compiled dll into the aforementioned /lib/ directory before running it.
The the Runscript target (or on Mono BuildGenerator and RunGenerator targets) create the actual js in the target chain
"Clean" ==> "RunScript" ==> "InstallDependencies" ==> "TryPackage"

Anything else I can clarify?

That should be enough for me to get started with. But I'm afraid you shouldn't expect any actual code from me for a while; it's getting late here in my time zone, and I need to go home. And then I have several other things on my schedule in the next 2-3 days, so I don't realistically expect to be able to do any actual work on this until next Monday at the earliest. If you want to jump in and fix it before then, please do.

Well, I may need a bit more hand-holding through the build process, as I haven't used FAKE before. My first problem was that Paket wasn't downloading when build.sh ran the bootstrapper, but that was solved by running mozroots --import --sync. But now I'm getting:
System.IO.FileNotFoundException: /home/rmunn/code/ionide/ionide-helpers/temp\FunScript.TypeScript.Binding.atom.dll does not exist

when I try to build ionide-helpers. And I'm not sure where to start debugging this problem. /home/rmunn/code/ionide/ionide-helpers/temp/FunScript.TypeScript.Binding.atom.dll does exist, so the problem might be that backslash in the path, but I don't know where to look for the erroneous backslash.

how are you running the build script? I dev on windows so I don't know if there's an error in the build.sh
your file tree for the repository should look like

The issue may have been with path strings for the .dlls I just pushed a small edit to them that might resolve the issue

I'm running it by doing ./build.sh in the root of the repo.
Also, I can confirm that the backslash was the problem. Changing lines 48-52 of build.fsx to "temp/FunScript.(etc)" allowed me to successfully build the project. Backslashes as path separators work only on Windows and not on Linux, but forward slashes as path separators work on both Windows and Linux, so forward slashes are better when writing cross-platform code. (Yes, forward slashes as path separators work on Windows; not enough people know that.)
If you like, I'll submit a PR to fix the build scripts on Linux.

we know they work, just look at our other build scripts :wink: but you're right it should be fixed with this one. I had a brain fart earlier and changed the strings to unescaped strings with the @ and didn't fix the slashes.

Okay, I was able to build this and test it out on my local copy of Atom (on Linux Mint Debian Edition), and it handled the "atom://about as project path" scenario just fine.
Do note that I haven't yet tested this on Windows. I'll have access to a Windows machine with Atom in a couple of days, so I can test it then. But I went ahead and made the PR now so you wouldn't have to wait a couple of days to merge it if the code looks good. Just be aware that you might want to test it on Windows.

Thanks a lot for tracking and fixing this bug! New versions of plugins got released.


Prompting bug when generating 100+token responses with Pygmalion

Right now, when you call runGenerate(getMessage) to resume a generation, only the previous block of text is passed in to generate the new prompt.  This means that once output accumulates, the prompt will not be generated properly, which will make the model more incoherent than it should be.
As an example, suppose the model wants to generate A B C D where each letter is a block of 50 tokens and history is H.
It will get prompted with "H", "H A", "H B", "H C" and so on. Whereas it should be "H", "H A", "H A B", "H A B C".
This should be simple enough to reproduce, but Pygmalion responses can be fairly short which is why it probably wasn't caught yet. Once a long message is generating, the bug is easy to spot in the log.
I believe replacing that line with runGenerate(cycleGenerationPromt + getMessage); should be enough to fix this, and that change seems to work in cursory testing.

Fixed https://github.com/TavernAI/TavernAI/commit/2b4e36340e80f582884f4ac8bfdca1a5b1ebf08a


Creation of pretraining data

Hi @zihangdai and @laiguokun ,
thanks for publishing the paper and open sourcing the implementation :heart:
I've one technical question regarding to the vocab. I just downloaded a TF checkpoint and in the vocab-uncased.txt file the e.g. [SEP] and [CLS] that are famously known from BERT are missing. Instead their <sep> and <cls> counterparts can be found.
Could you confirm that I can just use my BERT-compatible vocab and "transform" it to the <> notation 🤔
So the following symbols must be found in the vocab file:
<pad>
<s>
</s>
<eod>
<eop>
<unk>
<cls>
<sep>
<mask>

Like it is specified here:
https://github.com/laiguokun/Funnel-Transformer/blob/1085523bc768e499d8c55edf6af0d70cb1cd27d2/tensorflow/data_utils.py#L35-L45
I just wanted to train a Funnel-Transformer for Turkish (in addition to BERTurk models) and I would really like to use the same vocab for comparison reasons :)
Best,
Stefan

Hi Stefan,
Apart from <cls>, <sep> & <mask>, others are just some potential special tokens. As you can see from this chunk of code: https://github.com/laiguokun/Funnel-Transformer/blob/1085523bc768e499d8c55edf6af0d70cb1cd27d2/tensorflow/data_utils.py#L48-L58
If the tokenizer does not has a special symbol sym, the code will simply skip it.
Another case is that you may want to add other special symbols. To to that, you can simply modify the special_symbols_mapping (as along as <cls>, <sep> & <mask> are retained).
Zihang

Hi Zihang,
thanks for the explanation :)
It is working with the vocab and I'm currently training a funnel transformer on TPU, so I'm closing the issue here :)


Enable editor switch using tab on dex modals

Tab key should switch editors on Dex modals

Additionally, the "enter" key on the passwords entry does not work, it adds a newline as a character to the password input field.


Added new mission vision section---(Previous one had a issue)

Issue Number
ISSUE #5
Describe the changes you've made
Added a splash screen along with 2 pages for the vision and mission section.
Describe if there is any unusual behavior of your code(Write NA if there isn't)
NA
Additional context (OPTIONAL)

Test plan (OPTIONAL)
NA
Checklist

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.
[x] The title of my pull request is a short description of the requested changes.

@gharshg29 resolve conflicts


Create the function to calculate GDD

You've assigned this task to both @GulamHaqqani and @David406.  Usually, tasks with multiple assignees make it difficult who is really responsible for the task.

I'm also gonna work on this part, otherwise I can't start secondary tasks.

I finished this code, please review and checked it out!

the script works fine in calculating accumulated GDD. Can you please check the naming of the output file. Because when the "prefix" variable is empty, the output file name comes out as "-CumGDD.csv" which couldn't be opened in linux as the "-" is treated as some kind of option for a command.

Ok, I'll fix this


new: GitLabApi.create(url, user, pass)

force updated: javadocs added; exception thrown as is (not wrapped)


TODOs

@mpolden and @bratseth please validate intent and plan.

Duplicate the instance part of each write of an Application to a new instance path.
Switch to reading the instance part of an Application from the new instance path.
Stop writing the instance part of each Application to the application blob.
Split Application and use Instance where applicable.

These steps will have to roll out separately. (Well, I think the first two and the last two could be merged, if done properly, but it's easier to convince oneself that this is safe.)

Today we only write the instance path, and Application is the instance, so i think this should start with

Duplicate the common application part of each write of an application instance to a new application path.

etc. Perhaps there should be a 0. Rename Application to Instance.

I agree, conceptually, of course, but the current Application is much more similar to an application than it is to an instance, so that would lead to more code changes. It may still be the right thing to do, though ...

All right, I'll try that approach.


Add addJvmOverloads support to Input types

Version
v3.8.2
Summary
Problem
Adding a new nullable field to the schema is a “safe” operation from the point of GQL Schema checks, but it is not safe on the client side since the binary of the generated code changes. This may result in NoSuchMethodExceptions at runtime.
Example:
input Login {
  username: String!
  password: String!
  rememberMe: Boolean // A new Optional field is added
}

The input type above generates the following data class:
data class Login(
    val username: String,
    val password: String
)

If we modify the schema,
input Login {
  username: String!
  password: String!
  rememberMe: Boolean // A new Optional field is added
}

the generated data class constructor changes
data class Login(
    val username: String,
    val password: String,
    val rememberMe: Optional<Boolean> = Optional.Absent
)

When precompiled libraries call Login("user", "1234"), they will receive `NoSuchMethodException.
Proposal
We should extend the functionality added with #3848 and add addJvmOverloads support to Input types. Adding this support to Fragment Data Classes would be nice as well, but I don't know how this would work since they don't have default values. Possibly, we can add a flag that defaults nullable fragment arguments to null.
Considerations

Adding @JvmOverloads will not make it fully binary compatible. This only works if you add the new parameter to the end of the constructor. Since the schema dictates the order, this will be difficult to control on the client side but can be handled when designing the schema.
Even if you add the argument to the end, the copy() function will not be binary compatible. However, this may not be a problem for the majority of the use cases.
In v4, generateInputBuilders can be used to mitigate this. Can generateInputBuilders be extended to Operations and Fragments as well?

Steps to reproduce the behavior
No response
Logs
(Your logs here)

Good catch!
See also https://github.com/apollographql/apollo-kotlin/issues/4659 for a similar issue with argument order.
I think you're 100% right that addJvmOverloads should also apply to Input types. A PR would be very welcome!
In the longer term, I'd aim for generateInputBuilders as it seems to be a more robust solution to that problem space. Builders have been around in Java for as long as I can remember and they also have the advantage of being able to express null vs absent (see https://github.com/apollographql/apollo-kotlin/issues/4747).
To your question, I think generateInputBuilders also applies to operations and fragments. The "input" is more about "input value" than "input type". There is still time to bikeshed this name though as it's never been released in a stable version. Any suggestion?
Even with generateInputBuilders, there is still the question of required arguments order (see https://github.com/graphql/graphql-spec/issues/1062). One way to workaround would be to enforce named parameters (see https://youtrack.jetbrains.com/issue/KT-14934)

+1 to input builders and making constructors private.
I see, it makes sense to call it "input" but it may sound confusing. Maybe generateBuilders? If not, I think it is okay as it is. We can explicitly call this out in the documentation.
I don't see how (https://github.com/graphql/graphql-spec/issues/1062)[https://github.com/graphql/graphql-spec/issues/1062] would cause issues with builders. Maybe I'm missing something.
Another option to consider is to move away from data classes to fully fletched classes for better maintainability as outlined in this article.

If you have 2 mandatory parameters in your Builder constructor and they are the same type, changing the order could break the app at runtime without breaking build time

I haven't used the input builders yet so I'm not familiar how it works internally, but when I get a chance and when we start the spike for migration I will take a look.
The more we look into this, the more problems are surfacing. We discovered that @JvmOverloads works for Java clients but doesn't provide backward compatibility for Kotlin clients. The issue is reported here.
For Java clients @JvmOverloads may work but for Kotlin clients I think builders will be a much better option going forward.
Example:
Let's say you have this data class in v1 and FooTester precompiled with v1
data class Foo @JvmOverloads constructor(
    val a: String = "a",
    val b: String = "b",
)

class FooTester {

    val validCase1 = Foo(a = "a", b= "b")
    val validCase2 = Foo("a", "b")
    val validCase3 = Foo(a = "a", "")

    val failureCase1 = Foo("a")
    val failureCase2 = Foo(a = "a")
    val failureCase3 = Foo()
}

You add a new parameter c in v2.
data class Foo @JvmOverloads constructor(
    val a: String = "a",
    val b: String = "b",
    val c: String = "c",
)

When the library gets updated transitively failureCase1, failureCase2, failureCase3 will throw NoSuchMethod error.
Cause
If you decompile to Java, you will see those constructors are calling the synthetic constructor:

And the signature of the synthetic constructor changes between two versions:
From:

To:

Makes a lot of sense 👍 . Good old Builders will live forever! :).
I still think we should add the @JvmOverloads to input object types for consistency as long as we support addJvmOverloads
Re: generateInputBuilders, there are the questions of:

naming
wheter we enable them by default
whether we make required argument parts of the Builder constructor

If you get a chance to enable generateInputBuilders and let us know what you think, that'd be a great data point to decide whether or not we should make them the default.

V4 is now out, which includes generateInputBuilders. If you have a chance to enable it, please don't hesitate to give us your feedback on it.

FWIW, I changed my mind on @JvmOverloads. The more I look into this, the more I think generateInputBuilders should be the default so I'd rather not invest too much on the other thing.

Yes, I agree with the approach. JvmOverloads only helps with Java, not Kotlin. I will close this.


Inconsistent documentation regarding supported input data type and ParameterServerStrategy

Issue Type
Documentation Bug
Current Behaviour?
There is a direct conflict between the ParameterServerStrategy tutorial doc, which states:
Keras Model.fit with tf.distribute.ParameterServerStrategy can take input data in the form of a tf.data.Dataset, tf.distribute.DistributedDataset, or a tf.keras.utils.experimental.DatasetCreator, with Dataset being the recommended option for ease of use.

And these pages:

ParameterServerStrategy:

When using Model.fit, tf.distribute.experimental.ParameterServerStrategy must be used with a tf.keras.utils.experimental.DatasetCreator, and steps_per_epoch must be specified.

Distributed Input:

Users of tf.distribute.experimental.ParameterServerStrategy with the Model.fit API need to use a tf.keras.utils.experimental.DatasetCreator as the input

Model.fit:

If using tf.distribute.experimental.ParameterServerStrategy, only DatasetCreator type is supported for X

The former recommends using Dataset while the latter explicitly states only DatasetCreator is supported.

@bli00
In order to expedite the trouble-shooting process, could you provide more information on this issue ?
Thanks!


GPU statistics are not recorded in OpenSource version of YTsaurus

For some reason, GPU statistics are not recorded despite the working GPU info provider.
Sasha or I will look into it.

Oh, it was misconfiguration from my side. Everything works well.


All the sudden it starts crashing after using "click to move" in game

Acknowledgement of preliminary instructions

[X] I have read the preliminary instructions, and I am certain that my problem has not already been addressed.

What problem did you encounter?
I dont know if its the client or my PC but, I run Windows 11 and when I used click to move, the game randomly crashed. My computer sent me a notification saying "⚠_Roblox has crashed, please preform a clean boot_" then I did the clean boot like 5 times and still kept crashing.

As the preliminary instructions asked, does this happen only when using Bloxstrap? https://github.com/pizzaboxer/bloxstrap/wiki/Switching-between-Roblox-and-Bloxstrap

Issue has been awaiting clarification for over a month, closing as stale. If you have anything further to add, just respond and I'll reopen.


Build fix: specify template parameters

Clio doesn't build on Ubuntu/g++ because of failed template deduction. This PR contains the fix.

This change is

Can you run clang-format on this? We're using version 10


Cannot find namespace '_ZoteroTypes'.

When I import the zotero toolkit, I'm getting errors on missing namespaces like
node_modules/zotero-plugin-toolkit/dist/basic.d.ts:25:40 - error TS2503: Cannot find namespace '_ZoteroTypes'.

25     getGlobal(k: "Zotero" | "zotero"): _ZoteroTypes.Zotero;
                                          ~~~~~~~~~~~~

my tsconfig is
{
  "compilerOptions": {
    "declaration": false,
    "allowUnreachableCode": false,
    "importHelpers": true,
    "target": "es2017",
    "disableSizeLimit": true,
    "module": "commonjs",
    "noImplicitAny": false,
    "removeComments": false,
    "preserveConstEnums": false,
    "sourceMap": false,
    "downlevelIteration": true,
    "lib": [ "es2019", "dom", "ES2021.WeakRef" ],
    "types": [ "node" ],
    "typeRoots": [ "./node_modules/@types" ]
  },
  "include": [ "bootstrap.ts", "content/folder-import.ts", "content/globals.d.ts", "node_modules/zotero-types" ],
  "exclude": [
    "node_modules",
    "**/*.spec.ts"
  ]
}

{
  "compilerOptions": {
    "declaration": false,
    "allowUnreachableCode": false,
    "importHelpers": true,
    "target": "es2017",
    "disableSizeLimit": true,
    "module": "commonjs",
    "noImplicitAny": false,
    "removeComments": false,
    "preserveConstEnums": false,
    "sourceMap": false,
    "downlevelIteration": true,
    "lib": [ "es2019", "dom", "ES2021.WeakRef" ],
--- "types": [ "node" ],
--- "typeRoots": [ "./node_modules/@types" ]
+++ "skipLibCheck": true
  },
  "include": [ "bootstrap.ts", "content/folder-import.ts", "content/globals.d.ts", "node_modules/zotero-types" ],
  "exclude": [
    "node_modules",
    "**/*.spec.ts"
  ]
}

Is there a way without skiplibcheck? I have some libraries that offer typings, I'd prefer to keep those. Right now I just remove all d.ts files from the plugin toolkit.

The gecko types used by the package zotero-types conflict heavily with the built-in TS types, such as Window, Document, and all kinds of DOM things. This package relies on these types.
I'm not sure if there is a way to let the TS server recognize these TS definitions without skipping the lib check... I tried some suggestions from LLM but had no luck.

I don't follow. The problem exists because this package pulls in zotero-types? Geen why not remove that dependency?

Actually I didn't see this error if I set skipLibCheck to false, probably because I have node_modules/zotero-types included in the project's tsconfig. But other ts errors are mostly from the cause I mentioned above:
Found 1404 errors in 8 files.

Errors  Files
   121  node_modules/@types/react/index.d.ts:4312
    81  node_modules/typescript/lib/lib.dom.d.ts:23
     1  node_modules/zotero-types/types/gecko/index.d.ts:24
  1184  node_modules/zotero-types/types/gecko/lib.gecko.dom.d.ts:9
    13  node_modules/zotero-types/types/gecko/lib.gecko.tweaks.d.ts:25
     1  ../zotero-plugin-toolkit/dist/helpers/virtualizedTable.d.ts:1
     1  ../zotero-plugin-toolkit/dist/tools/ui.d.ts:335
     2  ../zotero-plugin-toolkit/node_modules/@types/react/index.d.ts:4318

Geen why not remove that dependency?

Well, the zotero-types provides TS definitions for both Zotero and Gecko platform (fetched from Mozilla's) and it's helpful for development in Zotero's environment... Unless you want to duplicate all that again manually, otherwise I don't see any reason to remove it.

Just updated all dependencies and now conflict should only exists between node_modules/typescript/lib/lib.dom.d.ts and node_moduels/zotero-types/gecko/lib.gecko.dom.d.ts

Just checked the https://github.com/retorquere/zotero-folder-import, it's because the zotero-types is not visible to the TS compiler. If you add "node_modules/zotero-types" to include and remove "node_modules" from exclude, it should be recognized; but still, the gecko types conflict with the TS builtin ones.

I've updated but I still get lots of errors around Cannot find namespace '_ZoteroTypes' and Cannot find name 'XUL...Element'.

Unless you want to duplicate all that again manually, otherwise I don't see any reason to remove it.

I'd prefer not to duplicate anything manually, but adding zotero-types still leaves me with the _ZoteroTypes and XUL...Element errors.
I currently just run
find node_modules/zotero-plugin-toolkit -name \*.d.ts -exec rm {} \;

before my build and then I can proceed.

The gecko types used by the package zotero-types conflict heavily with the built-in TS types, such as Window, Document, and all kinds of DOM things. This package relies on these types. I don't know if we can allow lib-checking on these dependencies.

Maybe I just don't understand what this says. It sounds like your lib depends heavily on zotero-types? But how does that cause these _ZoteroTypes errors?
But if there's nothing that can be done about this then that's just how it is. Setting skipLibCheck seems like the nuclear option to solving this problem though.

Just checked the https://github.com/retorquere/zotero-folder-import, it's because the zotero-types is not visible to the TS compiler. If you add "node_modules/zotero-types" to include and remove "node_modules" from exclude, it should be recognized; but still, the gecko types conflict with the TS builtin ones.

Ah, so I had effectively set skipLibCheck? Weird, I thought I was getting typings from zotero-plugin. But you're right, the _ZoteroTypes errors are gone but replaced by countless other errors that are likely from the clash with the gecko types.

My build runs when I remove node_modules from exclude (and remove all d.ts files from plugin-toolkit). If that's the best that can be achieved then so be it.

We can try to somehow... fix these gecko errors, maybe manually fix them all and generate a patch file so that hopefully the next time gecko updates something we don't need to do it again. (And presumably we don't need to update gecko types frequently)
Another option would be to include a copy of lib.dom.d.ts along with gecko's dom.d.ts , with @ts-nocheck, and ban the default DOM lib from the tsconfig.
I don't know if there are better options... What do you think?

I've been battling the same problems, I've used forced-typecasting in a lot of places. Another problem I've had is that I want to have Zotero.<plugin name> to be typed, and for that I need to re-type the Zotero object.

I've been battling the same problems, I've used forced-typecasting in a lot of places.

If we replace the default dom.d.ts, it should be fine without modifying anything. All/most of the errors from gecko are duplicate declaration, and the TS server can tolerate this.

Another problem I've had is that I want to have Zotero. to be typed, and for that I need to re-type the Zotero object.

Currently the Zotero is defined as a const variable in zotero-types, but it's possible to make it a module and then you can extend it elsewhere. Currently I am injecting a reference to this plugin instance to the globalThis of the plugin sandbox so that plugin code can access it with types.

TS server

what is a TS Server?

Currently the Zotero is defined as a const variable in zotero-types, but it's possible to make it a module and then you can extend it elsewhere. Currently I am injecting a reference to this plugin instance to the globalThis of the plugin sandbox so that plugin code can access it with types.

That seems desirable

what is a TS Server?

I meant to say the typescript language server that analyzes and compiles TS code

Is this something related to language servers/visual-studio type editors? I'm a command line guy, I invoke tsc to compile and use vi for editing.

They should work the same way -  but maybe that's how it's called in IDEs like vscode, idk. Anyway, I'll make some observations in that direction.


Add num_partitions to Blockstore rewards

Problem
When the partitioned rewards feature is activated, the only way for RPC and clients to find the rewards for a specific address will be to iterate through blocks at the beginning of each epoch. This is because we do not persist information about how the rewards were partitioned beyond the current epoch. In fact, we can't even tell how long such an iteration will take without knowing how many blocks the rewards distribution spans.
On the abandoned SIMD-106, we ultimately concluded that the best way to provide this data long-term would be to record it in the ledger.
Summary of Changes
Add num_partitions field to protobuf generated::Rewards
Populate num_partitions; it will only be Some for the epoch-boundary Bank after partitioned epoch rewards have been activated.
This will enable RPC and other client services to find stake rewards by:

Pulling the rewards and metadata from the first block of the epoch
Generating an EpochRewardsHasher from the parent_blockhash in the metadata and the num_partitions in the PartitionData reward

Closes #1322

@jstarry , can you please take a brief look at the direction in this PR and let me know if it's what you envisioned, or whether you see any red flags? Then I'll put it through its paces -- to/from Blockstore and Bigtable before and after the field add.

Yeah this is what I invisioned

Side note: https://github.com/anza-xyz/agave/pull/1677 contains the RPC and test-validator changes to facilitate testing

Pushed recommended changes; will push some commits with more tests shortly.

@jstarry , sorry for the delay. Added tests of new runtime methods.


Non-default install location doesn't work, default hard-coded in some places

Describe the bug
Installation seems to go ok, then on launching:
[...]
Saving to: ‘/tmp/fusion360/build-version.txt’

build-version.txt               100%[====================================================>]      20  --.-KB/s    in 0s

Last-modified header missing -- time-stamps turned off.
2022-09-21 17:20:54 (2.44 MB/s) - ‘/tmp/fusion360/build-version.txt’ saved [20/20]

Online Build-Version: 2.0.13168
Online Insider-Build-Version: 2.0.13367
The version.txt file not exist!
$REAL_INSTALL_PATH/bin/launcher.sh: line 71: $HOME/.fusion360/bin/update.sh: No such file or directory

To Reproduce
Steps to reproduce the behavior:

Install to non-default location

Expected behavior
Installs ok, launchable, uninstallable

e.g.
https://github.com/cryinkfly/Autodesk-Fusion-360-for-Linux/blob/6920f089432345f5c396fd6f144ebc6a80a1e42d/files/builds/stable-branch/bin/launcher.sh#L71
https://github.com/cryinkfly/Autodesk-Fusion-360-for-Linux/blob/6920f089432345f5c396fd6f144ebc6a80a1e42d/files/builds/stable-branch/bin/update.sh#L19-L27
https://github.com/cryinkfly/Autodesk-Fusion-360-for-Linux/blob/6920f089432345f5c396fd6f144ebc6a80a1e42d/files/builds/stable-branch/bin/uninstall.sh#L21-L30


DO NOT MERGE: Breaking changes for Lit 2

This PR makes the breaking changes required to update to Lit 2 (lit-html 2 and lit-element 3). The Lit 2 upgrade will be a coordinated effort, so this PR should not merge at this time.
Breaking change checklist:

[x] Update lit-html -> ^2, lit-element -> ^3, open-wc/testing -> ^3 (if used)
[x] createRenderRoot usages -> move from LitElement to ReactiveElement
[ ] Remove Dependabot Lit rules
[x] Bump any dependencies to their Lit 2 versions
[x] Remove any manual lifecycle calls to Lit 1 reactive controllers
[x] Backwards-compatible changes merged: (https://github.com/BrightspaceUI/core/pull/2034) (no changes required)

Renamed API's:

[x] UpdatingElement -> ReactiveElement
[x] @internalProperty -> @state (we don't use TypeScript decorators)
[x] static getStyles() -> static finalizeStyles(styles)
[x] _getUpdateComplete() -> getUpdateComplete()
[x] NodePart -> ChildPart

Testing checklist:

[x] CI/ unit tests pass
[ ] Local testing on demo pages
[ ] Testing component in the LMS

For more information on the Lit 2 upgrade, visit https://lit.dev/docs/releases/upgrade/ or contact team Gaudi.

This all looks good to me!  Didn't approve just so it's harder to accidentally merge early lol, but let me know if you want me to


ruler: how about implements the Prometheus remote write feature into thanos ruler?

I want to use both Thanos receive and Thanos ruler components, but because Thanos ruler only store rules result in local and ship the data to remote storage like s3, the ruler node will be a single node with HA risk.
In my opinion, I think if Thanos ruler implements the remote write feature, we can write the ruler data to Thanos receive with few seconds delay, then Thanos ruler would be a stateless component (of course if we can accept minutes data loss maybe can reduce by rule failover).

The state of the ruler wouldn't disappear. The tsdb would continue to need to be a local persistent buffer. I don't think we would actually gain anything from this unfortunately.

You should run the thanos ruler twice an identical pair if you are looking for HA probably.

@brancz yes the state wouldn't disappear, but this will help to set ruler as more lighten component right? I think this situation similar to the relationship between Prometheus and Thanos receive, the Prometheus will not act as a fully stateless component but we can treat it stateless because of Thanos receive.

@FUSAKLA yes, I thought like this before, but there are some disadvantages I think:

calc an identical pair twice or more will increase the system load.
fan out load on Thanos query side? data will store on the ruler component and we have to add Thanos ruler in Thanos query.
not simple enough for Thanos ruler, I think Thanos ruler focus on query -> exec -> write in the whole system, and Thanos receive and Thanos store act as query components, these sounds more clear, is it?

Yes, the HA always comes with some price. It will be calculated twice but it's similar to Prometheus HA where you do the scraping and evaluation twice as well. But I wouldn't say adding the ruler StoreAPI to the cluster is that much of a burden and even less if you use service discovery. Still, the remote write won't help you with HA.
But I don't see reason to not have the remote write in the ruler eventually. I agree it can make the component more lightweight same way as for a Prometheus with sidecar. But the benefits are now quite small IMHO.

I think the requirements should quite similar to why we create Thanos receive component.
Thanos had sidecar, query and store components before, these components also work well, and looks like already making Prometheus with distributing feature. but we create receive component now, I think it will be a little conflict with sidecar and query components, however, we had created it, and the concept looks like more clear and meaningful.
In my opinion, we can separate the components into several parts in the whole architecture as below:

generate: Prometheus and ruler
store: remote storage and compact
read: receive and store

each part should focus on the main function is provided, meanwhile, the same function should keep unified and we can improve it continues. for example, write to remote storage should done by receive component, raw data (include collect from exporter and calc from rules) should delivery to receive component asap, hot data query will be served in receive but not any other part.
so here is why I want to ruler implements the remote write feature, I think it will make the system overview more clearly and keep each part simple.

Running and scaling receive is somewhat involved I would hate for that kind of involvement to be necessary to use the component when people just use the sidecar approach and otherwise don’t need the receive component.

For now I’m going to close this. We might rethink this at a later point, but at the moment we don’t feel this is the right strategy for the project. Thanks a lot for starting the discussion! :)


fix: back to GH token

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Run tests in Travis

Can we use Travis CI to run mvn test -P allTests? https://github.com/bonej-org/BoneJ2/pull/283 illustrates that we are not using Travis CI to run the tests.

Think the idea is that we don't run the "slow" (annotated with @Category(org.bonej.wrapperPlugins.SlowWrapperTest.class) tests on Travis - as they tend to time out from time to time, indicating a bug when there is none. It's documented in the wiki (pt 5.) to check locally with -P allTests when we release the code... although probably good to do this more often than that.
I agree that the current situation is far from ideal wrt these issues. It might be resolved by switching to GitHub actions (who offer machines with better specs and longer time-out limits (although I wouldn't count on this - we should consider this if the whole of ImageJ switches to GitHub actions, which is a possibility apparently). So maybe it's worth experimenting with? I don't know

I wonder if this has something to do with #300 ?

In what way do you mean, @mdoube ? Are you suggesting that all wrapperPlugins annotated with @Category(org.bonej.wrapperPlugins.SlowWrapperTest.class) suffer from the same zombie thread business, and are too slow to not time out on Travis for this reason?

Closing, testing and CI has moved to Github Actions


Version 1 0

Pull Request to track changes. Not final yet.

Merging this branch due to the increasing amount of commits for this version. Documentation for version 1.0.0 will be finished on the development branch.


Join game view

User should be able to join available games.

list of opened games
user join game

Pull request opened #22

New commit and pull request #23