Discussion on the native support for BN254 curve

After the successful completion of addition of BLS12-381 curve, we want to start a discussion on addition of BN254 curve. We've already submitted a CPS proposal, and started a community forum discussion. While the forum post is open for comment, it makes sense to start a discussion here to reach a bigger audience and more stakeholders so that meaningful development of Zero-Knowledge powered applications on ICON is not postponed.
Implementation
We have utilized the code from the BN128 codebase of the EthereumJ client and have achieved a basic integration. Note that this is just for PoC and is not the final form.
In the final form, we intend to incorporate comprehensive trusted tests for curve operations sourced from Google  and Cloudflare to ensure the code's quality and reliability. These tests have helped maintain the security of curve operations on Ethereum and thus will help validate the correctness and performance of the curve operations, ensuring that the integration meets the desired standards.
Unless a bug is found on Ethereum's implementation, once released, our implementation would not require any changes. Given how long the code has been running on Ethereum, the chances of the bug is pretty unlikely.
Benefits of the BN254 Curve: Community Base, Tooling Support, and Circuit Compatibility
BN254 curve boasts a vast community base and robust tooling support, making it an attractive choice for numerous projects. The widely used circomlib library, for instance, retains all its circuits and Javascript client library compatible with the BN254 curve. Notably, this library capitalizes on the baby jubjub curve, which possesses a unique property: its base field aligns with the scalar field of the BN254. This alignment enables the native SNARK computation to be leveraged for an efficient implementation. Consequently, various crucial functionalities, such as the implementation of Pedersen hash/commitments, Poseidon hash/shared_key_encryption, EdDSA signature verification on a snark circuit, and other use cases, have been made possible.
Accelerating Privacy-Preserving Protocols and Applications in the ICON Network: Leveraging the BN254 Curve
The constructs discussed above serve as the foundation for major privacy-preserving protocols (like Semaphore) and applications (like Railgun). With BN254, bringing these into the ICON network would only require minimal changes to the frontend and smart contracts, as the circuits and utility libraries can be reused. Without the BN254 curve, however, developers would need to implement some of these circuits from scratch, which presents significant challenges. Writing circuits requires an in-depth understanding of mathematics and cryptography, making it an expensive, complex, and time-consuming task. Moreover, starting from scratch introduces a higher risk of potential bugs.
BN254 Curve Adoption: Security and Reliability in Practice
BLS12-381 indeed provides a better security of 120 bits over 100 bits for BN254. However, the BN254 curve is secure enough that many-large scale projects still rely on it. These include:

Polygon zkEVM: The alt_bn128 curve is employed within the Polygon zkEVM implementation.
zkSync: The BN254 or altBN128 curve is used for the final commitment verifier in zksync/VerifierTemplate.sol.
Loopring: The bn128 curve is employed in protocols/Verifier.sol within the Loopring project.
Scroll zkEVM: The bn254 curve is utilized in the PLONK verifier within the scroll-zkevm repository.

Conclusion
While it is anticipated that these projects will eventually transition to the BLS12-381 curve as tooling support improves, the current landscape suggests that this migration process will require a substantial amount of time. By incorporating the BN254 curve into the ICON network, we aim to facilitate a seamless development experience for dApp developers and enable them to build new applications or repurpose existing ones, while leveraging the available resources. It is worth noting that working with Zero Knowledge protocols can be challenging, even for experienced developers. Therefore, it is imperative that we prioritize simplifying the initial steps for developers to embark on their Zero Knowledge journey. Should individuals desire enhanced security, they can always opt to switch to utilizing the BLS12-381 curve.

Hi @sink772,
Did you have some time to review this, it'd be great to continue the discussion. We have some comments in our CPS proposal, where validators seems to want a participation from the goloop team.
https://cps.icon.community/proposals/bafybeicrrpvx7qiwswx7omhibunm67vaiowlhimb55vsfv7fonu7skfnry


[5.7] Use ::class notation for Container tests

Use short ::class notation for Container tests

You can just send the rest all at once.

@taylorotwell OK no prob 🙂
Wanted to do them module by module so it's easier to review 🙂


New plugin: codeclimate

https://github.com/codeclimate/SublimeLinter-contrib-codeclimate

Sorry for the slow response time! Please fix your TravisCI link in the README, make a release, and I'll put it in Package Control. Thanks!

Hi @groteworld -
No problem. I've fixed the TravisCI link and tagged a release. Let me know if you need anything else.
Thanks for this project!

Sorry for my speed! Should show up within an hour in Package Control. Thanks Again!
https://github.com/SublimeLinter/package_control_channel/commit/dbe1bad4a36f0934e46f45036bda67b97f981cfa


[test] Integration Tests Windows

Description
This PR closes https://github.com/microsoft/demikernel/issues/1030.

Merging this as it concerns improvements on the CI.


Update README.md

Remove the disclaimer as this project has now moved to https://github.com/dart-lang. @kevmoo @kwalrath

Coverage remained the same at 81.858% when pulling 5deda9704dffdfe034fbf248957243404311d843 on devoncarew-patch-1 into 231a096da25b84735ce23471a07190c1460085a8 on master.

Good catch.


Enhanced ingress configuration on anchore-engine chart

What this PR does / why we need it:
This PR allows a better way of managing ingress configuration of the ingress for the anchore-engine chart.

Hi @mvisonneau, I should have time to review this today or tomorrow and will update then. Just letting you know that it's on my radar and I'll get to it very soon. Thanks!

/ok-to-test

👋 thanks for the review @zhill. I updated accordingly and answered specific points under your comments.

/lgtm


'user-agent: okhttp/4.10.0\r\n' +
      'Content-Length: 577\r\n' +
      'Accept-Encoding: gzip, compress, deflate, br\r\n' +
      'Host: authglb.digital.panasonic.com\r\n' +
      'Connection: keep-alive\r\n' +
      '\r\n',
    _keepAliveTimeout: 0,
    _onPendingData: [Function: nop],
    agent: Agent {
      _events: [Object: null prototype],
      _eventsCount: 2,
      _maxListeners: undefined,
      defaultPort: 443,
      protocol: 'https:',
      options: [Object: null prototype],
      requests: [Object: null prototype] {},
      sockets: [Object: null prototype] {},
      freeSockets: [Object: null prototype],
      keepAliveMsecs: 1000,
      keepAlive: true,
      maxSockets: Infinity,
      maxFreeSockets: 256,
      scheduling: 'lifo',
      maxTotalSockets: Infinity,
      totalSocketCount: 1,
      maxCachedSessions: 100,
      _sessionCache: [Object],
      [Symbol(shapeMode)]: false,
      [Symbol(kCapture)]: false
    },
    socketPath: undefined,
    method: 'GET',
    maxHeaderSize: undefined,
    insecureHTTPParser: undefined,
    joinDuplicateHeaders: undefined,
    path: '/authorize',
    _ended: true,
    res: IncomingMessage {
      _events: [Object],
      _readableState: [ReadableState],
      _maxListeners: undefined,
      socket: null,
      httpVersionMajor: 1,
      httpVersionMinor: 1,
      httpVersion: '1.1',
      complete: true,
      rawHeaders: [Array],
      rawTrailers: [],
      joinDuplicateHeaders: undefined,
      aborted: false,
      upgrade: false,
      url: '',
      method: null,
      statusCode: 302,
      statusMessage: 'Found',
      client: [TLSSocket],
      _consuming: false,
      _dumped: false,
      req: [Circular *1],
      _eventsCount: 4,
      [Symbol(shapeMode)]: true,
      [Symbol(kCapture)]: false,
      [Symbol(kHeaders)]: [Object],
      [Symbol(kHeadersCount)]: 42,
      [Symbol(kTrailers)]: null,
      [Symbol(kTrailersCount)]: 0
    },
    aborted: false,
    timeoutCb: null,
    upgradeOrConnect: false,
    parser: null,
    maxHeadersCount: null,
    reusedSocket: false,
    host: 'authglb.digital.panasonic.com',
    protocol: 'https:',
    [Symbol(shapeMode)]: false,
    [Symbol(kCapture)]: false,
    [Symbol(kBytesWritten)]: 0,
    [Symbol(kNeedDrain)]: false,
    [Symbol(corked)]: 0,
    [Symbol(kOutHeaders)]: [Object: null prototype] {
      accept: [Array],
      'content-type': [Array],
      'user-agent': [Array],
      'content-length': [Array],
      'accept-encoding': [Array],
      host: [Array]
    },
    [Symbol(errored)]: null,
    [Symbol(kHighWaterMark)]: 16384,
    [Symbol(kRejectNonStandardBodyWrites)]: false,
    [Symbol(kUniqueHeaders)]: null
  },
  data: 'Found. Redirecting to https://authst-pn.digital.panasonic.com/error/?client_id=Xmy6xIYIitMxngjB2rHvlm6HSDNnaMJx&connection=&lang=&error=invalid_request&error_description=Missing%20required%20parameter%3A%20client_id&tracking=7abfae08443fdf7a807a'
}

it says so in the error message - missing required parameter client_id

it says so in the error message - missing required parameter client_id. this is rather off-topic. may I suggest to open a separate issue where you discuss things that are not related to the HA plugin?

Good. I will do so. But as you can see, "client_id" is given, so I don't understand why such an error. I don't see the difference between data in my code in Javascript and data in this one in Python.

I don't know how the javascript fetch request works, but you need to make sure that the data is actually sent as querystsring parameters (i.e. in uri path) and not body data (which GET requests do not support)

Hey @Agent6-6-6 , @erikgroentje
I have push another release that should hopefully address your setup issues.
As has been mentioned before setting up your devices in HA, you must have completed the 2FA login step using the Comfort Cloud app.
https://github.com/craibo/panasonic_cc/releases/tag/2024.6.2

Hey @Agent6-6-6 , @erikgroentje
I have push another release that should hopefully address your setup issues.
As has been mentioned before setting up your devices in HA, you must have completed the 2FA login step using the Comfort Cloud app.
https://github.com/craibo/panasonic_cc/releases/tag/2024.6.2

Will give it another go and post back

Right here you go...
I've create a release 2024.6.1 on my fork https://github.com/craibo/panasonic_cc with the fix that is working on with my unit.
Please can some users test and respond here if there are any issues. Its not very clean code but if its working well I can clean it up and create a PR for @sockless-coding branch.
If it works for you please add a 👍🏻 to this message so we do not fill up the thread with "it works for me" messages.
@sockless-coding no worries mate, we all get busy. If you're ok with it, I'd like to join you as a maintainer for this. So you keep control you can enable basic branch protection with a review required before merging of any PR. Ideally 3 maintainers would be a good number.
Thank you all.

Works like a charm!!
Installed it, rebooted HA and integration is working again. 👍👍👍👍

Any chance you hard-working geniuses can release a beta version on HA? You guys are our hero's btw!

@craibo - can confirm it works after logging out of app, then logging into app and going through the 2FA authentication. Great work! (to everyone involved).

Hey @Agent6-6-6 , @erikgroentje I have push another release that should hopefully address your setup issues. As has been mentioned before setting up your devices in HA, you must have completed the 2FA login step using the Comfort Cloud app.
https://github.com/craibo/panasonic_cc/releases/tag/2024.6.2

the code currently does force a new login session on startup, something that should probably be avoided once _acc_client_id and a refresh_token are known to not flod the oauth servers?
https://github.com/craibo/panasonic_cc/blob/dc650ec75d1d12c0e9889a7fbcce0c4abb1e5134/custom_components/panasonic_cc/pcomfortcloud/panasonicsession.py#L63

2024.6.2 did the trick for me. 5 minutes ago with 2024.6.1 I got the error "Unexpected error creating device" like in the 24 hrs before but the actual release now recognized all CC devices.
Awesome work!

I agree @heldchen. I am not too worried about flooding, but it takes quite a while to get a new token as there are a lot of calls between client and server.
It might be worth restructuring the underlying python-panasonic-comfort-cloud project to not rely on files (shift the file-handling to the CLI section of the tool, so that the ApiClient does not rely on it).

@sockless-coding here is the PR https://github.com/sockless-coding/panasonic_cc/pull/196
Hi @heldchen
I completely agree, this was the fastest way for me to get it to work.
Second step is to address the multiple logins and use the HA OAuth process which will handle the tokens for us.

Thanks @danielcherubini
I'll look at that now

@craibo I would also like to revisit how your fork uses the cli app... one of the things I am going to suggest over there, is we seperate the underlying API/Auth out of the CLI app and into its own project, so that both the CLI app and this HA Integration can use that directly.

ACs are back in HA .
Thanks for all your hard and FAST team work (in your spare time).
I had some troubles installing the right version - first I instated the one that is in HACS .. that did not work. So here is what I did to get it work again:
Then I went to HACS / Integration and with the 3 little dots (right top corner) I added a custom repository https://github.com/craibo/panasonic_cc/releases/tag/2024.6.2 as type integration.
Then restart HA and modify the configuration.yaml as described and restart again.

So just to get this straight, I currently have installed @sockless-coding latest version of the integration. Do we have to uninstall this version and install @craibo's fork now? Or will the current version/repository in HACS be updated?

Cosa si deve modificane nel configuration.yaml? Mi potresti spiegare tutta la procedura dettagliata per non sbagliare? Grazie mille.

For those wondering how to switch to the other fork. (No you don't need to uninstall the other!)
In the HACS section, click Integrations, then click the three dots on the top right corner.

From there click the Custom Repositories button

Fill in the form like in the screenshot above.

Then you need to find the correct Panasonic integration... it will be the one that looks like this and has @craibo above the title. it will also have a version number like 2024.x.x

From there you just need to click Redownload or something similar.. which gives you this dialogue.. just install and you're gold.

Thanks mates, awesome work! ;)

it says so in the error message - missing required parameter client_id. this is rather off-topic. may I suggest to open a separate issue where you discuss things that are not related to the HA plugin?

Good. I will do so. But as you can see, "client_id" is given, so I don't understand why such an error. I don't see the difference between data in my code in Javascript and data in this one in Python.

I don't know how the javascript fetch request works in details, but you need to make sure that the data is actually sent as querystsring parameters (i.e. in uri path) and not body data (which GET requests do not support). the fetch() api does not do this for you (see https://developer.mozilla.org/en-US/docs/Web/API/fetch). a quick google suggests that it's up to you to construct the path correctly, see: https://stackoverflow.com/a/58437909

Thank you! But still got incorect url error....
Redirecting to /login?state=hKFo2SAtUjBpMlJfYjBJRmZXUUhNVjBubmpwVllyc2tNZ000c6FupWxvZ2luo3RpZNkgNi10UmpNV1BTRXFPN3ljN1Ezb1NLUWxxd0ZKYkYzaWajY2lk2SBYbXk2eElZSWl0TXhuZ2pCMnJIdmxtNkhTRE5uYU1KeA&client=Xmy6xIYIitMxngjB2rHvlm6HSDNnaMJx&protocol=oauth2&scope=openid%20offline_access%20comfortcloud.control%20a2w.control&audience=https%3A%2F%2Fdigital.panasonic.com%2FXmy6xIYIitMxngjB2rHvlm6HSDNnaMJx%2Fapi%2Fv1%2F&response_type=code&code_challenge=yQWzaWah6PI0R83P11hYnmMlh1P6r2H0YV%2FZcqP2DJc&code_challenge_method=S256&auth0Client=eyJuYW1lIjoiQXV0aDAuQW5kcm9pZCIsImVudiI6eyJhbmRyb2lkIjoiMzAifSwidmVyc2lvbiI6IjIuOS4zIn0%3D&redirect_uri=panasonic-iot-cfc%3A%2F%2Fauthglb.digital.panasonic.com%2Fandroid%2Fcom.panasonic.ACCsmart%2Fcallback

thx a lot, work for me too :)

it says so in the error message - missing required parameter client_id. this is rather off-topic. may I suggest to open a separate issue where you discuss things that are not related to the HA plugin?

Good. I will do so. But as you can see, "client_id" is given, so I don't understand why such an error. I don't see the difference between data in my code in Javascript and data in this one in Python.

I don't know how the javascript fetch request works in details, but you need to make sure that the data is actually sent as querystsring parameters (i.e. in uri path) and not body data (which GET requests do not support). the fetch() api does not do this for you (see https://developer.mozilla.org/en-US/docs/Web/API/fetch). a quick google suggests that it's up to you to construct the path correctly, see: https://stackoverflow.com/a/58437909

Thank you! But still got incorect url error....
Redirecting to /login?state=hKFo2SAtUjBpMlJfYjBJRmZXUUhNVjBubmpwVllyc2tNZ000c6FupWxvZ2luo3RpZNkgNi10UmpNV1BTRXFPN3ljN1Ezb1NLUWxxd0ZKYkYzaWajY2lk2SBYbXk2eElZSWl0TXhuZ2pCMnJIdmxtNkhTRE5uYU1KeA&client=Xmy6xIYIitMxngjB2rHvlm6HSDNnaMJx&protocol=oauth2&scope=openid%20offline_access%20comfortcloud.control%20a2w.control&audience=https%3A%2F%2Fdigital.panasonic.com%2FXmy6xIYIitMxngjB2rHvlm6HSDNnaMJx%2Fapi%2Fv1%2F&response_type=code&code_challenge=yQWzaWah6PI0R83P11hYnmMlh1P6r2H0YV%2FZcqP2DJc&code_challenge_method=S256&auth0Client=eyJuYW1lIjoiQXV0aDAuQW5kcm9pZCIsImVudiI6eyJhbmRyb2lkIjoiMzAifSwidmVyc2lvbiI6IjIuOS4zIn0%3D&redirect_uri=panasonic-iot-cfc%3A%2F%2Fauthglb.digital.panasonic.com%2Fandroid%2Fcom.panasonic.ACCsmart%2Fcallback

but that is now the correct url, it does not redirect to the error page anymore but to the login form...

Great! Thank you, it works. I have to delete the old integration? Now I have two of them. Or we will go back to it in the future?

I just wanna say - WOW! What an effort. I will donate.

Great! Thank you, it works. I have to delete the old integration? Now I have two of them. Or we will go back to it in the future?

I think that it's better to work together in the main branch (this one).
Craibo repo is a fork of original one and, if I understand well, it will be integrated in the main branch. It's right to test it but divide the developer into different projects.
@sockless-coding says that I want to clean the code and then open the project to other contributors (craibo? Other?). Is it ok? If yes, I prefer waiting new release in main branch, mantening HA installation clean

excellent summary, thanks @danielcherubini - I completely agree with what you wrote. this addon is very important for a lot of people (as can be seen by the many opened issues and the intense discussions), it would be beneficial for all if the burden of maintaining it does not rest on @sockless-coding alone. for sure there will be future api changes again, the question is only "when", not "if". and as seen in the last few days, many are willing to dedicate precious time to help making it work again.

It works thx very much guys !

Just to add more to what @heldchen said, Panasonic will 100% make changes to their API, and auth again. They will for sure try and kick us off. It's too much work for a single person to bear. Especially one who's said that they don't have much time due to family commitments (which is fine)
I've maintained a highly used open source project for years, and it's a thankless job. But at a certain stage you need to realise time is valuable, and people rely on it, and you need to bring others in.

Thanks a lot to everyone

I'd like to add that while the mode and fan functions now work using 2024.6.4, the temperature dial does not and displays 'Idle - Off' no matter what. My unit is a S-140PE1R5A ducted reverse cycle system.


Feature/integrity analysis apiserver

Description

Addressed Issue

Additional Details

Checklist

[ ] I have read and understand the contributing guidelines
[ ] This PR fixes a defect, and I have provided tests to verify that the fix is effective
[ ] This PR implements an enhancement, and I have provided tests to verify that it works as intended
[ ] This PR introduces changes to the database model, and I have added corresponding update logic
[ ] This PR introduces new or alters existing behavior, and I have updated the documentation accordingly

closed as branch history became muddled


After successful login won't let me run commands

Hi I'm logging in successfully but then it says I can't run any commands cause I am not logged in.
Successfully logged in as: craig
➜  ~  resin drives
You have to log in

@craig-mulligan There was a hot fix for this issue: https://github.com/resin-io/resin-cli/commit/7e5ecd634dae409892e04f49eedb6a3062a04734. Make sure you're running v0.11.1.


Audio timestamp

added extraction of start and end timestamps of audio data to bag_extractor.py

I rebased the single seemingly applicable commit on top of the current master to clean up the commit chaos.


MM-42030: Migrate Threads RHS tour tip to tippy based tour tip

Summary

Ticket Link

https://github.com/mattermost/mattermost-server/issues/19802
Related Pull Requests

Has server changes (please link here)
Has mobile changes (please link here)

Screenshots

Release Note

NONE

Hello @codedsun,
Thanks for your pull request! A Core Committer will review your pull request soon. For code contributions, you can learn more about the review process here.

@codedsun Could you please fix the broken test.

This PR has been automatically labelled "stale" because it hasn't had recent activity.
A core team member will check in on the status of the PR to help with questions.
Thank you for your contribution!

@codedsun Were you able to add the patch?

/update-branch

@jfrerich @jgilliam17 This pr is ready for review

@jgilliam17 Gentle reminder on this one.

/update-branch

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/198144

/update-branch

/e2e-test

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/198266

Thanks @codedsun and also @saturninoabril for fixing the pipeline and running the e2es.
There are few failures related to this PR. @codedsun Please see the report here and let us know if you need help addressing these

This PR has been automatically labelled "stale" because it hasn't had recent activity.
A core team member will check in on the status of the PR to help with questions.
Thank you for your contribution!

closing in favour of https://github.com/mattermost/mattermost-webapp/pull/11107


[#494] Filter out Cochrane reviews that are not for publication

Fixes opentrials/opentrials#494

@georgiana-b Added a new commit simplifying the tests. There were a few unnecessary mocks, and I also used assert_called_with() instead of getting the arguments and checking them "manually". You can see the changes on https://github.com/opentrials/processors/pull/71/commits/9f8f5dc30883a63700f1fcded9c57bd5acd299ac.
Apart from that, all good. The tests are running now and, if they pass, I'll merge it tomorrow morning :+1:


#460 - Memory leak fix.

I investigated issue described in the provided thread. It's easy to reproduce it by creating Project in a following way (this code is copied from Roslyn and than modified to show the point):
public async Task Foo()
{
     string path = @"...some project link";
     var properties = new Dictionary<string, string>();

     for (int i = 0; i < 100; i++)
     {
          var xmlReader = XmlReader.Create(await ReadFileAsync(path, CancellationToken.None).ConfigureAwait(false), s_xmlSettings);
          var collection = new ProjectCollection();
          var xml = ProjectRootElement.Create(xmlReader, collection);
          xml.FullPath = path;

          var project = new Project(
                    xml,
                    properties,
                    toolsVersion: null,
                    projectCollection: collection);
     }
}

Whole memory will be consumed by cached items in ProjectStringCache. There no mechanism of freeing cache data automatically if the references to instances of ProjectCollection and Project classes will be lost. The only way to handle this is to call ProjectCollection.UnloadProject() or ProjectCollection.UnloadAllProjects() manually.
It seems that in Roslyn they ended up with some kind of a workaround by wrapping Project in ProjectFile class with destructor:
~ProjectFile()
{
     try
     {
          // unload project so collection will release global strings
          _loadedProject.ProjectCollection.UnloadAllProjects();
     }
     catch
     {
     }
}

So solution that I propose is pretty much simple. I just implemented Dispose pattern for ProjectCollection to clear all this stuff on GC.

This looks great, and thank you for putting in the effort to try to fix it.
However, it also looks very scary! Particularly the comment:
/// Because of the strongly typed list, some ProjectRootElement's will be held onto indefinitely. This is an acceptable price to pay for
/// being able to provide a commonly used ProjectRootElement immediately it's needed. It is mitigated by the list being finite and small, and
/// because we allow ProjectCollection.UnloadAllProjects to hint to us to clear the list.

Because we don't know the implications, this is definitely out of scope for Update 2 (or probably 3) unless we have a really solid understanding of 1) why is this list not finite and small as suggested and 2) what implications calling ProjectCollection.UnloadAllProjects will have when in a long running process (like Visual Studio IDE) that may still need project references even after the finalizer may run. This change also seems to undo a lot of the logic defined in the ProjectRootElementCache, and it looks like a lot of thought went into it. Is there an issue with the weak/strong reference implementation? Would it be better simply without the cache or that particular implementation? Without knowing the answer to those questions, we can't take this change for VS2015. We would like to have this issue addressed in Dev15 though, so I will leave this open until we can do some further analysis.

@AndyGerlicher thank you for quick and clear answer. Despite my fix currently affects ProjectRootElementCache the real issue is in ProjectStringCache. So I tried to find a safer solution. And after doing more testing I see some option how to resolve this.
ProjectStringCache purpose is to intern strings for loaded XmlDocuments to use less memory. But in fact it uses more than need. See below screenshots from memory profiler. I executed example code provided above to load same project file 1 and 100 times in a loop.

On second screenshot you can see that strings is really interned but it also keeps references to every instance of created XmlDocument (even loaded xml doc is same). The reason of this because XmlDocument.GetHashCode() implementation return hashcode of object instance, it's not overridden.
When I removed string internation for testing purposes, obviously memory leak was resolved. And also it works faster while loading and unloading projects simultaneously.
So the issue can be resolved quickly by removing internation and as second option override XmlDocumentWithLocation.GetHashCode() to compare xml entirely. Not sure what is better option here from perf perspective.
What do you think about it? Can we go with one of them? Or you have other thoughts.

@heXelium Thanks for looking in to this! For now, we're going to close this pull request since we don't have a clear direction on this fix.

I looked into this and ran into a similar situation where Visual Studio ends up taking more and more memory due to this.
Taking memory snapshots I'm seeing where 500MB of memory was being used purely by the ProjectStringCache._documents dictionary.
The problem is indeed that the projects aren't getting unloaded and ejected from this cache properly and the XmlDocument (used as the key) still has a reference to it.
So the end result is that every build causes a memory leak -- with build imports like Microsoft.Common.CurrentVersion.targets utilizing 247k, that adds up pretty quickly (and completely destroys the benefit of interning the strings).
I think a better solution might be to use XmlDocumentWithLocation.FullPath as the key to the dictionary (and not interning strings where that is null to account for when the project hasn't been loaded from disk) or to use something like ConditionalWeakTable<> to fix this huge memory leak.


Add an optional watchdog timer to Logstash

We had some stability issues with Logstash in our environment.
Occassionally Logstash would deadlock, and Monit wouldn't detect it
because it only checks if the process is still running.  We needed a way
to minimise the loss of production logs while we were debugging and
fixing the issue, so I added this watchdog timer.
I originally tried implementing it with some extra checks in
Monit, but ended up adding a separate monitoring job because Monit
doesn't support the kind of logic that's needed (at least not in the
very old version that's running on the standard BOSH Linux stemcells).

BTW, it would be really helpful to have some more documentation in this repo.  Here's how it looks from a contributor right now:  There's no guide for contributors, and it isn't obvious from the root directory that there are tests in src/logsearch-config.  I tried to figure out how the tests work, and the stuff in ci/ turned out to be a red herring.  It's obviously not used any more because it's all broken (e.g., this script doesn't even exist now: https://github.com/cloudfoundry-community/logsearch-boshrelease/blob/develop/ci/tasks/test-filters.sh#L5).  I got weird errors trying to install the test dependencies manually in my own environment, so I ended up using the docker image (andreikrasnitski/logsearch-pr) that gets referenced in the test failure logs.

Hi @sarneaud,
sorry for not looking into this issue for a long time.
We just upgraded Elastic stack to v6. Could you please check if the issue is still actual?
If so - could you please do rebase.
Thanks.

closed due to inactivity


Readme should use --self-contained when converting to HTML

It's best to add the pandoc --self-contained flag when generating html output:
pandoc --lua-filter=pangamebook.lua --self-contained -o example.html example.md
Otherwise the smart quotes pandoc automatically generates do not get the appropriate encoding, and in general the output looks pretty raw.
If you do this, you should also be able to remove the title header line from example.md (leaving it in the yaml header).  The redundant titles are a bit confusing.
P.S. I tested on Mac.  Everything else seemed to work fine.

Done. Also did this for the org-mode example (had to add meta-data to set title there). This makes the PDF output look better as well, without duplicate title. Thanks!


Request: 128 height worlds for Forge dimensions

For Forge dimensions add an option to create a 128 height worlds for nether/end/underground dimensions.

It would be cool if you could create a 255 Nether - there is a Spigot plugin that does that called TallNether (https://www.spigotmc.org/resources/tallnether.22561/) and the The Beneath also used 256 high generation. However, MC said it was going to create performance problems with OTG.

Yeah there's a a good reason that Mojang made the Nether only 128 tall. Unfortunately MC gets really laggy the taller the world is, so you can have a few mountains in an overworld but MC just cannot handle an entire world with height 128 filled with blocks (like an underground world).

Yeah, but TallNether and The Beneath have no performance problems

OK, I chatted with the creator of NetherEx and he also abandoned the tall nether after doing experiments with it.

For 1.17(?), we're expecting changes to world height anyway, will see if we can make it fully configurable.


Carthage support

Support Carthage dependency manager.

Many thanks.

I have merged your pull request.
But when I'am trying out with Carthage, it says "Dependency "Cupcake" has no shared framework schemes". Why does this happen?

Sorry for my late answer.
It's because when carthage checkout your repository the Cupcake.xcodeproj not appear, so carthage didn't find the shared scheme from this xcodeproj.
If I use it from my fork it's working seamlessly.
Maybe it's because your last tagged version is 1.2.1 and that tag doesn't contain the Cupcake.xcodeproj.
Try to create a new tag from the current master branch.

If you specify the master branch in the Cartfile it's working.
github "nerdycat/Cupcake" "master"
So it's because if  you don't specify the branch or version, carthage cloning the last tagged version (your 1.2.1) which doesn't contain the Cupcake.xcodeproj.
Create a new tagged version from the current master branch and it will be good.

Fixed that, thanks.


*: Upgraded dependencies

Ran cargo update for a security audit

You can set wasmtime to 0.21.0, the changes we needed have been released.


Copy header/payload from a subject to another subject.

Make functionality that copies messageheader and ,-payload from one subject to another. (Through posting a new message or "Clone" methods found in C# NATS library.)
This should be implemented in the messagetable as an action button.
Acceptance criteria:
1: Successfully copies messagecontent onto another subject.
2: Do not show sensitive information unnecessarily. (remember logging)
3: Verify that everything works as expected.

Decided on "posting a new message" instead of "Clone" or "Mirror" methods.


Accept Header for JSON API

Currently the Request Session is sending an accept header of */*. We have restrictions in place on our end that require the Accept header to be application/json when using the tables API. If I were to submit a PR would you entertain a change something like this?
if self.api == 'JSON':
   self.session.headers.update({'Accept': 'application/json'})

Done!


mismatched arch session-manager-plugin got bundled

While packaging for homebrew, I noticed that the wrong arch session-manager-plugin got bundled (below is npm build for a apple arm machine):
Warning: Binaries built for a non-native architecture were installed into basti's prefix.
The offending files are:
  /opt/homebrew/Cellar/basti/1.6.2/libexec/lib/node_modules/basti/node_modules/basti-session-manager-binary-darwin-arm64/session-manager-plugin	(x86_64)

Looks like the session-manager-plugin version used (1.2.295.0) doesn't output binaries for the darwin_arm architecture during the build so Basti uses darwin_amd64 instead. Here's the list of build outputs:

I'll take a look if a new version of the plugin supports darwin_arm this week. Meanwhile, I have a couple of questions:

Is this issue blocking the Brew formula release?
Maybe another binary from the screenshot above can be used for darwin_arm?

I replied in the Homebrew PR. Looks like the plugin doesn't support arm architecture natively and amd64 works well with arm Macs.

since it looks like the later versions still have the problem discussed in the issue.

is it the same issue with the newer version?

Unfortunately, I won't be able to test this right now but the same issue was reported again a month ago (https://github.com/aws/session-manager-plugin/issues/93) and people are reporting the same issues in comments (https://github.com/aws/session-manager-plugin/issues/57#issuecomment-2025129630, https://github.com/aws/session-manager-plugin/issues/93#issuecomment-2009941346).

I'm closing this issue due to inactivity. Brew formula was successfully implemented without fixing this.


COM can't be used on non-Windows platforms

Summary
I'm working on several cross-platform codebases, such as DWriteCore, which are based heavily on COM.  Currently I'm using the old com-rs crate, which is no longer supported.  I've converted everything in my codebase to use windows-rs, but currently compilation (linking, specifically) fails due to platform dependencies for Windows.
The first blocking issue I've hit is that the code generated for calling COM methods calls GetErrorInfo, which is only defined on Windows.
Crate manifest
No response
Crate code
No response

Note that non-Windows support has never been in scope. While I have no intention to block such support, this would definitely be considered a new feature, not a bug. Existing test coverage exists merely to support cross-compilation and to ease cross-platform development.
My general philosophy here continues to be that I am ok with enabling cross-platform support so long as it doesn't substantially impact the complexity and primacy of Windows support.

Now, android-rs would be something.

@sivadeilra this is awesome! It may one day help us migrate hassle-rs off of Microsoft's deprecated and unmaintained com-rs and onto windows-rs, while getting autogenerated/auto-maintained bindings for free!
Note that you'll face a lot of pushback, see for example https://github.com/microsoft/windows-rs/issues/1874 which @riverar already linked to you in #3082. Though a lot of folks will be very thankful if you do manage to sit this through to the end :crossed_fingers:

For the specific linker-error case, I did however manage to get at least https://github.com/microsoft/windows-rs/pull/1863 merged. This allows you to define symbols like GetErrorInfo in your own codebase, to provide a fallback when the symbol isn't defined by "the OS". For example: https://github.com/MarijnS95/windows-rs/commit/e595cda41cc8ca8163de6f15890da1f1e7150d2b#diff-1be65c4aa0a07017c9575e418540179c2b2f7638f022c867c0ba42d39e435e9bR63-R67
(But it's hacky and I hope you can come up with something better)

To be clear, I do not want this to become something like WINE.  I don't want additional functionality that tries to bridge the gap between Windows and other platforms.  I just want the subset of windows-core and windows crates that can sensibly used on other platforms.
COM is the big one, for me.  And that doesn't mean any runtime functions at all, it just means IUnknown, HRESULT, and the ability to use the #[implement] and #[interface] macros.

To be clear, hassle-rs's requirements (for using libdxcompiler.so) are identical to that.

FYI: We use COM on linux for service-fabric-rs. mssf-pal crate supplies a subset of win32 api to make it work.

@youyuanwu cool, thanks for linking. That looks to be using the exact functionality proposed in #1863 :slightly_smiling_face:

I took a stab at pulling out the COM support in windows-core into its own crate but so much of the code generation depends on certain facilities living in windows-core that it's not very practical. My next experiment is to see how much of the bells and whistles that have crept into windows-core can be moved out. I'm thinking of variant support moving into the windows crate as an extension and string support moving into its own windows-strings crate.


Fixed #864 : Remove "no data" Toast from Offline edit fragment

Description

Added an ImageView and a TextView in the OfflineEditFragment layout
Added Code to toggle the visibility of the elements in the layout based whether the no. of offline products are "zero" or not
Added code for "first offline fragment" that changes the text and image in the layout based on "firstUpload" boolean value stored in the shared prefference "usage"

Related issues and discussion
#864 Remove "no data" Toast from Offline edit fragment
Screen-shots, if any

Maybe the icons should be more consistent with each other, check out this and this


Outline guiding principles and decide on direction for queries in 3.0

There are several query-related efforts being worked on or considered for the 3.0 release. This issue is a single place to track and discuss these efforts at a high level as we make progress and decide on direction.
Initial areas to consider:

Increase test coverage
Fix query bugs
Decide on architecture changes, if any. For example internalize or remove Relinq.
Consolidate and document guiding principles for different types of translations and evaluations. For example, #12668, #12672, #12736, #12737, #12667, #12728, #12736, #12756, #12764, #12775, #12776, #12794, #10434, #12284 (null safety), #13899, #14046, https://github.com/aspnet/EntityFrameworkCore/issues/12761#issuecomment-408555481 (top projection rewrite), #12195
Decide on keeping full client-eval fallback, going to explicit client-eval only, or moving to a mixture. For example, see #12765, #12844, #12945, #13048, #10265

Related: https://github.com/aspnet/EntityFrameworkCore/issues/12761#issuecomment-408555481,

Including here the seminal client-side evaluation proposal Colin Meek wrote:
Implicit boundaries in LINQ to Entities (client-side evaluation)
Overview
LINQ forces us to blur the boundary between the server and the client. For a provider like LINQ to Entities, this means that a query supplied to the stack is always partly evaluated in the client application and partly in the database server. For instance, the query
int productId = 1;
var q =
    from p in context.DbSet<Product>()
    where p.ProductID == productId
    where DetermineProductPriority(p) == "High"
    select new XElement(
        "Product",
        new XAttribute("Name", p.Name),
        new XAttribute("ID", p.ProductId));

includes a selection evaluated by the server (p.ProductId = @productId), but also client expressions, e.g. the binding of the free variable productId and materialization of XML nodes in the projection. It also includes a call to a client-side method on a predicate over a server correlated expression, something that is not supported by either LINQ to Entities or in most LINQ to * implementations.
While we have found it convenient to talk about LINQ to Entities as a strict implementation – all server or nothing – and other implementations such as LINQ to SQL as hybrid implementations – splitting the query into client and server expressions – these implementations really exist along a continuum, and it makes sense to examine this continuum in more detail to clarify the current behavior and how we could improve EF.
It is convenient to discuss this continuum with respect to the following expression scopes:

Independent sub-expressions: in the above query, the access to the free variable productId is compiled into something like Expression.Field(Expression.Constant(CS$<>8__locals6), fieldof(<>c__DisplayClass5.productId)), which does not depend on the current scope of the query (i.e. it is not correlated to the contents of any table in the database). The process of identifying and evaluating these independent sub-expressions has become known as expression funcletization at Microsoft.
Client sources: LINQ queries are typically bootstrapped by IQueryable roots, e.g. dbContext.DbSet<Product>() in the above example.
Client projections: while client sources introduce typed iterators as the root or roots for remote queries, client projections close the loop by shaping typed query results. In the above example, entity results are shaped into XML nodes, new XElement…
Dependent sub-expressions: certain sub-expressions can only be evaluated by the client, e.g. DetermineProductPriority…, but depend on intermediate results from the server, e.g. DetermineProductPriority(<u>p</u>).

Note that these categories are somewhat arbitrary. A client source is a kind of independent sub-expression and a dependent sub-expression is in some ways just a generalization of client projections. The categories are still intuitively useful however: independent sub-expressions often map to parameters while client sources mostly map to scans, and; client projections are frequently benign while dependent sub-expressions are often cause for concern (consider high selectivity filters).
Implicit boundaries are dangerous but also an essential feature of LINQ. Finding the appropriate balance is important. From feedback we have received over the years, we know that users expect magic and may be disappointed if we either throw because we’re overly strict or we end up streaming 1,000,000 rows from the database into the client to find the 10 rows matching a client predicate because we were too loose.
Independent sub-expressions and client sources
Options:

Necessary: free variables and constants are unavoidable. We need to allow expressions that represent access to field, property and constant in order to implement a viable LINQ provider.
Literals: the LINQ equivalent of value literals. Allow constants (1), primitive type constructors (new DateTime(2008, 5, 28)) and even array initialization patterns ( new int[] {…}).
Root construction: currently LINQ to Entities only supports some forms of roots inside the query, e.g. context.Products is recognized, but unfortunately inline construction of query roots is not, e.g. dbContext.DbSet<T>() and context.CreateQuery<T>(string) are not recognized.
Server-unsupported expressions that can be converted to server parameters: currently LINQ to Entities will throw if it finds any expression that it cannot evaluate on the server in a query. If we could turn an independent expression into a query parameter and we know that no part of the expression could ever be evaluated by the server, at least with the same semantics, we could funcletize it. There is an interesting challenge on deciding how much of a sub-expression we can funcletize. For instance, consider the expression stringBuilder.ToString().Length. While the stringBuilder.ToString() part can only be evaluated on the client, .Length could either be evaluated on the client alongside the rest of the expression or translated to LEN(@param) in the store. LEN() in SQL Server has subtly different semantics form string.Lenght in the CLR in that it ignores trailing blanks. We have three options on what we can funcletize:
a. The minimal sub-expression that cannot be evaluated on the server
b. The minimal sub-expression that cannot be evaluated on the client plus any expression that can be evaluated on either the client or the server with identical semantics
c. The maximal sub-expression that can be evaluated on the client
Options (a) and (b) guarantee that at least for a particular server, all occurrences of such expressions would have consistent semantics, regardless of where they appear in the query. The alternative to this is to evaluate. Option (c) would imply that any sub-expression that can be turned into a parameter would be evaluated on the client. This is the most flexible approach but means we apply inconsistent semantics to some operators depending on which side of the boundary they find themselves on.
Server-unsupported expressions that cannot be turned into server parameters: We should also explore these. There are interesting solutions where you pipe values through the query to the result, which works when the value is never cracked or cracked at need on the client. This greatly increases the cost of the feature for LINQ to Entities which would need to introduce its own intermediate metadata representation for such values.

Client projection
Options:

Composable: projections that can be composed within a query are supported as client projections. For instance, entities, complex types and “rows” can be projected but arbitrary method calls or constructors cannot.
Non-composable: top-level projections could include arbitrary method calls and constructors. For efficiency, reverse funcletization occurs for the client projection, e.g. select ClientMethod1(ClientMethod2(e.X, e.Y), e.Z) becomes select new { e.X, e.Y, e.Z } into f select ClientMethod1(ClientMethod2(f.X, f.Y), f.Z).

Note that method calls may introduce additional round-trips to the server. Before people start shouting about “nanny state APIs”, consider that users are not complaining that they wanted the round-trips but that we failed to crack the methods to figure out how to avoid them…
Dependent sub-expressions
What happens when a sub-expression cannot be evaluated by the server but depends on intermediate results?

Whenever an unsupported expression is encountered, we could simply split the query at that point (modulo the kinds of local optimizations described for client optimization).
We should attempt to push as much server logic “down” the tree as possible to minimize the amount of work in the client. This is critical where joins, selections and even some projections are involved.

Interface considerations
If we implement support for these patterns, we should also consider allowing the user to disable them. The user can exercise whatever level of control they want over the client-server partitioning of the query. In addition, we should make the partitioned plan visible to the user, either by using documented boundary expressions or through a debugger visualizer.
Implementation considerations
We can include a separate pass to identify supported and unsupported expressions in the query tree, similar to other LINQ implementations.

I love the example with new XElement(...) and immediately see the potential translation into FOR XML PATH. 😉
As for architectural changes and managing query bugs, I feel like there is a lot I could say but I don't know how effective I would be at communicating it.

@ajcvickers - In regards to looking at ReLinq.  Is #12048 the driving issue behind that, or are there other things driving it as well?
It seems like a major undertaking to remove/replace it given how the tightly the query system is architected around it with a lot of risks for breaking things.

@pmiddleton Other things too, And yes, I agree that it is risky; that is one of the considerations. Sorry for being a bit ambiguous here. Like I said, stay tuned. Don't be impatient. 😉

@ajcvickers - I have the open PR for TVF and am currently working on a pivot feature.  Both have tie-ins to ReLinq so the possible change peaked my interest as there might be some rework required on my part.  :)

@pmiddleton Agreed.

@ajcvickers Heyyy, I’m onto you 👀

(not linking individual issues here).
Currently, there are few blockers in current query pipeline due to various reasons which requires some architecture level changes. e.g tracking entity types in all cases properly, Include/correlated subquery with client eval, Type inference in Sql Translation, Include path specification. let queries, client evals and side effects.
Some of them are also linked to things coming from relinq like let queries which generates same reference to subquery model which can be mutated. EF property over collection type props, tracking which we try to identify by tracing querysources.
#12048 Brought us need to do something about external dependencies. As "grass is always greener" on other side, we debated & researched on the line, how about removing relinq as query parser and work with expression trees directly. Certainly it doesn't solve all issues with query but it still allows us to get away from limitations we are facing from Relinq right now. As our friend @tuespetre have already written implementation of EF Core query compiler without relinq, we got inspiration that it may be possible to have a query compiler just with expression tree world. So that is what we are going to attempt for 3.0. Even though 3.0 is little bit further out, replacing whole query compiler is a huge task. The big chunk of work will come into SQL translation which is currently done from relinq query model. Trying to change the whole pipeline and also fixing issues (or make it extensible to fix in future), is optimistic goal. We are going to try for it though we may keep older pipeline for a while till we reach good progress with new compiler. This will also require changes in provider codebase.
In the same timeframe we are also going to internalize/remove IX-async and also replace IAsyncEnumerable coming from C# 8.0.
Along the way we are also going to make design decision for some of the issue linked here on the basis of value/usefulness vs complexity. Individual issues would be updated as we make decision.
Hope this clarifies people looking for update. If any confusion, feel free to ask questions.

I have have run into two areas where Relinq has given me a bunch of grief which you might want to think about as you do the rework.

Trying to use functions as data sources.  When implementing both bootstrapped functions and table valued functions I needed a function to act as the source of the query.  Relinq is geared around having a class act as the query source, while a function which returns IQuerable is perfectly acceptable to Linq.

Trying to add support for a custom Linq method.  The work I am doing for the pivot feature required a new Linq method.  There was a lot of formality to get Relinq to understand that new method and parse it correctly.  Even once I had the parsing working Relinq still didn't treat the method the way I would have liked.  The extension point required pivot to be treated as a result operator when it is really acting as a new query source which change the shape of the query.

Removing the AST should help with both of these issues.

@smitpatel - Is there going to be a public working branch at some point so that we can check things out as they are built?

@pmiddleton - Yes. It will take some time to initially bootstrap the compiler.

Update on the plan for query in 3.0
We have been having some deep discussions over what we should do with query in 3.0. These discussions fall into two areas:

Query behavior and whether we can make query more robust, conceptually easier to understand, and with less chance of regressing by making some high-level changes to the way it works. Most of the issues linked from here relate to this.
Query architecture. Specifically, should remove Relinq, and, if so, what are the mechanics and architecture changes around this?

Focusing on the second question, at a high-level Relinq provides value in that it normalizes and “simplifies” the expression trees into a model that is conceptually easier to handle. However, as is usually the case with normalization and simplification, the end result can be less flexible to work with and can result in added complexity when the simplified model hides things in the underlying system that are needed. We have found this to be increasingly the case with EF Core, especially when considering non-relational databases. Also, any time a new abstraction is introduced it requires that people understand that abstraction. In this case, this means that people with a good understanding of LINQ expression trees also need to understand Relinq when contributing to EF Core, which can be a blocker for collaboration.
Before going any further, the EF Core team at Microsoft want to make it 100% clear that we have been extremely happy with the quality of Relinq, and working with the Relinq team has been a complete pleasure. Relinq is a great product and does a great job of doing what it does. That being said, after carefully considering our options, we believe that now is the right time to experiment with moving away from Relinq. As we learn from this experiment we may change our mind and keep Relinq, but we expect that the flexibility and experience of working directly with expression trees will turn out to be a better way forward for EF Core.
We will follow up on this post next week with some more details on how we are approaching this from the technical side.
What this means for applications
The removal of Relinq should not impact most application code, since application code does not typically interact directly with the query model.
What this means for database providers
Assuming we achieve the desired results and we end up using expression trees directly instead of Relinq in 3.0, database providers will have to change accordingly. We will update the database providers that we ship (SQL Server, In-memory, Cosmos DB, Oracle sample provider, SQLite) to ensure that they are ready for the 3.0 release. We will also work closely with the owners of other popular database providers like Pomelo MySQL, Npgsql, and SQL Compact, to ensure that they are updated to support this change in 3.0.

@ajcvickers - Is SelectExpression et al going to stay the same, or do you plan to generate the final sql via a walk of the expression tree?

@pmiddleton - There will be SelectExpression but it would change some of the features. Perhaps the biggest change in SelectExpression would be around its projection handling. Currently, we add SQL fragment to projection and bind using the index in projection. Which causes quite a few issues in terms of changing projection ordering since there is already ValueBuffer binding somewhere else. (this is the reason we needed RequiresMaterializationExpressionVisitor). The plan is to deviate away from binding straight away. Also we were looking to make it Immutable. I am not certain if that is going to achieve best result but we'll see.

@tuespetre Sent you an email last week--did you get it? If not, what is a good email address to use?

@ajcvickers sorry, I did get that but was kind of busy. I replied the other day 👍
Regarding what @smitpatel said, I think he covered what I would have suggested as the two 'first motions':

moving towards a more 'natural' projection model instead of a 'list' projection model. I accomplished this by heavy use of ValueTuple to tack on and retrieve extra properties, and extension Expressions where appropriate for carrying things 'up the tree' (like Include information.) This goes hand-in-hand with the ValueBuffer thing -- by having placeholder 'column reference' expressions in the projection instead of binding early to a data source, you get multiple benefits:

every kind of binding can be handled using Expression constructs, like, "does this member access expression make sense?" No more of the "ok, is what we want in this list of expressions, no, ok, is it in this value buffer, maybe..."
you can optimize away duplicate reader accesses at the latest possible stage

making SelectExpression immutable (and everything else Expression/ExpressionVisitor based, for that matter) will make transformations easier to reason about, although it does involve stepping back at times to visualize some higher-order transformations. It makes 'copying and pasting' whole subtrees during transformations fairly easy and worry-free!

Would https://github.com/aspnet/EntityFrameworkCore/issues/6482 be relevant in this context? For instance there are entire 3rd party libraries that re-implement much of the Linq->SQL builder because they want it for things like bulk insert/update etc. that EF Core doesn't natively support.

@ajcvickers & @divega can answer that.
The current pipeline of EF, especially multiple SQL for collection include etc. makes it impossible to provide SQL easy way.

@wizofaus I don't think the decisions we make about this issue will change much about #6482. The main challenge is still that there are are places in which we generate mutliple SQL statements for on LINQ query, and that is not likely going away in 3.0.

Pity, that always seemed to me one of the biggest weaknesses of LINQ-to-SQL - that you couldn't really use it at all to construct valid SQL just for condition clauses (in at least one application I worked on, we were able to use an earlier version of EF to generate parameterised SQL then chop off everything before "WHERE" to get the part we needed, but even that doesn't seem to be possible with EF Core).

These questions are very important to solve in order to enable customers to migrate from LINQ to SQL. We are currently stuck on L2S in a large code base because we depend on client side evaluation.
Client side evaluation was very convenient and useful in L2S. We routinely check the SQL generated by L2S using SQL Profiler and we are very performance conscious. From years of practical experience I can say that L2S got the design right. It just worked.
We absolutely need the final Select to just do the right thing: Pull the required data from the server and evaluate the "tail" on the client. We do not need anything inside the query to be arbitrary code (e.g. a Where). This just has not been useful to us. Maybe there are 1 or 2 places we accidentally do this but it's not important.
We do depend on local evaluation of query parameters, though. This was your stringBuilder.ToString().Length example. A more practical example: where entity.SomeCol == GetFilterValue(httpContext). It is very useful to be able to do this.
We are often hitting the "root construction" issue. For example, we have code patterns like:
var results = (from t1 in db.Table1
                      join t2 in GetFilteredTable2() on ...
                      select new { t1, t2 });

Sub-queries enter the main query in the form of an IQueryable value that is externally generated and passed around. L2S seems to inline these values into the main query and execute this as normal. This is what we want. We also want compiled query caching for this. L2S does not have that but we built a sophisticated intermediate LINQ provider to build caching on top of L2S. This drastically optimized client side CPU cost for us. If EF were to handle that that would be awesome. We could delete a large amount of brittle code.

In general, CPU usage is important to us. We found that simple SQL queries take 0.25ms to execute on SQL Server. L2S could easily add 1ms of CPU overhead to this. This is a problem for resource usage as well as for latency if many queries are executed in a row.
As explained we solved this by compiling queries using a proxy LINQ provider.
EF should be conscious of CPU usage: Compiled query caching is important. Evaluation of client expressions should use runtime compiled functions (not interpretation or reflection).
In order to efficiently support caching of "holes" (the GetFilteredTable2() example) we found it necessary to perform two levels of caching. We first match the outer query structure to a cache item. Then, we evaluate the holes. The shape of the hole expressions is of course dynamic. Therefore, when creating the final query tree in order to compile it we need to plug in the holes and perform another round of caching. I assume arbitrary amounts of such nesting are theoretically possible. We found a fixed two levels sufficient for our needs.

From feedback we have received over the years, we know that users expect magic

Yes, that is a good characterization of our situation. We require magic to finally be able to migrate. Also, we found this kind of magic to be extremely useful.

@GSPP thanks! This is great feedback! I will let @smitpatel provide details about any deltas, but I think what we are set to deliver in EF Core 3.0 matches the design you describe very closely.

That would be awesome. We are eager to be able to consume all the innovation that is happening with EF.

@smitpatel, @ajcvickers any need to keep this open now?

I will write certain notes overall about things and close it. Or I can add it to the docs and close the issue with reference to it.

Adding a version of this with your notes to a “query architecture” section in the docs sounds great. I guess we can create a docs issue for that and close this anyway.

Even simple group by is broken in ef 3, why is this? Seems like a group by should translate to sql statement just fine...

I second that question and also wonder how this situation makes any sort of sense.
This makes me feel miffed, very miffed.

@jjxtra @linkerro - Did you guys look at #17068?


Menu: excessive DOM nodes

It appears that each menubar-menu-button instance has a menubar-menu-title as a child which seems irrelevant to the whole show. We should remove those extra menubar-menu-title nodes.

Oh!!! I see. I thought the menu would be outside. 👍


Factory reset via serial Interface does not reset

Executing a factory reset trough the Serial Interface does not reset the Box to the default Values.
The interface addresses remain to the previously configured values. The captive portal is still shown as active and the Voucher support as well active.
I have also tried to do a reset trough the GUI. Same result.

I did not see this behaviour, but I made sure the config cannot possibly linger in the file system by rebooting directly. It would try to back up vouchers probably refreshing the whole config in the process. Now also kills dhcpleases and rrd data. Cheers for the report.

Franco, I go to Diagnostics:Factory defaults and click "Yes" and wait until the box comes up again.
Below is the end of the Startup log.
Starting configd.
Launching the init system... done.
Initializing.................. done.
Starting device manager (devd)...done.
Loading configuration...done.
Updating configuration...done.
Setting up extended sysctls...done.
amdtemp0:  on hostb4
Setting timezone...done.
Configuring loopback interface...done.
Starting syslog...done.
Starting Secure Shell Services...done.
Setting up polling defaults...done.
Setting up interfaces microcode...done.
Configuring loopback interface...done.
Creating wireless clone interfaces...done.
Configuring LAGG interfaces...done.
Configuring VLAN interfaces...done.
Configuring QinQ interfaces...done.
Configuring WAN interface...done.
Configuring LAN interface...done.
Configuring GAST interface...done.
Syncing OpenVPN settings...done.
Configuring firewall......done.
Starting PFLOG...done.
Setting up gateway monitors...done.
Synchronizing user settings...done.
Starting webConfigurator...done.
Configuring CRON...done.
Starting DNS forwarder...done.
Starting NTP time client...done.
Starting DHCP service...done.
Starting DHCPv6 service...done.
Configuring firewall......done.
Starting captive portal(buero)... done
Enabling voucher support... done
Generating RRD graphs...^[[B^[[Bdone.
Starting UPnP service... done.
Starting syslog...done.
Starting CRON... done.
*** Welcome to OPNsense <IP_ADDRESS>-c6240d38f (amd64) on OPNsense ***
WAN (re1)       -> v4/DHCP4: <IP_ADDRESS>/24
LAN (re0)       -> v4: <IP_ADDRESS>/24
GAST (re2)      -> v4: <IP_ADDRESS>/24
FreeBSD/amd64 (OPNsense.teamstrebel.ch) (ttyu0)
HERE is the LOG:
Apr 3 00:22:34 	opnsense: /diag_defaults.php: Successful login for user 'admin' from: <IP_ADDRESS>
Apr 3 00:20:54 	getty[99167]: open /dev/ttyv3: No such file or directory
Apr 3 00:20:54 	getty[99167]: open /dev/ttyv3: No such file or directory
Apr 3 00:20:54 	getty[98652]: open /dev/ttyv0: No such file or directory
Apr 3 00:20:54 	getty[98652]: open /dev/ttyv0: No such file or directory
Apr 3 00:20:54 	getty[99538]: open /dev/ttyv6: No such file or directory
Apr 3 00:20:54 	getty[99538]: open /dev/ttyv6: No such file or directory
Apr 3 00:20:54 	getty[98992]: open /dev/ttyv2: No such file or directory
Apr 3 00:20:54 	getty[98992]: open /dev/ttyv2: No such file or directory
Apr 3 00:20:54 	getty[99198]: open /dev/ttyv4: No such file or directory
Apr 3 00:20:54 	getty[99198]: open /dev/ttyv4: No such file or directory
Apr 3 00:20:54 	getty[99358]: open /dev/ttyv5: No such file or directory
Apr 3 00:20:54 	getty[99358]: open /dev/ttyv5: No such file or directory
Apr 3 00:20:54 	getty[98707]: open /dev/ttyv1: No such file or directory
Apr 3 00:20:54 	getty[98707]: open /dev/ttyv1: No such file or directory
Apr 3 00:20:54 	sshlockout[99980]: sshlockout/webConfigurator v3.0 starting up
Apr 3 00:20:54 	getty[99557]: open /dev/ttyv7: No such file or directory
Apr 3 00:20:54 	getty[99557]: open /dev/ttyv7: No such file or directory
Apr 3 00:20:53 	kernel: done.
Apr 3 00:20:53 	kernel: done.
Apr 3 00:20:53 	opnsense: /usr/local/etc/rc.bootup: miniupnpd: Starting service on interface: lan
Apr 3 00:20:53 	opnsense: /usr/local/etc/rc.bootup: Creating rrd update script
Apr 3 00:20:47 	rtsold[15659]:  sendmsg on re1: Operation not permitted
Apr 3 00:20:43 	rtsold[15659]:  sendmsg on re1: Operation not permitted
Apr 3 00:20:42 	kernel: done
Apr 3 00:20:42 	lighttpd[63300]: (log.c.164) server started
Apr 3 00:20:41 	kernel: ipfw2 (+ipv6) initialized, divert loadable, nat loadable, default to accept, logging disabled
Apr 3 00:20:41 	kernel: .....done.
Apr 3 00:20:41 	configd.py: Updating all dyndns
Apr 3 00:20:41 	kernel: done.
Apr 3 00:20:40 	opnsense: /usr/local/etc/rc.bootup: ROUTING: setting default route to <IP_ADDRESS>
Apr 3 00:20:40 	kernel: done.
Apr 3 00:20:40 	opnsense: /usr/local/etc/rc.bootup: ROUTING: remove current default route to <IP_ADDRESS>
Apr 3 00:20:40 	kernel: done.
Apr 3 00:20:40 	lighttpd[49851]: (log.c.164) server started
Apr 3 00:20:39 	rtsold[15659]:  sendmsg on re1: Operation not permitted
Apr 3 00:20:39 	kernel: done.
Apr 3 00:20:39 	opnsense: /usr/local/etc/rc.bootup: The command '/usr/sbin/pw groupadd test1 -g -M 2000 2>&1' returned exit code '65', the output was 'pw: gid `0' has already been allocated'
Apr 3 00:20:39 	kernel: done.
Apr 3 00:20:36 	kernel: pflog0: promiscuous mode enabled
Apr 3 00:20:36 	opnsense: /usr/local/etc/rc.bootup: Resyncing OpenVPN instances.
Apr 3 00:20:36 	opnsense: /usr/local/etc/rc.bootup: Accept router advertisements on interface re1
Apr 3 00:20:35 	configd.py: Linkup starting re1
Apr 3 00:20:35 	devd: Executing '/usr/local/opnsense/service/configd_ctl.py 'interface linkup start re1''
Apr 3 00:20:35 	kernel: re1: link state changed to UP
Apr 3 00:20:33 	configd.py: Linkup stopping re1
Apr 3 00:20:33 	devd: Executing '/usr/local/opnsense/service/configd_ctl.py 'interface linkup stop re1''
Apr 3 00:20:33 	kernel: re1: link state changed to DOWN
Apr 3 00:20:33 	kernel: done.
Apr 3 00:20:33 	opnsense: /usr/local/etc/rc.bootup: Failed to clone interface ath0 with error code 1, output ifconfig: SIOCIFCREATE2: Device not configured
Apr 3 00:20:33 	kernel: done.
Apr 3 00:20:33 	opnsense: /usr/local/etc/rc.bootup: Cloning new wireless interface ath0_wlan1
Apr 3 00:20:33 	kernel: done.
Apr 3 00:20:33 	sshlockout[5932]: sshlockout/webConfigurator v3.0 starting up
Apr 3 00:20:33 	sshd[5738]: Server listening on <IP_ADDRESS> port 22.
Apr 3 00:20:33 	sshd[5738]: Server listening on :: port 22.
Apr 3 00:20:33 	kernel: amdtemp0:  on hostb4
Apr 3 00:20:33 	kernel: Trying to mount root from ufs:/dev/ada0s1a [rw]...
Apr 3 00:20:33 	kernel: Timecounter "TSC" frequency<PHONE_NUMBER> Hz quality 800
Apr 3 00:20:33 	kernel: SMP: AP CPU #1 Launched!
Apr 3 00:20:33 	kernel: da0: quirks=0x2<NO_6_BYTE>
Apr 3 00:20:33 	kernel: da0: Attempt to query device size failed: NOT READY, Medium not present
Apr 3 00:20:33 	kernel: da0: 40.000MB/s transfers
Apr 3 00:20:33 	kernel: da0: Serial Number 058F63666485
Apr 3 00:20:33 	kernel: da0: <Multiple Card Reader 1.00> Removable Direct Access SCSI-4 device
Apr 3 00:20:33 	kernel: da0 at umass-sim0 bus 0 scbus6 target 0 lun 0
Apr 3 00:20:33 	kernel: ada0: Previously was known as ad4
Apr 3 00:20:33 	kernel: ada0: 15258MB (31248704 512 byte sectors: 16H 63S/T 16383C)
Apr 3 00:20:33 	kernel: ada0: Command Queueing enabled
Apr 3 00:20:33 	kernel: ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
Apr 3 00:20:33 	kernel: ada0: Serial Number YTAK140600316
Apr 3 00:20:33 	kernel: ada0: <SuperSSpeed S238 16G V4629N> ATA-7 SATA 2.x device
Apr 3 00:20:33 	kernel: ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
Apr 3 00:20:33 	kernel: umass0:6:0:-1: Attached to scbus6
Apr 3 00:20:33 	kernel: umass0: SCSI over Bulk-Only; quirks = 0x4001
Apr 3 00:20:33 	kernel: umass0: <Generic Flash Card ReaderWriter, class 0/0, rev 2.01/1.00, addr 2> on usbus6
Apr 3 00:20:33 	kernel: ugen6.2:  at usbus6
Apr 3 00:20:33 	kernel: uhub3: 5 ports with 5 removable, self powered
Apr 3 00:20:33 	kernel: uhub1: 5 ports with 5 removable, self powered
Apr 3 00:20:33 	kernel: uhub6: 4 ports with 4 removable, self powered
Apr 3 00:20:33 	kernel: uhub5: 4 ports with 4 removable, self powered
Apr 3 00:20:33 	kernel: uhub2: 5 ports with 5 removable, self powered
Apr 3 00:20:33 	kernel: uhub0: 5 ports with 5 removable, self powered
Apr 3 00:20:33 	kernel: uhub4: 2 ports with 2 removable, self powered
Apr 3 00:20:33 	kernel: uhub6: <ATI EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus6
Apr 3 00:20:33 	kernel: ugen6.1:  at usbus6
Apr 3 00:20:33 	kernel: uhub5: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus5
Apr 3 00:20:33 	kernel: ugen5.1:  at usbus5
Apr 3 00:20:33 	kernel: uhub4: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus4
Apr 3 00:20:33 	kernel: ugen4.1:  at usbus4
Apr 3 00:20:33 	kernel: usbus6: 480Mbps High Speed USB v2.0
Apr 3 00:20:33 	kernel: usbus5: 12Mbps Full Speed USB v1.0
Apr 3 00:20:33 	kernel: usbus4: 12Mbps Full Speed USB v1.0
Apr 3 00:20:33 	kernel: uhub3: <ATI EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
Apr 3 00:20:33 	kernel: ugen3.1:  at usbus3
Apr 3 00:20:33 	kernel: uhub2: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
Apr 3 00:20:33 	kernel: ugen2.1:  at usbus2
Apr 3 00:20:33 	kernel: uhub1: <ATI EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
Apr 3 00:20:33 	kernel: ugen1.1:  at usbus1
Apr 3 00:20:33 	kernel: uhub0: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
Apr 3 00:20:33 	kernel: ugen0.1:  at usbus0
Apr 3 00:20:33 	kernel: usbus3: 480Mbps High Speed USB v2.0
Apr 3 00:20:33 	kernel: usbus2: 12Mbps Full Speed USB v1.0
Apr 3 00:20:33 	kernel: usbus1: 480Mbps High Speed USB v2.0
Apr 3 00:20:33 	kernel: IPsec: Initialized Security Association Processing.
Apr 3 00:20:33 	kernel: Timecounters tick every 1.000 msec
Apr 3 00:20:33 	kernel: usbus0: 12Mbps Full Speed USB v1.0
Apr 3 00:20:33 	kernel: random: unblocking device.
Apr 3 00:20:33 	kernel: acpi_throttle0:  on cpu0
Apr 3 00:20:33 	kernel: uart1: <16550 or compatible> at port 0x2f8-0x2ff irq 3 on isa0
Apr 3 00:20:33 	kernel: uart0: console (115200,n,8,1)
Apr 3 00:20:33 	kernel: uart0: <16550 or compatible> at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
Apr 3 00:20:33 	kernel: ppc0: cannot reserve I/O port range
Apr 3 00:20:33 	kernel: orm0:  at iomem 0xee000-0xeffff on isa0
Apr 3 00:20:33 	kernel: acpi_button0:  on acpi0
Apr 3 00:20:33 	kernel: usbus6 on ehci2
Apr 3 00:20:33 	kernel: usbus6: EHCI version 1.0
Apr 3 00:20:33 	kernel: ehci2: <AMD SB7x0/SB8x0/SB9x0 USB 2.0 controller> mem 0xfeb04600-0xfeb046ff at device 22.2 on pci0
Apr 3 00:20:33 	kernel: usbus5 on ohci3
Apr 3 00:20:33 	kernel: ohci3: <AMD SB7x0/SB8x0/SB9x0 USB controller> mem 0xfeb03000-0xfeb03fff at device 22.0 on pci0
Apr 3 00:20:33 	kernel: pci4:  on pcib4
Apr 3 00:20:33 	kernel: pcib4:  at device 21.0 on pci0
Apr 3 00:20:33 	kernel: usbus4 on ohci2
Apr 3 00:20:33 	kernel: ohci2: <AMD SB7x0/SB8x0/SB9x0 USB controller> mem 0xfeb02000-0xfeb02fff at device 20.5 on pci0
Apr 3 00:20:33 	kernel: device_attach: pcib4 attach returned 6
Apr 3 00:20:33 	kernel: pci0: couldn't attach pci bus
Apr 3 00:20:33 	kernel: pcib4: failed to allocate initial prefetch window: 0-0xfffff
Apr 3 00:20:33 	kernel: pcib4: failed to allocate initial memory window: 0-0xfffff
Apr 3 00:20:33 	kernel: pcib4:  at device 20.4 on pci0
Apr 3 00:20:33 	kernel: isa0:  on isab0
Apr 3 00:20:33 	kernel: isab0:  at device 20.3 on pci0
Apr 3 00:20:33 	kernel: ata1:  at channel 1 on atapci0
Apr 3 00:20:33 	kernel: ata0:  at channel 0 on atapci0
Apr 3 00:20:33 	kernel: atapci0: <ATI IXP700/800 UDMA133 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x4010-0x401f at device 20.1 on pci0
Apr 3 00:20:33 	kernel: usbus3 on ehci1
Apr 3 00:20:33 	kernel: usbus3: EHCI version 1.0
Apr 3 00:20:33 	kernel: ehci1: <AMD SB7x0/SB8x0/SB9x0 USB 2.0 controller> mem 0xfeb04500-0xfeb045ff at device 19.2 on pci0
Apr 3 00:20:33 	kernel: usbus2 on ohci1
Apr 3 00:20:33 	kernel: ohci1: <AMD SB7x0/SB8x0/SB9x0 USB controller> mem 0xfeb01000-0xfeb01fff at device 19.0 on pci0
Apr 3 00:20:33 	kernel: usbus1 on ehci0
Apr 3 00:20:33 	kernel: usbus1: EHCI version 1.0
Apr 3 00:20:33 	kernel: ehci0: <AMD SB7x0/SB8x0/SB9x0 USB 2.0 controller> mem 0xfeb04400-0xfeb044ff at device 18.2 on pci0
Apr 3 00:20:33 	kernel: usbus0 on ohci0
Apr 3 00:20:33 	kernel: ohci0: <AMD SB7x0/SB8x0/SB9x0 USB controller> mem 0xfeb00000-0xfeb00fff at device 18.0 on pci0
Apr 3 00:20:33 	kernel: ahcich3:  at channel 3 on ahci0
Apr 3 00:20:33 	kernel: ahcich2:  at channel 2 on ahci0
Apr 3 00:20:33 	kernel: ahcich1:  at channel 1 on ahci0
Apr 3 00:20:33 	kernel: ahcich0:  at channel 0 on ahci0
Apr 3 00:20:33 	kernel: ahci0: AHCI v1.20 with 4 6Gbps ports, Port Multiplier supported
Apr 3 00:20:33 	kernel: ahci0: <AMD SB7x0/SB8x0/SB9x0 AHCI SATA controller> port 0x4020-0x4027,0x4040-0x4043,0x4028-0x402f,0x4044-0x4047,0x4000-0x400f mem 0xfeb04000-0xfeb043ff at device 17.0 on pci0
Apr 3 00:20:33 	kernel: re2: Ethernet address: 00:0d:b9:33:16:02
Apr 3 00:20:33 	kernel: rgephy2: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
Apr 3 00:20:33 	kernel: rgephy2: <RTL8169S/8110S/8211 1000BASE-T media interface> PHY 1 on miibus2
Apr 3 00:20:33 	kernel: miibus2:  on re2
Apr 3 00:20:33 	kernel: re2: MAC rev. 0x00200000
Apr 3 00:20:33 	kernel: re2: Chip rev. 0x2c000000
Apr 3 00:20:33 	kernel: re2: ASPM disabled
Apr 3 00:20:33 	kernel: re2: Using 1 MSI-X message
Apr 3 00:20:33 	kernel: re2: <RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet> port 0x3000-0x30ff mem 0xfea00000-0xfea00fff,0xfe900000-0xfe903fff at device 0.0 on pci3
Apr 3 00:20:33 	kernel: pci3:  on pcib3
Apr 3 00:20:33 	kernel: pcib3:  at device 6.0 on pci0
Apr 3 00:20:33 	kernel: re1: Ethernet address: 00:0d:b9:33:16:01
Apr 3 00:20:33 	kernel: rgephy1: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
Apr 3 00:20:33 	kernel: rgephy1: <RTL8169S/8110S/8211 1000BASE-T media interface> PHY 1 on miibus1
Apr 3 00:20:33 	kernel: miibus1:  on re1
Apr 3 00:20:33 	kernel: re1: MAC rev. 0x00200000
Apr 3 00:20:33 	kernel: re1: Chip rev. 0x2c000000
Apr 3 00:20:33 	kernel: re1: ASPM disabled
Apr 3 00:20:33 	kernel: re1: Using 1 MSI-X message
Apr 3 00:20:33 	kernel: re1: <RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet> port 0x2000-0x20ff mem 0xfe800000-0xfe800fff,0xfe700000-0xfe703fff at device 0.0 on pci2
Apr 3 00:20:33 	kernel: pci2:  on pcib2
Apr 3 00:20:33 	kernel: pcib2:  at device 5.0 on pci0
Apr 3 00:20:33 	kernel: re0: Ethernet address: 00:0d:b9:33:16:00
Apr 3 00:20:33 	kernel: rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
Apr 3 00:20:33 	kernel: rgephy0: <RTL8169S/8110S/8211 1000BASE-T media interface> PHY 1 on miibus0
Apr 3 00:20:33 	kernel: miibus0:  on re0
Apr 3 00:20:33 	kernel: re0: MAC rev. 0x00200000
Apr 3 00:20:33 	kernel: re0: Chip rev. 0x2c000000
Apr 3 00:20:33 	kernel: re0: ASPM disabled
Apr 3 00:20:33 	kernel: re0: Using 1 MSI-X message
Apr 3 00:20:33 	kernel: re0: <RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet> port 0x1000-0x10ff mem 0xfe600000-0xfe600fff,0xfe500000-0xfe503fff at device 0.0 on pci1
Apr 3 00:20:33 	kernel: pci1:  on pcib1
Apr 3 00:20:33 	kernel: pcib1:  at device 4.0 on pci0
Apr 3 00:20:33 	kernel: pci0:  on pcib0
Apr 3 00:20:33 	kernel: pcib0:  port 0xcf8-0xcff on acpi0
Apr 3 00:20:33 	kernel: Event timer "HPET1" frequency 14318180 Hz quality 450
Apr 3 00:20:33 	kernel: Event timer "HPET" frequency 14318180 Hz quality 550
Apr 3 00:20:33 	kernel: Timecounter "HPET" frequency 14318180 Hz quality 950
Apr 3 00:20:33 	kernel: hpet0:  iomem 0xfed00000-0xfed003ff on acpi0
Apr 3 00:20:33 	kernel: acpi_timer0: <32-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
Apr 3 00:20:33 	kernel: Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
Apr 3 00:20:33 	kernel: Event timer "i8254" frequency 1193182 Hz quality 100
Apr 3 00:20:33 	kernel: Timecounter "i8254" frequency 1193182 Hz quality 0
Apr 3 00:20:33 	kernel: attimer0:  port 0x40-0x43 irq 0 on acpi0
Apr 3 00:20:33 	kernel: Event timer "RTC" frequency 32768 Hz quality 0
Apr 3 00:20:33 	kernel: atrtc0:  port 0x70-0x71 irq 8 on acpi0
Apr 3 00:20:33 	kernel: cpu1:  on acpi0
Apr 3 00:20:33 	kernel: cpu0:  on acpi0
Apr 3 00:20:33 	kernel: acpi0: Power Button (fixed)
Apr 3 00:20:33 	kernel: acpi0:  on motherboard
Apr 3 00:20:33 	kernel: padlock0: No ACE support.
Apr 3 00:20:33 	kernel: cryptosoft0:  on motherboard
Apr 3 00:20:33 	kernel: module_register_init: MOD_LOAD (vesa, 0xffffffff80f618f0, 0) error 19
Apr 3 00:20:33 	kernel: kbd0 at kbdmux0
Apr 3 00:20:33 	kernel: random: <Software, Yarrow> initialized
Apr 3 00:20:33 	kernel: wlan: mac acl policy registered
Apr 3 00:20:33 	kernel: module_register_init: MOD_LOAD (ipw_monitor_fw, 0xffffffff805dcc70, 0) error 1
Apr 3 00:20:33 	kernel: ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
Apr 3 00:20:33 	kernel: ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
Apr 3 00:20:33 	kernel: module_register_init: MOD_LOAD (ipw_ibss_fw, 0xffffffff805dcbc0, 0) error 1
Apr 3 00:20:33 	kernel: ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
Apr 3 00:20:33 	kernel: ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
Apr 3 00:20:33 	kernel: module_register_init: MOD_LOAD (ipw_bss_fw, 0xffffffff805dcb10, 0) error 1
Apr 3 00:20:33 	kernel: ipw_bss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
Apr 3 00:20:33 	kernel: ipw_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw/.
Apr 3 00:20:33 	kernel: ioapic0 <Version 2.1> irqs 0-23 on motherboard
Apr 3 00:20:33 	kernel: cpu1 (AP): APIC ID: 1
Apr 3 00:20:33 	kernel: cpu0 (BSP): APIC ID: 0
Apr 3 00:20:33 	kernel: FreeBSD/SMP: 1 package(s) x 2 core(s)
Apr 3 00:20:33 	kernel: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
Apr 3 00:20:33 	kernel: ACPI APIC Table:
Apr 3 00:20:33 	kernel: Event timer "LAPIC" quality 400
Apr 3 00:20:33 	kernel: avail memory =<PHONE_NUMBER> (1917 MB)
Apr 3 00:20:33 	kernel: real memory =<PHONE_NUMBER> (2017 MB)
Apr 3 00:20:33 	kernel: TSC: P-state invariant, performance statistics
Apr 3 00:20:33 	kernel: AMD Features2=0x35ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,IBS,SKINIT,WDT>
Apr 3 00:20:33 	kernel: AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
Apr 3 00:20:33 	kernel: Features2=0x802209<SSE3,MON,SSSE3,CX16,POPCNT>
Apr 3 00:20:33 	kernel: Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
Apr 3 00:20:33 	kernel: Origin = "AuthenticAMD" Id = 0x500f20 Family = 0x14 Model = 0x2 Stepping = 0
Apr 3 00:20:33 	kernel: CPU: AMD G-T40E Processor (1000.02-MHz K8-class CPU)
Apr 3 00:20:33 	kernel: FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512
Apr 3 00:20:33 	kernel: root@sensey64:/usr/obj/usr/src/sys/SMP amd64
Apr 3 00:20:33 	kernel: FreeBSD 10.1-RELEASE-p8 #0 2822f85(master): Sat Mar 21 03:09:56 CET 2015
Apr 3 00:20:33 	kernel: FreeBSD is a registered trademark of The FreeBSD Foundation.
Apr 3 00:20:33 	kernel: The Regents of the University of California. All rights reserved.
Apr 3 00:20:33 	kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
Apr 3 00:20:33 	kernel: Copyright (c) 1992-2014 The FreeBSD Project.
Apr 3 00:20:33 	syslogd: kernel boot file is /boot/kernel/kernel
Apr 3 00:19:39 	syslogd: exiting on signal 15
Apr 3 00:19:39 	shutdown: reboot by root:
Apr 3 00:19:39 	shutdown: reboot by root:
Apr 3 00:19:33 	configd.py: Syncing firewall
Apr 3 00:18:54 	opnsense: /index.php: Successful login for user 'admin' from: <IP_ADDRESS>

@jstrebel the code used by the GUI is the same as previously fixed for the console; just tested and it looks good too. Small follow ups have been committed to polish the code some more.


How to Add menu from database value in array menu ?

Question
Answer

Issue or Enhancement
Issue

Laravel Version
Used version of Laravel

Project Version
Used version of this project

Current Behavior
Hi, Please Help me, how to add menu into array menu adminlte.php get from database value ? please Help ...
Thanks You
Expected Behavior

Steps to Reproduce

Possible Solutions

If you are looking to build your menu at runtime, please read Menu At Runtime Dcoumentation. Also, I'm sure you will find more examples if you search on the CLOSED ISSUES

If you are looking to build your menu at runtime, please read Menu At Runtime Dcoumentation. Also, I'm sure you will find more examples if you search on the CLOSED ISSUES

Thanks for your help and information

@finalcode21 Please, consider closing this issue if you have found a solution for your problem. Also, you can share a summary of your solution for future readers (if you have the time).


Update what-are-bots.md

Acrolinx Scorecards
We currently enforce a minimum score of 80.
Click the scorecard links for each article to review the Acrolinx feedback on grammar, spelling, punctuation, writing style, and terminology:

Article
Score
Issues
Scorecard
Processed

msteams-platform/bots/what-are-bots.md
96
2
link
:white_check_mark:

More info about Acrolinx
Use the Acrolinx extension, or sidebar, in Visual Studio Code to check spelling, grammar, style, tone, clarity, and key terminology when you're creating or updating content. For more information, see Use the Visual Studio Code extension to run Acrolinx locally.

Docs Build status updates of commit 6359af9:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/bots/what-are-bots.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Question about the client and InfinityFree (#2)

Since the previous issue had been closed, I'm going to ask my questions here. Let me know if we should just reopen the previous issue.
Hi, I had a few question after a while...

To create an account, we need to provide our own username to the API. What is this username used for (as I couldn't find it on MOFH Reseller Panel)? Also how do InfinityFree generates these usernames? And is it safe if we just use increments to generate this usernames?

How to retrieve the main domain? As far as I know, https://myprohost.ga uses this client to interface with MOFH's API and it has a Domain and Main Domain column as seen in the screenshot below:

I assume the domain column is filled with the domain supplied during account creation.

The 8 character username field is just used for the main domain. I think that some methods of the api require this username instead of the vpusername, so storing it is important.
Because of the main domain just being the username, I just store the username part and then in the view I add the .undisk.ga.

Okay, thank you for clarifying about what it is used for and how you generated it.
I still wanted to know how IF generates those usernames because IF has lot's of users and the username must be unique.
About the Main Domain tho, how can you be sure that the domain will end in undisk.ga instead of nixit.ga in my case? I checked InfinityFree has their main domain ends in .rf.gd for my accounts with .rf.gd subdomain.

If I remember right, Hans mentioned on the old hosters.cc forum (now dead) that he used Str::random. This is actually the way that I found out about Laravel.
The main domain is based off of which API keys are used. If I use the api keys for nixit.ga then it’s a nixit.ga main domain. For undisk.ga keys it’s an undisk.ga main domain.

Initially, I generated the main domain based on the local database primary key, but that blew up in my face once when I setup a copy of the production database for development and created an account with that. Random generation is easier and less error prone.
There were times when I used separate API key pairs for all the different subdomains, but now I use one API key pair for all accounts. I like the easy of managing only a single key pair, but it's mostly personal preference as to what you should use.


refactor: internal functions and variables are unnecessarily exposed

We keep a lot of variables public for the purposes of testing. This isn't desirable since it makes the contract harder to understand and audit and may consume more gas in some cases. For example, the recoveryOf in the NameRegistry is already published as an event and can be restricted to internal.
Unfortunately, there aren't any great solutions to being able to test internal state in Foundry, though there is a ticket in the v1 Roadmap. Hardhat's hardhat-exposed has a library that performs this by auto-generating a contract with exposed state.
It seem unlikely that Foundry will have a working solution in time for launch, and moving to hardhat isn't desirable for a variety of other reasons. We should consider one of these options before launch:

Do nothing and deploy with the variables as public (ensures that our tests are thorough)
Remove all internal state asserts from tests permanently and accept that our tests are lower fidelity
Remove all internal state asserts from tests temporarily for launch and then revert

Name Registry
Initial testing with the NameRegistry contract shows that making the currently variables private might actually be worse in terms of gas costs because the most common operation renew is more expensive, while less common operations are cheaper.
renew is +43
transfer is -16
register is - 45

For functions making the currYear and currYearFee private reduces transfer costs by about 17 gas.
ID Registry
Testing with ID registry shows that making all variables private has significant savings, so this is definitely worth exploring for the contract.
trustedRegister is - 23 gas
register(address) is -45 gas
register(address,address,string) is -307 gas

IDRegistry improvements were made here: https://github.com/farcasterxyz/contracts/pull/76

NameRegistry changes contributed negatively to gas usage and inheriting the contract to test private variables is complicated. The added complexity doesn't bring any benefit, and so we're avoiding this for now.


Tapping on a link in a project README fails

It pushes in a Safari VC, whereas I was expecting it to be rendered like the README is. It also failed to load the file.
Note that this is for relative URLs, like linking to another .md file within the project.

Bug Report Dump (Auto-generated)

Version 1.25.0<PHONE_NUMBER>)
Device: iPad8,5 (iOS 12.1)
TestFlight: false

I think this is a dupe, can’t find it atm
Sent with GitHawk

#1769 perhaps?

I believe it's this one: #2247.


Add link to deployed version

I deployed it to https://validator.spacedirectory.org/v1/ :)

https://validator.spacedirectory.org/v1/

I get 502 Bad Gateway

Oh, the systemd unit file was missing the autorestart option. I'll fix it.

I started the service manually for now.

It reverted to bad gateway again

Thanks for letting me know, it should be fixed for good now.


Upgrade rails 5.0.5

Changes Unknown when pulling e95ab1339223356a83146b003680c58f1f720e3d on upgrade/rails-5.0.5 into ** on master**.


README link is old

The link to the API page is old and should be updated.  New link:
http://votesmart.org/share/api?utm_campaign=python-votesmart&utm_source=github

Eh, or for the sake of clarity, get rid of UTM vars.


Support span_id (zipking, jaeger, open tracing etc) in Message structs.

I did an experiment with this last year: https://github.com/conorbros/shotover-proxy/tree/open-telemetry

Its not clear to me what this issue meant.
It's been 4 years so at this point I think we should close it and move on.


Feature/outbond ip

This exactly changes the original behavior. I think this change can be applied when neither address nor interface is configured.
So if address use address, then if interface query from interface, otherwise from DNS

Fixed

Have you tested if this work on some cloud servers? Because I can see we have some pion code using the engine Interface and IP.

func GetOutboundIP() {
	log.Println(GetOutboundIP2())
	log.Println(getIPFromInterface())
}

// Get preferred outbound ip of this machine
func GetOutboundIP2() string {
	conn, err := net.Dial("udp", "<IP_ADDRESS>:80")
	if err != nil {
		log.Fatal(err)
	}
	defer conn.Close()

	localAddr := conn.LocalAddr().(*net.UDPAddr)

	return localAddr.IP.String()
}

func getIPFromInterface() (string, error) {
	iname := "eth0"

	ifaces, err := net.Interfaces()
	if err != nil {
		return "", err
	}
	for _, i := range ifaces {
		if i.Name != iname {
			continue
		}
		addrs, err := i.Addrs()
		if err != nil {
			return "", err
		}
		for _, addr := range addrs {
			switch v := addr.(type) {
			case *net.IPNet:
				return v.IP.String(), nil
			case *net.IPAddr:
				return v.IP.String(), nil
			}
		}
	}

	return "", fmt.Errorf("no address for interface %s", iname)
}

Tests in Google cloud:
2022/02/22 09:17:09 10.128.x.x
2022/02/22 09:17:09  no address for interface eth0
Tests in Aliyun:
2022/02/22 17:17:27 172.2x.1xx.90
2022/02/22 17:17:27 172.2x.1xx.90
Discussions in stakeoverflow:
https://stackoverflow.com/questions/23558425/how-do-i-get-the-local-ip-address-in-go

I mean your new modifications, leave interface and address empty.
And for the specific error, that's just because you were not using a correct interface name. Use ip addr to see available interfaces at first.


⚠ gorzdrav.org has degraded performance

In 3ec2147, gorzdrav.org (https://gorzdrav.org) experienced degraded performance:

HTTP code: 200
Response time: 2721 ms

Resolved: gorzdrav.org performance has improved in 67352b0.


Import issue: Re-export serde_json::Value

Since some functions, such as configuration(&self, items: Vec<ConfigurationItem>) return types that contain serde_json::Value it would be useful to re-export this type.
Link to original issue: https://github.com/ebkalderon/tower-lsp/issues/404

I don't think this is necessary and want to suggest a different approach:
Since lsp-types is exported already, it would be possible to use LSPAny instead, which is more fitting in the context of a language server, but this should be reflected in the public API of tower-lsp then.


Trouble converting Terraform to Pulumi with the Helm provider

Problem description
Hello, I am using the following guide to convert my Terraform stack to Pulumi:
https://www.pulumi.com/docs/guides/adopting/from_terraform/
I am having trouble with the following step:

Next, run tf2pulumi.

I see the following output when I run this command:
❯ tf2pulumi --target-language typescript
error: no resource plugin 'helm' found in the workspace or on your $PATH

I believe the issue is there is no Helm plugin to correspond with the Helm Terraform provider in my TF stack:
❯ pulumi plugin ls
NAME          KIND      VERSION  SIZE    INSTALLED     LAST USED
aws           resource  3.38.1   256 MB  5 months ago  5 months ago
aws           resource  3.34.0   253 MB  6 months ago  6 months ago
aws           resource  3.32.2   251 MB  6 months ago  6 months ago
digitalocean  resource  3.8.0    38 MB   5 months ago  5 months ago
digitalocean  resource  3.6.0    38 MB   6 months ago  6 months ago
kubernetes    resource  2.9.1    70 MB   5 months ago  5 months ago
kubernetes    resource  2.8.3    66 MB   6 months ago  6 months ago
random        resource  3.2.0    29 MB   5 months ago  5 months ago
random        resource  3.1.1    29 MB   6 months ago  6 months ago

Affected product version(s)
❯ tf2pulumi version
v0.11.1

Suggestions for a fix
I think that for now, my resolution is to comment the Helm provider and resources out, then try to manually resolve this by converting these resources into Pulumi kubernetes/helm/v3.
This error message seems a bit misleading in that there is no Helm plugin available, so it would be nice if it could give me a bit more context on what I should do here.
Long-term, it would be nice to support automatic migration from the official Hashicorp Helm provider into Pulumi kubernetes/helm/v3.

A customer is trying to convert (pulumi convert --from terraform --language python --out pulumi) terraform modules:

https://github.com/awslabs/data-on-eks/tree/main/ai-ml/jupyterhub
https://github.com/awslabs/data-on-eks/tree/main/ai-ml/mlflow

They get something like
Converting from terraform...

warning: failed to install provider "helm": could not find latest version for provider helm: 401 HTTP error fetching plugin from https://api.github.com/repos/pulumi/pulumi-helm/releases/latest
Converting to python...
Installing dependencies...
... (shortened)

and the process seemingly succeeds but the resulting Python file is almost empty (just module imports).
The modules are almost entirely based on the helm TF provider and its helm_release resource, which we should ideally convert to our kubernetes.helm.Release.
Related to https://github.com/pulumi/pulumi-kubernetes/issues/2744
If converting to our Release is troublesome (as this comment suggests), we should still generate the code and let them use a locally bridged provider (or do that automatically, as https://github.com/pulumi/pulumi/issues/17357 suggests).


*: resolve when flashback meets error in the first batch

Signed-off-by: husharp<EMAIL_ADDRESS>
What is changed and how it works?

Issue Number: Close #13672
What's Changed:
resolve when flashback meets error in the first batch.

Check List
Tests

Integration test

Release note
None

/cc @JmPotato

/cc @Connor1996 @AndreMouche
Please help take a look, thx!

@sticnarf @MyonKeminta PTAL, thanks

/cc @AndreMouche @Connor1996
PTAL again, thx a lot.

/cc @sticnarf  @MyonKeminta PTAL, thanks!

All comments are addressed. PTAL, thx!

/run-check-issue-triage-complete

Why not just use Vec::new() instead of Vec::with_capacity(0)?

/merge