Feature Request: pyformat param types

Describe the Feature
Would you consider adding support for pyformat param types? I've run a few of our snowsql scripts throught the formatter and for the most part it's absolutely fantastic, but it trips up on our param types which use, e.g. %(end_date)s.
I thought I should ask before even considering diving in and attempting a PR, in case it was something that had been previously considered and rejected.
Why do you want this feature?
This would increase the number of situations in which the tool could be used, and is expanding on an existing feature rather than adding something entirely new.

It would be nice to support this kind of parameters, but I wouldn't really like to add a specific option for this Python formatting syntax. Rather I would consider providing a more flexible way of customizing the possible parameter types.
But I'm not fully sure what would be the best API for this. Currently the paramTypes setting categorizes parameters to 4 types:

positional (?)
numbered (like :1)
named (like :name)
quoted (like :"name")

This Python syntax looks most similar to the quoted type, with parenthesis sort-of used for quoting, plus there's this extra s code at the end (which I understand can be a lot more complex than just a single letter). But we can't simply add another quote type that would match (name)s part as that would most likely interfere with parsing of parenthesis pairs.
Assuming these pyformat parameters can be parsed with regular expressions, the syntax could be something like:
paramTypes: {
  custom: [
    { regex: String.raw`%\((\w+)\)[a-z]` }
  ]
}

Additional consideration is that we also need a way to determine the actual name inside the parenthesis. This is used by the parameter-substitution feature. A simplest approach would be to just use a regex capture group as I've done above.

That would definitely be good for us, though it sounds like a bit of a bigger rewrite of the param code, unless it's already using regex behind the scenes.

Well, it is indeed using regexes behind the scenes. It's more about how the final regexes for matching parameter tokens get built.

Related discussion: https://github.com/sql-formatter-org/sql-formatter/discussions/551

This is now implemented in 12.2.0


Will warp support the contract verification at starknet ?

Current the contract deploy to starknet by using warp is not verified.
Will support in future ?

Hi, by verified, do you mean verified on Voyager?

Yes


Promise support for form.parse()

I am thinking that it could be nice to make form.parse() promisified: if no callback is provided, it will return a promise.
This could allow us to rewrite the following code:
form.parse(req, (err, fields, files) => {
  if (err) {
    // ... handle error
    return
  }

  // do something with fields and files
})

Into something like this:
try {
  const [fields, files] = await form.parse(req)
  // ... do something with fields and files
} catch (e) {
  // ... handle error
}

Is this something that has been considered already?
I had a quick look at the code and it seems that parse currently returns this (i suppose to allow method chaining). This seems to be a potential blocker for this feature (as it will require a breaking change), but as an alternative, it could be possible to create a new method called parsePromise or something similar, which can act as a wrapper for the original parse method...
The main advantages I see in supporting this syntax are the following:

Unified error handling (one catch to rule them all)
Integrates well with async handlers where there might be other async logic before or after the form parsing
Fewer callbacks nesting

Thoughts?
If this idea resonates with the maintainers and the community, I'd be happy to find some time to submit a PR...

Wait for v2 to be finalized. We will try to make no breaking change for v2 but for v3 this could be good.

Awesome, thanks for the quick reply!

If you want to help review this PR https://github.com/node-formidable/formidable/pull/686

Of course. Added some comments there. :)

Yep, it was considered.
Maybe we can introduce it in v2 as parseAsync, later in v 3 just merge them.

@lmammino you can make a PR against the v3.x branch

Are async/await already available in the upcoming v.3.0?

No the issue is still open
We are waiting for someone to make a PR

Should form.parse()

awlays return a promise (vote with 👍 )
or be callback based if a function is passed to it and else return a promise (vote with ❤ )
other 😕 (explain in comment)

@GrosSacASac

or be callback based if a function is passed to it and else return a promise

for v2 and above. Then we should consider the Streaming things and APIs, but lets that be v4 haha.

Still no Promise support?

Soon ™

Just a thought... instead of supporting both callbacks and promises in the same function you could mimic the way fs does it. Something like:
// For Callback
import { formidable } from 'promises';
const formidable = require('formidable');

// For Promise
import { formidable } from 'formidable/promise';
const formidable = require('formidable').promises;

@jwerre yep, i'm leaning more towards that, or just a separate exports.
@GrosSacASac haha :rofl: on v3 or for v4? I think I said we will drop v2 (as main) around march or april? Uh, let me see where that thing was haha.

How wrapping function in promise would take a year?
const [fields, files] = (await new Promise(async (r, re) => {
    await form.parse(req, async (err, fields, files) => {
      if (err) return re(err)
      return [fields, files]
    })
  })) as [formidable.Fields, formidable.Files]

but please do coverage test on a code above

It's not hard. It's just that there are several other things and at least 2 parallel versions used.
Plus, it's that hard to just use util.promisify, or whatever it was called, on form.parse.
It's a standard callback function anyway.

here is a "one line" solution for creating promise
const parseForm = async (req) => new Promise((resolve, reject) => new formidable.IncomingForm().parse(req, (err, fields, files) => err ? reject(err) : resolve([fields, files])))

@ds2345 right. The beauty of callbacks APIs, you can easily turn them into promises.. haha.

Hi @ds2345 , I believe your code works, but it seems my code never enters into callback (err, fields, files) => err ? reject(err) : resolve([fields, files])), no error or exception either. The file is uploaded by axios.postForm(), it's just an 82bytes markdown file. Did I miss anything to make this code work?
async #parseForm(request) {
  return new Promise((resolve, reject) => {
    // const form = formidable({ multiples: true, api: { bodyParser: false } });
    console.log('START form.parse()');
    new formidable.IncomingForm().parse(request, (error, fields, files) => {
      console.log(
        'form.parse() error:',
        error,
        ', fields:',
        fields,
        ', files:',
        files
      );

      error ? reject(error) : resolve([fields, files]);
    });
  });
}

async handle(request) {
console.time('TaskAttachmentSaveHandler.handle() parseForm');

const [fields, files] = await this.#parseForm(request);

console.timeEnd('TaskAttachmentSaveHandler.handle() parseForm');
}

It's very disappointing to see that a library in 2023 isn't capable of using promises.
I would happily submit a PR, as this fix would be trivial to make ... but from the comments here it sounds like the maintainers are against promises?

It is not delayed because it is hard, but because I was busy and I try to fix all the test, see the ongoing PR for commit details

So based on votes I decided to make it both callback and promise as a transition and then later make it promise only (breaking change)

https://github.com/node-formidable/formidable/pull/940

Awesome job! Thanks.
Also, I should mention, can you update npm so the types for the library will be updated too? TY

It will be on npm once pr is merged

Published as<EMAIL_ADDRESS>

Oh nice, v3 is now latest on npm 👀 🔥

Migration guide?
Unfortunately, in upgrading just now, it appears it's breaking our code.
I was looking for a "Migrating from v2 to v3" guide, I only found this Version History file which seems more like (partly outdated) historical information:

But maybe this is not documentation that the Formidable team will provide? 😬

Release notes?
There are also release notes for v3.0.0, which seem to potentially have breaking changes, although none of them are explicitly called out as such:

3.0.0

feat: remove options.multiples (https://github.com/node-formidable/formidable/pull/730)
use modern URLSearchParams https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams internally
files and fields values are always arrays
fields with [] in the name do not receive special treatment
remove unused qs and querystring dependency
feat: Use ES modules (https://github.com/node-formidable/formidable/pull/727)
options.enabledPlugins must contain the plugin themselves instead of the plugins names

Broken types?
Maybe the DefinitelyTyped types at @types/formidable are also outdated now, because of the data structure changes described in the 3.0.0 release notes.
cc @martin-badin @gboer @peterblazejewicz as "code owners" of @types/formidable

Soo... looks like it's time for a bit of trial and error to see how to get this working... 😬

In my first investigation into the problems, it seems like it's associated with this change:

files and fields values are always arrays

I was accessing single files and fields as objects instead of the first index of the array that they are now inside.

I added an overload for form.parse() to allow for promise usage:

https://github.com/DefinitelyTyped/DefinitelyTyped/pull/65819

Maybe the DefinitelyTyped types at @types/formidable are also outdated now, because of the data structure changes described in the 3.0.0 release notes.

this could be, I'm not sure if tests on DT for v2 covers code patterns that are changed in v3 (including changed imports, module type direct support, etc)


Returning error Elasticsearch connection failed

[2024-07-18 14:52:47.672620] [ERROR] Returning error Elasticsearch connection failed
Traceback:
Traceback (most recent call last):
File "/home/sophia/.local/lib/python3.10/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/home/sophia/.local/lib/python3.10/site-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functionsrule.endpoint
File "/home/sophia/git_repo/SBOLExplorer/flask/explorer.py", line 169, in sparql_search_endpoint
if utils.get_es().indices.exists(index=utils.get_config()['elasticsearch_index_name']) is False or utils.get_es().cat.indices(format='json')[0]['health'] is 'red':
File "/home/sophia/git_repo/SBOLExplorer/flask/utils.py", line 109, in get_es
raise ValueError('Elasticsearch connection failed')
ValueError: Elasticsearch connection failed
GET http://localhost:9200/part/_search [status:N/A request:10.017s]
Traceback (most recent call last):
File "/home/sophia/.local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 384, in _make_request
six.raise_from(e, None)
File "", line 2, in raise_from
File "/home/sophia/.local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 380, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib/python3.10/http/client.py", line 1375, in getresponse
response.begin()
File "/usr/lib/python3.10/http/client.py", line 318, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.10/http/client.py", line 279, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/lib/python3.10/socket.py", line 705, in readinto
return self._sock.recv_into(b)
TimeoutError: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/sophia/.local/lib/python3.10/site-packages/elasticsearch/connection/http_urllib3.py", line 171, in perform_request
response = self.pool.urlopen(method, url, body, retries=False, headers=request_headers, **kw)
File "/home/sophia/.local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 637, in urlopen
retries = retries.increment(method, url, error=e, _pool=self,
File "/home/sophia/.local/lib/python3.10/site-packages/urllib3/util/retry.py", line 343, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/home/sophia/.local/lib/python3.10/site-packages/urllib3/packages/six.py", line 686, in reraise
raise value
File "/home/sophia/.local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 597, in urlopen
httplib_response = self._make_request(conn, method, url,
File "/home/sophia/.local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 386, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
File "/home/sophia/.local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 306, in _raise_timeout
raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % timeout_value)
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10)
GET http://localhost:9200/part/_search [status:N/A request:10.017s]
Traceback (most recent call last):
File "/home/sophia/.local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 384, in _make_request
six.raise_from(e, None)
File "", line 2, in raise_from
File "/home/sophia/.local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 380, in _make_request
httplib_response = conn.getresponse()
File "/usr/lib/python3.10/http/client.py", line 1375, in getresponse
response.begin()
File "/usr/lib/python3.10/http/client.py", line 318, in begin
version, status, reason = self._read_status()
File "/usr/lib/python3.10/http/client.py", line 279, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/lib/python3.10/socket.py", line 705, in readinto
return self._sock.recv_into(b)
TimeoutError: timed out

Search "tetR promoter rbs laci cds with gfp"
sophia@sophia:~/git_repo/SBOLExplorer$ pidstat -p 2558561 30
Linux <IP_ADDRESS>-microsoft-standard-WSL2 (sophia)       07/18/24        x86_64        (8 CPU)
14:49:10      UID       PID    %usr %system  %guest   %wait    %CPU   CPU  Command
14:49:40     1000   2558561    8.87    3.20    0.00    0.00   12.07     3  python3
14:50:10     1000   2558561   36.23    6.43    0.00    1.07   42.67     7  python3
14:50:40     1000   2558561   50.32   13.16    0.00    2.03   63.48     1  python3
14:51:10     1000   2558561   63.90   12.83    0.00    1.40   76.73     0  python3
14:51:40     1000   2558561   27.33    7.40    0.00    0.67   34.73     3  python3
14:52:11     1000   2558561   68.27   12.77    0.00    1.90   81.03     0  python3

sysctl vm.max_map_count
sophia@sophia:~/git_repo/SBOLExplorer$ sysctl vm.max_map_count
vm.max_map_count = 262144
sophia@sophia:~/git_repo/SBOLExplorer$ docker exec es01 sysctl vm.max_map_count
vm.max_map_count = 262144


Issue with post ajax request and request at startup

Hello,
When I'm using this plugin with ajax request, the "async=false" parameter does not work, the program continue in next functions before came back in response of the request.
Of course I want to wait for the response of request before executing next functions in my code.
Also when I'm doing request at startup of my application, the error fixed by this plugin is present.
Thank you for your time,
neoCobol

unrelated to plugin


fix the code coverage

Codecov Report

Merging #10 into master will not change coverage.
The diff coverage is 100%.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 419a211...79b2f91. Read the comment docs.


Confusing UI

Hey,
Some product feedback, the Same Profit and Custom tabs are quite confusing. I'm not actually what they even do. I'm guessing one is how much I earn before fees and one is after fees, but I'm not really sure (till I look at the code). May want to remove these tabs or rename them to something more clear.

Hi!
I think this is because only one marketplace is selected, which makes this UI look a little off. We're going to hide the toggle at the top when only one marketplace is selected as it doesn't make any sense to have it showing. Do you think that makes sense in this scenario?
Thanks for the feedback, we really appreciate it.

Hi!
I think this is because only one marketplace is selected, which makes this UI look a little off. We're going to hide the toggle at the top when only one marketplace is selected as it doesn't make any sense to have it showing. Do you think that makes sense in this scenario?
Thanks for the feedback, we really appreciate it.

Ah got it. So it means you want the same profit across all marketplaces. So yes, hiding when only selected would be simpler. Although I'm still not sure it would be fully intuitive as to what it means.
Also profit might be a confusing word. If I bought for 20eth and sold for 21eth, my profit is 1eth and not 21eth. Although maybe users will understand 🤷♂
May need to iterate on it a bit till it becomes clear what the intent is.

This is now fixed in the latest release. By the way there is a little indicator next to the profit that explains a bit what we mean by profit:
How much ETH you will receive after marketplace fees and creator royalties are subtracted.

By the way there is a little indicator next to the profit that explains a bit what we mean by profit:
How much ETH you will receive after marketplace fees and creator royalties are subtracted.

I know but it's hidden away. I wonder if there's a better word that captures the intent here.
"You'll receive:" perhaps.
Or just copy OS and other sites I guess.


I cannot access running ghost on localhost, instead docker machine IP should be used

This could be documented.
See related issue:
https://github.com/docker-library/official-images/issues/1487#issuecomment-190810346

This is related to how Docker functions and handles networking. It is well documented behavior in Docker itself, and is not specific to this image. As such, inclusion of details in image-specific documentation makes little sense.


Fix ordering of mouse events in Canvas renderer

Another quick one (sorry for all the PRs!). I think the ordering of the mouse events in the canvas renderer doesn't quite reflect how the browser would do it if using SVGs. As far as I can tell, the events are usually occur in the following order: mouseout -> mouseover -> mousemove.
Context behind the problem: I'm using the mouseover and mouseout events to show/hide a tooltip. Depending on the "order" of _layers, the tooltip sometimes hid on hovering over a new feature because the mouseout event was triggering after the mouseover.

Great! This fixes #3568, right?

Oh yeah, looks like it!

(sorry for all the PRs!)

Not at all — they're great, keep'em coming!


[Experimental] Move working mod

leaving this blankish as its straight forward whats happing in this pr
move mods
from - /unleash_the_mods/mods
into - /unleash_the_mods/working_mods

not finished dont remove

not finished dont remove


[6.5.x] RHBRMS-2626: Compilation of spreadsheet fails with specific condition

See https://issues.jboss.org/browse/RHBRMS-2626
(cherry picked from commit e2f03cf0b95fe1123777a6abdacf9d4b513653d3)
This is the corollary of https://github.com/droolsjbpm/drools/pull/940 for 6.5.x

Hi @mariofusco
Could you please review this for 6.5.x (I've removed the use of the diamond operator).
Thanks,
Mike


Propose to have a configurable Key ID for Token Auth

Today, the Key ID is generated internally by go/libtrust, which is something can't be changed or easily find out from outside e.g. the certificate owner.
And as this Key ID in JWT header is required to match the internal generated Key ID within registry, I would suggest to have it configurable in Registry rather than enforce a special algorithm.

The Key ID was intended to be a cryptographically secure fingerprint as generated by libtrust. Do you have other fingerprint methods you are requesting be supported? Support for additional fingerprint methods could be added to the auth package, but I am not convinced the registry could be strict enough about enforcing a cryptographically secure Key ID if it allowed the expected fingerprint to be passed in as an opaque value and matched directly with the JWT header.  If the current Key ID mechanism is too rigid, I would suggest generating JWTs using either the x5c or jwk header.
Additional information about the use case you are trying to support would be helpful to fully understand your issue.

In my user case, we have an identity service written in Java, and I was fail to find any library which provide the same algorithm (at least from my knowledge). I actually implemented the algorithm in java for the past few days.
And I also tried to use x5c, but this somewhat double the size of the JWT which I want to minimize. Also tried x5t but unfortunately after code reading, it is not supported by libtrust (or other go library)
KID seems to me only for lookup process within a set of JWK, and doesn't affect signature verification in JWS. It would be great if you can point me to some readings, thanks a lot.
Again, I have solved this by implement the same algorithm in the identity service

let's close this issue, if there is secure concern


Remove the input and output folder. Repo users will have to supply a command line arg of the input and output folder paths

Done: ab874fb


using ofColor instead of ofVec3f for LED pixels

I'm not sure at all you'd want to merge this pull request because it breaks backward compatibility
so, since there is a native, char-based type for colors, it seemed more logical to use it...
what do you think ?
if you think the change is worth it, I could add support for converting ofVec3f into ofColor so backward compatibility is not broken, for the cost of some extra conversion...
if you don't think that's worth, feel free to reject the PR
cheers!

ok ofcolor


Fixing local / secrets / install from existing site

I've tested this with make demo, make local with and without a codebase, with and without existing config, with and without secrets, and with Fedora 5 and 6.  Which, when you look at it, is a pretty big matrix :sweat_drops:
I added some config to make sure make local works when you want to install from existing config.  Check the README changes for instructions on that.
This should fix a lot of issues reported by @Tristaan hopefully.

make demo worked well for me, make local brought up a site successfully but I couldn't add content without getting a white screen.
This is from the watchdog logs.
InvalidArgumentException: A valid cache entry key is required. Use getAll() to get all table data. in Drupal\views\ViewsData->get() (line 140 of /var/www/drupal/web/core/modules/views/src/ViewsData.php).

Notice: Undefined index: base in Drupal\views\Plugin\views\query\QueryPluginBase->getEntityTableInfo() (line 292 of /var/www/drupal/web/core/modules/views/src/Plugin/views/query/QueryPluginBase.php)
#0 /var/www/drupal/web/core/includes/bootstrap.inc(305): _drupal_error_handler_real(8, 'Undefined index...', '/var/www/drupal...', 292)
#1 /var/www/drupal/web/core/modules/views/src/Plugin/views/query/QueryPluginBase.php(292): _drupal_error_handler(8, 'Undefined index...', '/var/www/drupal...', 292, Array)
#2 /var/www/drupal/web/core/modules/views/src/Plugin/views/query/Sql.php(1344): Drupal\views\Plugin\views\query\QueryPluginBase->getEntityTableInfo()
#3 /var/www/drupal/web/core/modules/views/src/Plugin/views/query/Sql.php(1453): Drupal\views\Plugin\views\query\Sql->query()
#4 /var/www/drupal/web/core/modules/views/src/ViewExecutable.php(1321): Drupal\views\Plugin\views\query\Sql->build(Object(Drupal\views\ViewExecutable))

Turns outs it's the EVA fields on the Article/Basic Page content types which is causing the problem, as it's looking for a taxonomy term that is not present.

Merging as the issue resides in the islandora modules default behavior: https://github.com/Islandora/islandora/blob/8.x-1.x/modules/islandora_core_feature/config/install/views.view.display_media.yml#L254 and otherwise the changes are good.


Devel branch - service won't start without permission tweeks

I just did a fresh install of the devel branch and had to do a few things before the service would start.
I am running the systemd version of the installation.
For several months, I have been running rev 1.0 that I had patched to publish the sensor_name in the MQTT output. I decided to update to the latest devel branch to start using the newest code that supports the sensor_name in MQTT natively.
For a fresh install of devel branch:
cd /tmp
git clone -b devel https://github.com/raetha/wyzesense2mqtt.git
I moved my old version off to the side and set up the new one.
After copying the tree to /wyzesense2mqtt I found that the config and logs directories did not exist
mkdir config
mkdir logs
After setting up all the yaml files, I found that the service would not start. The issue was that the python and shell scripts were not executable.
chmod +x *.py
chmod +x *.sh
Fixed the problem.

@dennyfmn, thanks so much for passing this along. I've updated the readme to include instructions to create config and logs folders, and also changed the default permissions for the Python and shell scripts to include the executable bit. These are in devel now, so if you update, hopefully you'll see the right permissions now.


Could not load file or assembly 'Serilog.Ui.Core, Version=2.1.0'

Describe the bug
In production, i have 2 webapps and one background processor, both using nearly the same serilog configuration.
Recently some package upgrades came to my attention;

This build works locally, but for some reason not on my production server;
Unhandled exception. System.IO.FileNotFoundException: Could not load file or assembly 'Serilog.Ui.Core, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.
File name: 'Serilog.Ui.Core, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null'
   at System.Reflection.RuntimeAssembly.InternalLoad(AssemblyName assemblyName, StackCrawlMark& stackMark, AssemblyLoadContext assemblyLoadContext, RuntimeAssembly requestingAssembly, Boolean throwOnFileNotFound)
   at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
   at Serilog.Settings.Configuration.ConfigurationReader.LoadConfigurationAssemblies(IConfiguration section, AssemblyFinder assemblyFinder)
   at Serilog.Settings.Configuration.ConfigurationReader..ctor(IConfiguration configSection, AssemblyFinder assemblyFinder, ConfigurationReaderOptions readerOptions, IConfiguration configuration)
   at Serilog.ConfigurationLoggerConfigurationExtensions.GetConfigurationReader(IConfiguration configuration, ConfigurationReaderOptions readerOptions, DependencyContext dependencyContext)
   at Serilog.ConfigurationLoggerConfigurationExtensions.Configuration(LoggerSettingsConfiguration settingConfiguration, IConfiguration configuration, ConfigurationReaderOptions readerOptions)

Both web apps run under IIS and they both are not able to startup. The background processor seems to run fine - but it's hard to see.

I'm unsure why this happens and i'm curious if this impacts others, or any suggestions on what may cause this.
To Reproduce
Steps to reproduce the behavior:
Have not found a way to reproduce (yet) - but i'll hope to gather more information after some time.
Expected behavior
The app to startup normally.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: Windows
Browser n/a
Version 2.12.0 (latest)

The latest version is 2.4.0, please upgrade to the latest version.

The latest version is 2.4.0, please upgrade to the latest version.

Apologies this was the serilog version.
I'm thinking this has something to do with the serilog.settings.configuration package, not so much with this library.


Providing OpenAIOptions not really works

I want to create a differeny OpenAIService per key. Here's what I tried:

I can use static OpenAIService.CreateService and provide options. This works but generally ServiceProvider should be disposed. Therefore I'm left with hanging undisposed instance of a container. This is an issue on it's own.

I can use AddForgeOpenAIAsTransient in my root container and after call:
 IOpenAIService CreateService(OpenAIOptions options, IServiceProvider serviceProvider, IProviderEndpointService providerEndpointService)

This function gives me the impression that I can really provide OpenAIOptions but in reality each service resolves on it's own IApiHttpService that ends up with default IProviderEndpointService with default options inside.
Specifically this code is copypasted in many services:
        _apiHttpService = serviceProvider.GetRequiredService<IApiHttpService>();
        _providerEndpointService = providerEndpointService;

But it's actually wrong because provided providerEndpointService is not passed to _apiHttpService

It doesn't matter what function I call to register services OpenAIProviderEndpointService is registered as singleton. Therefore it is shared and you can't really have different options in one app. Ok you can call a function from step 2. creating this service with options as constructor parameter but that still leaves IApiHttpService out of the box.

Hello,
Thanks for the feedback.
1, Need to understand when and where we can use the methods and what are our responsibilities. This method is just for the very simple scenarios, when a developer wants to use it without DI just with a provided options AND when the instance is global with same lifecycle of the running application. In that case, it does not necessary to dispose the internal ServiceProvider.
You can find examples in the Playgrounds, project named "MultipleApiKeyUsage" project.
2, I am trying to follow the description what you wrote, but I do not understand the workflow. After you registered the servuce the "AddForgeOpenAIAsTransient", than why do you call the CreateService method? This method designed to use when you manually create your own IServiceProvider instance and you are not in a hosting model envronment. If you create your own ServiceCollection -> register the services an your own requirements -> build your ServiceProvider and ask an instance of IOpenAIService from your provider instance. This is what exactly happen in the "CreateService(OpenAIOptions options)".
In a hosting model environment, instances will be injected, for example in blazor, or also use the IServiceProvider of the host.
Btw, the method "CreateService(OpenAIOptions options, IServiceProvider serviceProvider, IProviderEndpointService provider)" is wrong, the third option does not necessary, it was left after one of my refactoring. I will remove the third option in the next release and it was acquired from the given "serviceProvider".
3, You are right, it is a singleton. It is mostly okay for the apps, but I know, it is not good for every scenario. But feel free to copy/paste my extension methods and create your own ones which are fit for your requirements. That was the reason why I was not set the class visibilities to "internal" as normal in other libraries to give freedom for the SDK users to configure thier environment as they wish. So just simply copy/paste this "AddForgeOpenAIAsTransient" into your own extension class, give it a name and change registration modes. For example from singleton to transient.
Once again, thank you for your feedback. I would like to make the SDK as flexible as possible, sometimes this lead to misunderstandings and of course, I cannot provide service registrations for every cases in the universe :)
Happy coding!

My point is method registrations *AsScoped will not do what you want in all cases.
For example AddForgeOpenAIAsScoped adds scoped OpenAIOptions but OpenAIProviderEndpointService is still registered as a singleton. Therefore it will not get OpenAIOptions from scope but instead will get a global one.
It doesn't matter how many scopes you create because OpenAIProviderEndpointService is a singleton.
The same issue with ApiHttpLoggerService. Singleton but depends on the scoped OpenAIOptions.
If you want *AsScoped to get scoped options you need to register all services that depend on OpenAIOptions as scoped.
As for *AsTransient this is even worse. Because in all services you directly use
  _apiHttpService = serviceProvider.GetRequiredService<IApiHttpService>();

Which means that ApiHttpService will not get the options you provided.

I tried to write an example but it failed unfortunately because of other reasons.
Note that ambigous constructors issue.
// See https://aka.ms/new-console-template for more information
using Forge.OpenAI;
using Forge.OpenAI.Authentication;
using Forge.OpenAI.Interfaces.Services;
using Forge.OpenAI.Settings;
using Microsoft.Extensions.DependencyInjection;

ServiceCollection services = new ServiceCollection();
services.AddForgeOpenAIAsScoped();

// Need to do it because OpenAIOptions is not anyhow registered
// services.AddScoped<OpenAIOptions>();

using var provider = services.BuildServiceProvider();

var scope1 = provider.CreateScope();
var opt1 = scope1.ServiceProvider.GetRequiredService<OpenAIOptions>();
opt1.AuthenticationInfo = new AuthenticationInfo()
{
    ApiKey = "first_key"
};
// Fails - ambigous constructors
var service1 = scope1.ServiceProvider.GetRequiredService<IOpenAIService>();

var scope2 = provider.CreateScope();
var opt2 = scope1.ServiceProvider.GetRequiredService<OpenAIOptions>();
opt1.AuthenticationInfo = new AuthenticationInfo()
{
    ApiKey = "second_key"
};
// Fails - ambigous constructors
var service2 = scope1.ServiceProvider.GetRequiredService<IOpenAIService>();

I got you. I am going to fix this in the next release soon.

Still great thanks for the library - we already integrated and it works.
My only concern was that it's not really friendly for a person who doesn't do deep dive in DI :-)

My original plan was also to use the library without DI as well. Not just because a developer does not have experience with, but because it is not possible to use in the target environment. Maybe I need to improve somehow this question and/or create more examples in the Playground

I am just curious, what is your project about? :) I have never got feedback yet!

Please check v1.4.0, a couple of improvements made, supporting v2 APIs, DI changes, etc.


CallAPI Error with updatedPropertyDetails

When calling Client.GetUpdatedPropertyDetails("48749425") it returns an error stating that the xml response does not match up with the object that it's being deserialized into "There is an error in xml document (1, 1383)

Inner Exception: Input String was not in a correct format

The error is ocurring in CallAPI(), specifically during ser.Deserialize(new StringReader(xml)); where the class it's deserializing into is updatedPropertyDetails.
I've spent at least 10 hours trying to find and correct the issue but unfortunately I'm not fluent in deserializing xml or c# so I have not been able to locate the issue.
Please help.

Can you post a pastbin or the likes of the XML you are receiving and I can take a look?

I am encountering the same issue.
Line 106 of ZillowClient.cs is throwing the error. return ser.Deserialize(new StringReader(xml));
My output XML looks like below:
"<?xml version=\"1.0\" encoding=\"utf-8\"?><UpdatedPropertyDetails:updatedPropertyDetails xmlns:UpdatedPropertyDetails=\"http://www.zillow.com/static/xsd/UpdatedPropertyDetails.xsd\" xsi:schemaLocation=\"http://www.zillow.com/static/xsd/UpdatedPropertyDetails.xsd https://www.zillowstatic.com/vstatic/b20c067/static/xsd/UpdatedPropertyDetails.xsd\" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"><request><zpid>10604179</zpid></request><message><text>Request successfully processed</text>0</message><response><zpid>10604179</zpid><pageViewCount><currentMonth>1</currentMonth><total>1</total></pageViewCount><address><street>1601 Spring Garden St APT 412</street><zipcode>19130</zipcode><city>PHILADELPHIA</city><state>PA</state><latitude>39.9633</latitude><longitude>-75.165021</longitude></address><links><homeDetails>https://www.zillow.com/homedetails/1601-Spring-Garden-St-APT-412-Philadelphia-PA-19130/10604179_zpid/</homeDetails><photoGallery>https://www.zillow.com/homedetails/1601-Spring-Garden-St-APT-412-Philadelphia-PA-19130/10604179_zpid/#image=lightbox%3Dtrue</photoGallery><homeInfo>https://www.zillow.com/homedetails/1601-Spring-Garden-St-APT-412-Philadelphia-PA-19130/10604179_zpid/</homeInfo></links><editedFacts><useCode>Condominium</useCode><bedrooms>1</bedrooms><bathrooms>1.0</bathrooms><finishedSqFt>550</finishedSqFt><lotSizeSqFt>550</lotSizeSqFt><yearBuilt>1832</yearBuilt><numRooms>3</numRooms><heatingSources>Other</heatingSources><heatingSystem>Other</heatingSystem><coolingSystem>Central</coolingSystem><numUnits>1</numUnits></editedFacts><homeDescription>Secure doorman building in a fantastic location at a great price! Must see this lovely unit in the Colonnade! The building features security and a gym. The unit is a well proportioned 1 bedroom 1 bath with large living room, nice kitchen, large separate bedroom with closet, and laundry in unit. Steps to all that Center City has to offer and incredibly easy access to public transportation. Must see!</homeDescription><schoolDistrict>Philadelphia City</schoolDistrict></response></UpdatedPropertyDetails:updatedPropertyDetails>"

Looks like there is an issue with the XML schema.. Currently points to https://www.zillow.com/static/xsd/UpdatedPropertyDetails.xsd which is throwing an error.

I reached out to Zillow support and received a pretty dismissive answer.
Is there any way to register and use a custom / local XSD file in the solution? I am not too familiar with XML either but the problem appears to be the fact that their XSD file for the UpdatedPropertyDetails.xsd is missing closing tags for
<xsd:element> <xsd:sequence> <xsd:complexType>
in lines 4, 5, and 6.

I believe I fixed the XSD when I generated the c# code from it years back. There shouldn't be a need to embed the xsd. I will take a look at see what the issue might be.

I have gotten it working with a few changes. Any insight into best practices would be great, this is my first experience with XML.
First, I replaced the XML line that was directing to the defunct UpdatedPropertyDetails.xsd hosted by Zillow to my own local version:
ZillowClient Line 105:
if (t == typeof(updatedPropertyDetails))
                    {
 xml = xml.Replace(
                            "schemaLocation=\"http://www.zillow.com/static/xsd/UpdatedPropertyDetails.xsd https://www.zillowstatic.com/vstatic/b20c067/static/xsd/UpdatedPropertyDetails.xsd",                             @"schemaLocation=""File:///PATH_TO_MY_CUSTOM_XSD File:///PATH_TO_MY_CUSTOM_XSD");
                    }
Second, there was an issue with the returned XML where the bathroom value was being returned as a double. So I updated my custom UpdatedPropertyDetails.xsd so that the bathrooms field was a type double.
Finally, I updated ZillowSchema.cs to reflect that change as well.

Definitely some good sleuthing here. I'll review the error tonight and search for a longer term solution. Or if you have something that works without having to load a local file and want to pull request it I'd be happy to take a look at that too. Also interesting on the bathroom find, I mean it makes sense but not sure why they'd not define it as such.

@nbw71791 I changed bathrooms type to decimal and it appears to have started working. I had some other pending code changes to the async api and another schema issue that I found a while ago that I push up with the last commit. Please confirm that the issue is resolved and I will close out the issue. Thanks.

@amassaro, I just pulled down the changes and everything is working great. Consider it resolved.

Also, thinking back on it. I imagine the bathroom field is a float because a home can have a "half bath".

@nbw71791 Yeah totally makes sense. Not sure why they have never updated the type OR fixed the XSD. I think originally I had created this library for use with another project back in 2007.


Node 60 shown as "

Node 60 is not shown on the map. Instead, a double quote (") is shown as the node name.

Thanks to FrankOverman for reporting this problem.

When zooming futher in to the highest zoomlevels, node 60 is shown:

The node with the double quote as name is a separate node.  Investigation learns that it is parameter "declutter" in the Openlayers VectorTileLayer that causes node 60 to only be shown at the highest zoomlevels.
When setting  "declutter" to false, node 60 is also shown at lower zoomlevels also:
.
The effect of setting the "declutter" parameter to true is that for map objects that are very close to eachother, only 1 object is shown instead of multiple overlapping objects.  The nodes 60 and double-quote-node are only 5 meters appart.
But: setting  "declutter" to false, has a negative impact on the way many of the other nodes are displayed, as shown here:

Not so nice:

node 61 is cut off at the tile boundary
the way nodes 25 and 45 are displayed is a bit messy

This is what the map looks like with "declutter" parameter set to true:

I suspect that the reason for introducing the double-quote-node was that routes with the same node as start- and end-node (loop routes) were not correctly processed in the past. I believe that removing that node and closing the loop would now work ok. See for example 62-62 and demo for a route that is a loop.
Conclusion: I would propose to leave the knooppuntnet code as-is, and try to close the loop to create a loop route 60-60 to resolve this particular problem. I would expect this will display node 60 nicely and better reflect reality on the ground.

I agree with the proposed solution.
Thanks for attending this one!

Updated the route in OSM:
https://www.openstreetmap.org/changeset/101446462
https://overpass-api.de/achavi/?changeset=101446462
Begin- & end-piece of highway are now the same one: let's see how this renders.

Status: route / relation 60-60 now gives issues; as these are related to issue 114, that one has been reopened:
https://github.com/vmarc/knooppuntnet/issues/114

Problem resolved.
In the meanwhile route route 60-60 has been changed to 25-60 (see changeset: 101526187).


s3/bucket: default to 30-second request timeout

No (infinite) timeout is a bad default. It's easy for libraries to override their bucket objects' timeout, but we should probably default to some timeout, rather than letting requests remain stuck forever (up to TCP timeout).
I'm not 100% convinced this change is a good idea: large uploads or downloads can last a more than 30 seconds... On the other hand, it's easier to observe a timeout and relax the maximum duration, compared to realising that a program is stuck because there's no timeout on S3 calls. What do you think?

Thank you! Good one


Use flag instead of getattr as it was not working as expected. Provid…

…ed raw pages instead of recursive call to har_parser.pages which reached the recursion limit on failure

Coverage decreased (-0.3%) to 93.898% when pulling 99f468398dac0184ccfc25fe9a8120722f653d4c on alan-unravel:recursion_fix into 2f8d3d4e0aec20854bc07a287f3ee81d26345b17 on mrname:master.

Not sure how the coverage decreased since I didn't touch anything except for some local variables and the way pages were accessed in an Error

LOL, yes coveralls can be janky, don't wory about it :-)

cool, do you want me to bumpversion or anything?

Aside from my comment about the error output everything looks pretty good @alan-unravel. Just curious, how did you run into this problem and what did the stack trace look like? Just wondering if we should add a regression test to address the issue you hit.

Just a simple run through the parser and requesting the HarPages, surprised the tests didn't catch this. The last pages getattr(self, 'title', None)  was always resolving to None and then I was getting a maximum recursion depth exceeded.

Coverage decreased (-0.3%) to 93.919% when pulling 1e1a56a24ca8e65ad27f1daa1ad6c45a0829c631 on alan-unravel:recursion_fix into 2f8d3d4e0aec20854bc07a287f3ee81d26345b17 on mrname:master.

:+1:


Sparse access control documentation.

Its not entirely clear how to setup such that iceoryx clients can be under different unix users/groups.  A brief introduction would be helpful. Thanks.

@mossmaurice you are currently working on the docu side, do you have time for this topic?

I would suggest a short icedelivery based example where we have two publishers A, B and two subscribers X, Y with some groups so that A & X and B & Y are separated and cannot read the data of the non corresponding ports.

Looking through existing pull requests - Pull request in progress  #630  adds some color to it.

@ankitkk have a look at this https://github.com/eclipse-iceoryx/iceoryx/blob/master/doc/website/advanced/usage-guide.md
It shows how to configure RouDi for multiple user. It's quite long since I tried it, but theoretically you just need to start the application as different user and it should work if there is a segment for that user configured.

Thanks @ankitkk for the feedback! We should definitely improve the documentation regarding users and groups.
I've add this point to #482 and will close this issue here. If you think other topics are not covered, please raise them in #482, thanks!


Project Test Case 9_11

Summary
Hi i have been stuck on this test case for project. anybody faced this before have any ways to solve this?
Description
Describe the problem and code snippets if necessary
Screenshots (if any):
Insert Images here if necessary

Looking at the test case output, it seems that the code isn't able to determine whether self-check 3 is being occupied when customer 15 arrives. Because for your output, 3 done serving by self-check 3 (in line 63) is correct, so from line 34 to line 63, self-check 3 shouldn't be serving anyone yet self-check 3 serves customer 15 when customer 15 arrives. Also, for your output, self-check 4 is serving 2 customers (customer 13 & 14) at the same time. So you might want to check the code for that -- the code for server, arrive event etc.! I suspect (though I am not too sure) that when creating the next event for the particular customer, the code created a serve event for that customer when it should not been created so early on. (?)
You may want to look at the comments given in issue #780. Someone shared about potential pitfalls for the project and another mentioned the presence of a really useful test case that can be found in this forum for debugging. Maybe this will help!

your self-check 4 serves both 13 and 14 at the same time so maybe you could check whether you are correctly updating the finish time of your self-checkouts when a customer is being served so that the same self-checkout does not serve 2 people at the same time. You can try adding print statements for your server finishTimes/ queues and using the test cases provided in #758 to check whether your code works as intended.

had the exact same issue, after tracing my code i realised the issue came from my serve class. you may have implemented yours in a different way but the serve class is a good place to start.

make sure you are using the most updated server every time u process an event. i would suggest getting it from the list of servers every time rather than using this.server of the event as the list of servers may have been updated by other events.

okay thanks everyone! just figured out.

You need to check your self check outs. Can only queue and dequeue at the first self check out. Hence in your arrive class you need to check if it queues at SCO, queue at first SCO. Same for wait class and serve class when you dequeue


WIP - Remove "destructive approval" from our approval step for existing providers

This PR changes the behaviour to remove the default of deleting some data about a provider when we try to merge a new verification request into an existing provider.
Previously we had some logic like this:
if self.provider:
hp = Hostingprovider.objects.get(pk=self.provider.id)

# delete related objects, they will be recreated with recent data
hp.services.clear()

for asn in hp.greencheckasn_set.all():
    asn.delete()

for ip_range in hp.greencheckip_set.all():
    ip_range.delete()

for doc in hp.supporting_documents.all():
    doc.delete()

We want to preserve existing IPs and documents where possible, so this PR moves to archive these safely instead.

OK, we have some new logic now for this to avoid deleting existing IP ranges, ASNs and supporting evidence.
I want to document the approach being taken here, and see how it works on staging before we merge into prod, as I ended up introducing a new 'archived' property on uploaded evidence, that we check for.

 Update logic to archive obsolete information about a provider, not delete it


Introduces MongoDBSubmodelAggregator

Signed-off-by: Frank Schnicke<EMAIL_ADDRESS>

Builds on https://github.com/eclipse-basyx/basyx-java-sdk/pull/91


update webportal job submit form

Coverage increased (+28.9%) to 80.249% when pulling 1ac7f6ace1f9bcce07dbf59b5fa465be2113705c on zhaonan/fix into 64161acd14e3dac25acc9b2ca666e9d5fa8360ff on master.


Inconsistencies in overlap behavior?

Hi there,
I was doing some testing and it seems that there are some inconsistencies in how NCLS defines overlaps?
Given the following code snippet...
import numpy as np
from ncls import NCLS

starts = np.array([0, 20])
ends = np.array([10, 30])
ids = np.array([0, 1])

tree = NCLS(starts, ends, ids)

...the following results seem inconsistent:
tree.has_overlap(-1, 0)
>>> True

tree.first_overlap_both(np.array([-1]), np.array([0]), np.array([0]))
>>> (array([], dtype=int64), array([], dtype=int64)) # no results

As shown above, though -1, 0 is reported as having an overlap (using has_overlap), the same query interval fails to find an overlapping interval in the tree (using first_overlap_both).
There also seems to be some inconsistencies regarding, perhaps, the treatment of intervals containing 0?
Given this snippet...
tree.has_overlap(-1, 0)
>>> True

tree.has_overlap(19, 20)
>>> False

...it seems odd that -1, 0 matches 0, 10 (presumably) while 19, 20 doesn't match 20, 30.

Thanks for reporting. It is not meant to be used with negative positions though. What is your use-case for that?
I guess I should add a check to see that no negative numbers are used. 19, 20 does not match 20, 30 because ends are non-inclusive in Python :)

Yeah, it seems that -1, 0 shouldn't be matching any intervals that start with 0.
I don't actually have a use-case for negative numbers; I'm using NCLS for biological purposes, as it seems it was somewhat intended for. However, this inconsistency simply came up in my testing. I reported it because I thought it was rather strange.

I completely agree and I am happy you reported it. I actually only use the vectorized functions because they are so much faster.

Huh, I guess maybe I should be using first_overlap_both and then just checking for results in the interest of speed. Although, that doesn't lend itself too well to readability.

Yeah, if you have lots of queries represented with starts and ends in a
vector, that is faster :)
On Wednesday, July 3, 2019, Rohan Vanheusden<EMAIL_ADDRESS>wrote:

Huh, I guess maybe I should be using first_overlap_both and then just
checking for results in the interest of speed. Although, that doesn't lend
itself too well to readability.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/hunt-genes/ncls/issues/14?email_source=notifications&email_token=AEHURUSRMSGJNKRZRGNPTZDP5TPWPA5CNFSM4H4V22X2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZFFPWY#issuecomment-508188635,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEHURUTMKASSQUOTU5MCCLTP5TPWPANCNFSM4H4V22XQ
.


YAML Merge key << not supported

Issue seen in version 0.8.12
YAML Merge Key << is read as a String and merge is not performed.
code:
fn main() {
    let untyped = serde_yaml::from_reader::<std::io::Stdin, serde_yaml::Value>(std::io::stdin());

    println!("\nuntyped YAML read as:");
    println!("{:?}", untyped);
}

Running with:
Yaml with the merge key, <<:
.town_anchor: &town { town: greenbow }
name: brian
age: 42
other:
  << : *town
  some: field

Yaml with merge key in explicit form: !!merge '<<':
.town_anchor: &town { town: greenbow }
name: brian
age: 42
other:
  !!merge '<<' : *town
  some: field

Result in:
$ cat data-with-merge.yaml | cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.03s
     Running `target/debug/serde_yaml_anchors`

untyped read as:
Ok(Mapping(Mapping { map: {
    String(".town_anchor"): Mapping(Mapping { map: {
        String("town"): String("greenbow")
    } }),
    String("name"): String("brian"),
    String("age"): Number(PosInt(42)),
    String("other"): Mapping(Mapping { map: {
        String("<<"): Mapping(Mapping { map: {
            String("town"): String("greenbow")
        } }),
        String("some"): String("field")
    } })
} }))

$ cat data-with-merge-explicit.yaml | cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.04s
     Running `target/debug/serde_yaml_anchors`

untyped read as:
Ok(Mapping(Mapping { map: {
    String(".town_anchor"): Mapping(Mapping { map: {
        String("town"): String("greenbow")
    } }),
    String("name"): String("brian"),
    String("age"): Number(PosInt(42)),
    String("other"): Mapping(Mapping { map: {
        String("<<"): Mapping(Mapping { map: {
            String("town"): String("greenbow")
        } }),
        String("some"): String("field")
    } })
} }))

(I've tried to indent that to be more readable).
So it looks like the anchors are respected but not the merge key.

Four months later I've actually spent 10 minutes looking into this (rather than just complaining then getting on with my life):
This is down to the merge key not being supported in yaml-rust.
https://github.com/chyh1990/yaml-rust/issues/82
https://github.com/chyh1990/yaml-rust/issues/68
It's pointed out that the merge key is a YAML 1.1 feature and yaml-rust is specifically for YAML 1.2, which is stricter, and a request to support YAML 1.1 hasn't had much traction.
This crate is reasonable up front about being for YAML 1.2.  I'm going to use the yaml-merge-keys crate to get merge keys working in my application.
Would it make sense for serde-yaml to support merge keys, perhaps via a feature flag?  It would be an oddly specific fit.  A feature flag could be "YAML 1.1 Support" but that's far bigger than this issue.  Closing


HOW_WE_USE_GITHUB.md: Updates to working with tickets, labeling and general updates

Added section to How_We_Use_Github.md on Working on Tickets; updated information on labels to highlight mandatory labels, include information on label descriptions and provide a general overview of the three mandatory label categories; separated out information on the process of issue sorting from information on what information sorting actually is; added information on spikes; changed infra --> infrastructure, in line with current repo name

Description
Resolves #677

Added instructions on how to assign yourself to a ticket you are actively reviewing (see section Working on Tickets, beginning on line 293)
Added Instructions for how to use the source::* labels (see section What labels are required for each issue?, beginning on line 160)
Reviewed provided templates for answering an issue: no change made
Changed references to conda/infra to conda/infrastructure (current repo name)
Added information to section on spikes

@jezdez good call on conforming to using issue and not ticket
I remember struggling with this since its weird talking about a feature request as an issue or phrasing things like "if you have an issue, open an issue"


Cannot nest multiple index paths

Describe the bug
A route with path: '/' cannot be nested inside a route with path: '/'.
A good usecase for this is if you had a homepage with tabs. The homepage would be an index route, and then one of the tabs would also be an index route. The obvious way to do this would be with two nested index routes, but this shows a console error, and the app shows a blank screen.
Of course the alternative is to use a pathless route, or to just add an id. But I think that intuitively you want to add it as an index route, and it seems weird that it would error.
Your Example Website or App
https://stackblitz.com/edit/github-entwu2-qyrbic?file=src%2Fmain.tsx
Steps to Reproduce the Bug or Issue

Add an index route
Add another index route, with the first index route as the parent
Observe error

Expected behavior
I expect to be able to nest index routes in a simple usecase.
Screenshots or Videos

Platform

OS: MacOS
Browser: Chrome
Version: 0.0.1-beta.160

Additional context
No response

I can see why you would expect that to to work. The way the type-safe route matching works right now doesn't allow this , since both routes would have the same exact unique navigation identifier (the root path /). I guess it's reasonable that if you did do this, you would expect the types of the routes to be merged together (search params, loader data, etc)

Since it's possible to work around this right now, as you mentioned, by turning the wrapping route into a non-path route with id: tabs or something similar, I think I'd rather table this until 2.0, since it could significantly break the type/matching resolution.


Not working anymore with latest screenshot-basic version

Taking a screeenshot using latest screenshot-basic version now throw an error

None of the latest commit changes affect the exports. Are you sure you set up everything correctly?

From what I gather with the latest version the package can't be built anymore. The dist folder isn't created so the export doesn't exist.


Error 200 - realtime command

Hi,
I have the same behavior as issue 26, but for realtime command.
$ ./centctl list realtimeService --limit 10 -v unhandled -r "Disk.*"
ERROR: 200
If i do the same with debug mode, i have the body response which looks good, but with the error at the end.
Same thing with realtimeHost.
Environment:
Centreon 23.10.5
cenctl 2.5.8
debian 12
I am using v2 API version
Best regard

Hello,
The v2.5.9 should fix your problem
https://github.com/YPSI-SAS/centctl/releases/tag/v2.5.9
Can you test it and tell me if you still have the error?
Thanks
Regards

it works perfectly !
Thanks a lot


Gateway management in APIGateway CR

Description
API Gateway modules comes with default kyma gateway for a quick start. We encourage our users to configure their gateway with their own domain to serve production ready applications. So far gateway creation and configuration is only covered by documentation.
Describe proces should be automated and simplified byt extending APIGateway CRD. There should be possibility to create and configure Istio Gateway CR. Simple and mutual TLS should be supported, either with certificate provided by user as well as generated using Gardener Certificates.
TODO:

[ ] API proposal
[ ] implementation
[ ] rollout

Reasons
Simplify flow of Istio Gateway creation
Attachments
https://github.com/kyma-project/api-gateway/blob/main/docs/user/tutorials/01-20-set-up-tls-gateway.md
https://github.com/kyma-project/api-gateway/blob/main/docs/user/tutorials/01-30-set-up-mtls-gateway.md

Please include this use case: Create gateway with multiple hosts and mtls

I have added the fully fledged custom domains and gateways with multiple hosts and mtls to the btp runtime context automation...

https://gist.github.com/ptesny/be81b8f9c6a07f021284ab13ce5670cd#file-expose-and-secure-kyma-workloads-with-client-certificates-and-kyma-system-defined-mtls-gateways-md
https://gist.github.com/ptesny/d3fbbad2e7b6896d951ee95ea5e07447#file-mtls-gateways-with-custom-business-domains-md

However, that should rather part of the future btp kyma runtime environment configuration so kyma API gateway CR can self-configure accordingly during the kyma cluster provisioning


TLS SNI Support is missing

Is your feature request related to a problem? Please describe.
Currently, it doesn't seem possible to specify an SNI name when creating a connection, like you can with the redis-cli or redisinsight, meaning that if you are proxying a DNS server behind a service in a cluster, like Kubernetes, it's not possible to connect to that service with Redisson.
When attempting a connection with a SNI domain, Redisson states how it is unable to connect while spitting out numerous 400 bad request errors.
Describe the solution you'd like
I would like Redisson to support providing an SNI over TLS.

Should anyone run into this in the future, you can simply change redis:// to rediss:// and the issue immediately is solved, likely due to the extra s indicating an SSL connection, where the TLS/SNI kicks into action


[PUL-235]fix: excludes with code or gloabl

[ ] Ready for review
[ ] Linked to Jira ticket

What does this PR do?
This pr fixes the bug that you need to have a global and code field under excludes in the .snyk file to work

:tada: This PR is included in version 4.2.4 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Fix exception on node less than 6.x

@retyui, do you have a reason to stay on Node 6.x?

I am not, but my colleague is yes, and we spent a lot of time to understand why everything works for me and he doesn’t


Update main.py

What’s updated:
All 50 states and the District of Columbia have been added with their respective electoral votes, including adjustments for the 2020 census.
Voting Locations: The list of voting locations is unchanged, but votes are now randomly assigned a location from the list, enhancing the realism of the voting simulation.

you can work off that if you want


Can not adjust height

Hey there,
i have got everything working with this plugin, but unfortunately i can not set the height for the chart. If i adjust the width, it is working well. So lets say, setting the width to this parameters:

results in this size on the magic mirror:

However, if i change the height to 300%:

Nothing happens.

Changing the width to e.g. 500% is working well.

Anyone has an idea on that?

see #15 use px instead of %


CheckpointRestoreDirectory

Hi.
flag -crd with presets: Pendulum_DDPG, CartPole_PPO output AssertionError: 1 != 0

should be fixed in release 0.9: 125c7ee


[tool_crash] DartDevelopmentServiceException

Command
flutter run

Steps to Reproduce

...
...
...

Logs
DartDevelopmentServiceException
#0      DartDevelopmentServiceImpl._startDDSServer (package:dds/src/dds_impl.dart:213:7)
<asynchronous suspension>
#1      DartDevelopmentServiceImpl.startService (package:dds/src/dds_impl.dart:116:7)
<asynchronous suspension>
#2      DartDevelopmentService.startDartDevelopmentService (package:dds/dds.dart:97:5)
<asynchronous suspension>
#3      DebugService.startDartDevelopmentService (package:dwds/src/services/debug_service.dart:156:12)
<asynchronous suspension>
#4      DevHandler._createAppDebugServices (package:dwds/src/handlers/dev_handler.dart:468:7)
<asynchronous suspension>
#5      DevHandler.loadAppServices (package:dwds/src/handlers/dev_handler.dart:244:27)
<asynchronous suspension>
#6      Dwds.debugConnection (package:dwds/dwds.dart:91:30)
<asynchronous suspension>
#7      WebDevFS.connect.<anonymous closure> (package:flutter_tools/src/isolated/devfs_web.dart:694:15)
<asynchronous suspension>

[✓] Flutter (Channel stable, 3.3.6, on macOS 12.6.1 21G217 darwin-x64, locale en-IN)
    • Flutter version 3.3.6 on channel stable at /usr/local/Caskroom/flutter/3.3.1/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 6928314d50 (7 days ago), 2022-10-25 16:34:41 -0400
    • Engine revision 3ad69d7be3
    • Dart version 2.18.2
    • DevTools version 2.15.0

[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
    • Android SDK at /Users/r0s0bkm/Library/Android/sdk
    • Platform android-33, build-tools 33.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 13.4.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 13F100
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2021.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.12+0-b1504.28-7817840)

[✓] IntelliJ IDEA Community Edition (version 2022.1.4)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] VS Code (version 1.72.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.52.0

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-x64     • macOS 12.6.1 21G217 darwin-x64
    • Chrome (web)    • chrome • web-javascript • Google Chrome 107.0.5304.87

Flutter Application Metadata
Type: app
Version: 1.0.0+1
Material: true
Android X: false
Module: false
Plugin: false
Android package: null
iOS bundle identifier: null
Creation channel: stable
Creation framework version: 6928314d505d2bb4777be05e45d7808a5aa91d2a

Closing this as a duplicate of https://github.com/flutter/flutter/issues/97604.


Fix DNA scrambler ending nukie rounds early

About the PR

resolves #18145
This PR gives the scrambled entities the NukeOperatice component.
Apparently entities in a paused map don't count as alive, so the DNA scrambled entities need the NukeOperative component.
This prevents the game thinking all nukies are dead the moment it checks for alive operatives when one of them gets killed.
Media

[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Changelog

:cl: Dygon

fix: DNA scrambled nuclear operatives now count as alive for the end of round check.

good solution will remove the scrambled prototype so imo this is a fine bandaid considering it can end nukie rounds early

I can probably just remove the scrambled prototype and make it turn someone into MobHumanNukeOp since only the name is currently different.

shouldn't they also still be a syndie faction member?

I feel like the polymorph method is the real issue here. Components should be whitelisted for getting moved over to the new entity.

why is it even polymorphing an entity instead of just randomizing the appearance? It's like one function call to do so.

why is it even polymorphing an entity instead of just randomizing the appearance? It's like one function call to do so.

A wizzard polymorphing a nukeops into a chicken and back should not cause the same issue I think.

A wizzard polymorphing a nukeops into a chicken and back should not cause the same issue I think.

Nuclear operatives can safely all polymorph without the round ending, the issue is that whenever an operator dies the game starts a check to see if it was the last operative. If it can't find any entity in an unpaused map with the NukeOperative component it will end the round. The original bodies with the NukeOperative component are placed in a paused map upon polymorphing  so they won't be seen by the check.
This PR is a bandaid to at least remove the big issue of rounds ending too early while I try to rework it.
Another option is temporarily disabling the ability to buy the implant for nukies.


[bug] Syntax error on linux .desktop file

Describe the bug
I'm on arch linux based distribution. Once installed Onlook cannot be run from UI menu because of a syntax error in /usr/share/applications/onlook.desktop file which has the following content:

Please note the leading " at the end of the binary name

[Desktop Entry]
Name=Onlook
Exec=onlook" %U
Terminal=false
Type=Application
Icon=onlook
StartupWMClass=Onlook
Comment=The first-ever devtool for designers
Categories=Utility;

Thanks for reporting. This may require setting better options in the linux target. Truthfully we don't really have the capacity to test on Linux atm.
https://github.com/onlook-dev/onlook/blob/main/apps/studio/electron-builder.json5#L37


Recompile plugin with steampipe-plugin-sdk v5.6.0 and Go version 1.21

Dependencies updated in https://github.com/turbot/steampipe-plugin-exec/commit/21b57674283473916a67c89436613258fcee9449, closing this PR.


console window (v215) ERROR: ObjectTypeDB::bind_integer_constant: Method/Function Failed.

In the console window when running windows-engine-215-64bit.exe (...32bit.exe too) this error message will appear 5 times in a row:
ERROR: ObjectTypeDB::bind_integer_constant: Method/Function Failed.
At: core\object_type_db.cpp:413
When running either the "official" Godot_v2.1.5-stable_win64.exe build or windows-engine-306-64bit.exe build there is no error. I've not gotten far on my project and I haven't noticed any actual problems/bugs but when searching "core\object_type_db.cpp:413" I do find https://itch.io/t/343010/error-trying-to-run-on-linux where someone's Godot based game on Linux isn't running. Which version of Godot is used isn't mentioned so I don't know if there would be any similar issues though. I'm using v2.1.5 for its older openGL support. Thanks for any help or clarification regarding this and thank you for your work in creating this branch of Godot.

Hmm, I haven't actually noticed it before but I'll check it out.  I haven't had any actual issues arise though, nor none reported with any games using it.  There might be some weird bug in a function that is rarely or never actually used.
I will check into it and report back my findings!

Thank you for checking it.

Sorry for the delay.     I am going to update the 2.1.5 branch with changes present in the 3.0.6 branch (usually try to keep them similar anyways) to see if that fixes the issue as there really isn't any way to track down what is causing that error.
I hopefully will have something pushed to Github soon-ish.

OK, so I think I solved the error and found out why it didn't have any impact.  It seems that five of the BIND_METHODS were either missing or didn't have corresponding ENUMs in the godotsteam.h file.  I didn't both tracking them down, just replaced both lists with the ones from the Godot 3 branch and the errors went away.
Whatever functions used those constants were either never used by anyone or the lack of a bound constant just... assumes 0?
Either way, I'm gonna push a change to the repo sometime today after I get a Windows version compiled to make sure there aren't any issues on that platform, as well as get some new pre-compiles put together for folk.  Thanks again for bringing this to my attention!

Changes have been pushed and pre-compiles updated!


can't close tabs on ipad

closing the tab will always prompt a rename. this is a new bug that doesn't occur on latest release.

I assume this is when tapping the 'x' rather than with the cmd+w keyboard shortcut

@tvanderstad Yup, just when pressing the "x".


Please whitelist my Twitter account @amadeobrands

https://twitter.com/amadeobrands

Whitelisted in https://github.com/MrLuit/EtherScamDB/pull/1010


Storyskip not working

I just redownload the newest script
The function like choose card, auto battle, skip result etc working without error, but when it come to the story, when the skip button come theres nothing happen like clicking randomly on the screen, its like the storyskip feature not working or maybe is just happen in my phone
So i just want to ask any advice for this problem
Im already change the story skip from 0 to 1
My phone is Redmi note 7
Sorry for my bad English

Already solve my problem xD

How did you solve it?

How did you solve it?

In my phone there's an app game booster
This app will automatically auto play when you playing game
And when playing game theres a hidden button in left screen
When i active ankulua to try the skipstory is active or not
Those button always blinking non stop, so i think the script found that hidden button is same as button skipstory
So i try to turn of the game booster feature and try again
And everything work fine


test: Add unit tests to artifacts.rs module

Closes #2625
Introduced changes
Tests directory in crates/scarb-api/tests/data/basic_package for integration tests starknet artifacts.

Integration tests artifacts only get built when there's a tests directory in a cairo package
Checklist

[x] Linked relevant issue
[x] Updated relevant documentation
[x] Added relevant tests
[x] Performed self-review of the code
[x] Added changes to CHANGELOG.md

Hi @cptartur This particular test keeps failing, tests::package_matches_version_requirement_test

Hi @cptartur This particular test keeps failing, tests::package_matches_version_requirement_test

Rerunning tests, this shouldn't fail and there's nothing in your code that should cause that.

Okay, thank you @cptartur you've been great all through. ✊

When I did, a couple previous tests were failing, so I resorted to generating the file using indoc.
failures:

---- tests::get_starknet_artifacts_path_for_test_build_when_integration_tests_exist stdout ----
thread 'tests::get_starknet_artifacts_path_for_test_build_when_integration_tests_exist' panicked at crates/scarb-api/src/lib.rs:325:36:
called `Result::unwrap()` on an `Err` value: Os { code: 17, kind: AlreadyExists, message: "File exists" }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- tests::package_matches_version_requirement_test stdout ----
thread 'tests::package_matches_version_requirement_test' panicked at crates/scarb-api/src/lib.rs:421:9:
assertion failed: !package_matches_version_requirement(&scarb_metadata, "starknet",
            &VersionReq::parse("2.8").unwrap()).unwrap()

---- tests::get_starknet_artifacts_path_for_test_build stdout ----
thread 'tests::get_starknet_artifacts_path_for_test_build' panicked at crates/scarb-api/src/lib.rs:307:9:
assertion `left == right` failed
  left: StarknetArtifactsFiles { base: "/tmp/.tmp3EXCX4/target/dev/basic_package_integrationtest.test.starknet_artifacts.json", other: ["/tmp/.tmp3EXCX4/target/dev/basic_package_unittest.test.starknet_artifacts.json"] }
 right: StarknetArtifactsFiles { base: "/tmp/.tmp3EXCX4/target/dev/basic_package_unittest.test.starknet_artifacts.json", other: [] }

failures:
    tests::get_starknet_artifacts_path_for_test_build
    tests::get_starknet_artifacts_path_for_test_build_when_integration_tests_exist
    tests::package_matches_version_requirement_test

test result: FAILED. 10 passed; 3 failed; 0 ignored; 0 measured; 0 filtered out; finished in 21.40s

These were the failing tests.
So I adjusted test_load_contracts_artifacts to this
#[test]
    #[cfg_attr(not(feature = "scarb_2_8_3"), ignore)]
    fn test_load_contracts_artifacts() {
        let temp = crate::tests::setup_package("basic_package");
        let tests_dir = temp.join("tests");
        fs::create_dir(&tests_dir).unwrap();

        temp.child(tests_dir.join("test.cairo"))
            .write_str(indoc!(
                r"
                #[test]
                fn mock_test() {
                    assert!(true);
                }
            "
            ))
            .unwrap();

        ScarbCommand::new_with_stdio()
            .current_dir(temp.path())
            .arg("build")
            .arg("--test")
            .run()
            .unwrap();

        // Define path to the generated artifacts
        let base_artifacts_path = temp.to_path_buf().join("target").join("dev");

        // Get the base artifact
        let base_file = Utf8PathBuf::from_path_buf(
            base_artifacts_path.join("basic_package_integrationtest.test.starknet_artifacts.json"),
        )
        .unwrap();

        // Load other artifact files and add them to the temporary directory
        let other_files = vec![Utf8PathBuf::from_path_buf(
            base_artifacts_path.join("basic_package_unittest.test.starknet_artifacts.json"),
        )
        .unwrap()];

        // Create `StarknetArtifactsFiles`
        let artifacts_files = StarknetArtifactsFiles::new(base_file, other_files);

        // Load the contracts
        let result = artifacts_files.load_contracts_artifacts();
        println!("{result:?}");

        // Ensure no errors and non-empty result
        assert!(result.is_ok());

        // Assert the Contract Artifacts are loaded.
        let artifacts_map = result.unwrap();
        assert!(!artifacts_map.is_empty());
        assert!(artifacts_map.contains_key("ERC20"));
        assert!(artifacts_map.contains_key("HelloStarknet"));
    }
    ```

    And sorry I forgot I didn't push that change... Just Did

![Screenshot_20241205_145214](https://github.com/user-attachments/assets/f0d55acd-38f6-4252-964d-82e5ad47c55d)

Done @cptartur

@Abeeujah seems that lint and fmt failed, please run ./scripts/scarbfmt.sh to fix format and run cargo lint and address any issues that it throws out.
Also it seems that the file you added to basic_package broke get_starknet_artifacts_path_for_test_build and get_starknet_artifacts_path_for_test_build_when_integration_tests_exist. I'd remove the file and use the original approach you did with .write_str to a created file to create test.


Time series based workload sort optimization through Lucene segment sorting in DocumentReader

Observation
I have noticed through http_logs worklad of OSB, that desc sort on @timestamp field is taking a lot more then asc order sort. There is definitly an improvement we can do here arranging lucene segments in respective order of asc/desc based on sort type specified. We can address this on query time and before I put down my POC numbers & performance gains we got with this, let me give you context.
For the context,
We enabled NumericField Point based sort optimiaztions for all NumericTypes in these 2 PRs (6321 & 6424).
As mentioned in PRs, the performance gains we got through this was a lot, as it skips iterate/compare over non-competitive documents based on its Point values. The basic concept here is, we give a target value and try to find 'k' closest points to these target value and skip other value which we consider non-competitive. Lucene already does it on _score values. now instead of _score, we do it on Point values through above optimizations.
Suggested fix
Now in time series based workload, since all latest timestamp comes in last for a shard in its segment arrangment, the desc query tends to take lot of time and if we reorder them in descending order at DocumentReader level, we can skip iterating over non-competitive segments which were aligned ahead of competitive segments in case of time series event.
Descending/ascending order document retrieval in timeseries workload is very common scenario and based on POC, I see we are gaining 3x to 4x on this optimization.
All we need to do is, at query time, we need to provide sort order for segment here in subReaderSorted while creating DocumentReader object.
Below are the numbers for my POC.  Sorting is performed on @timestamp field
This work load is used (http_logs)
The operation was performed on single index size of 12.9 G with 4 shards with 136 segments on it.

Sort Type
hits = 5
hits = 100
hits = 1000

ASC
11
14
693

ASC (in POC)
10
11
11

Desc
185
670
995

Desc (in POC)
92
250
323

If you see, we are gaining anywhere between 3x to 4x here on @timestamp field sorting.
Few considerations before implemeting this,

This will be only optimizing wokloads which has time series and getting ingested in particular order. Worth doing this only for timeseries workloads ? How common are those workloads or scenarios for our users.
The segment merge scenario, it can still alter the sequence of documents (I am testing this what could be the impact here)
Can we provide user an alternative to IndexSort field at API level to define this optimization ? with clear documentation of when this should be used. Index sort definitly user want to avoid since it adds overhead on ingestion time.

@dblock @reta @nknize @backslasht @Bukhtawar  , what do you guys think on this ?

@gashutos thank you, have you seen this one TimeSeries optimizations in OpenSearch ?
On more general note, the improvements are looking really worth the optimization, but I do not see clear means on how we could right conclude that the index fits this type of workload (== has time series and getting ingested in particular order), unless we restrict it to data streams where @timestamp is explicit (Elasticsearch shared similar optimizations https://www.elastic.co/blog/optimizing-sort-queries-in-elasticsearch-for-faster-results).

@reta I think that proposal has some limitations with Point based sort optimizations we have now. By merging dimensions, we will lose ability to have them sort based on Point Based values.
I think ir-respective of that, the code change with this approach are very less and gets sizable improvement, while we fine grain above approach, we can productize and improve with mentioned approach, since changes are within OpenSearch repo itself.

@gashutos - This looks like a promising optimization, few questions.

Echo'ing @reta question, how to determine from the workload if this optimization needs to be enabled or not.
What is the performance impact for other workloads where this optimization is not applicable, especially when the number of segments is high.

Tagging @nknize

Couple of quick questions:

Have you benchmarked with the ShuffleForcedMergePolicy? If so, can you post those here? If not, please do and post here?

I think that proposal has some limitations with Point based sort optimizations we have now
You're talking about two different sorts, Segment vs Field. The "point based sort optimizations" you're referring to uses the BKD tree through IndexOrDocValuesQuery to skip non-competitive documents at query time. Segment sorting is more about Segment Geometry and how the segments are visited / merged, which I think is what you're discussing in this issue (hence me asking about the shuffle merge policy benchmarks).

Brainstorming out loud. What about concurrent search? Could we add an enhancement to  the concurrent-search sandbox module that searches segments concurrently in something like a crossing pattern? (e.g., threads in reverse and threads forward)

@nknize @backslasht thank you for looking at proposal.
@nknize  while I am checking 1st & 3rd one, just quick clarification on 2nd one...

I think that proposal has some limitations with Point based sort optimizations we have now

You're talking about two different sorts, Segment vs Field. The "point based sort optimizations" you're referring to uses the BKD tree through IndexOrDocValuesQuery to skip non-competitive documents at query time. Segment sorting is more about Segment Geometry and how the segments are visited / merged, which I think is what you're discussing in this issue (hence me asking about the shuffle merge policy benchmarks).

This I was talking about  https://github.com/opensearch-project/OpenSearch/issues/3734 proposed by Rishabh. That appraoch has some contradiction with point based optimization from the high level it looks like to me.

This I was talking about #3734 proposed by Rishabh.

Ah, it's been a while since we worked that one!  So at the Lucene level that's for dedicated TS* field types and the Lucene BKD changes include rolling up summary stats so you can easily filter based on the precomputed (per segment) stats. There's still a lot of outstanding questions around that one so I'd actually consider that separate from this issue for now. Any improvements you make on this issue can be rolled back into that design, or alternatively you could leverage the precomputed stats to more efficiently sort the segment geometry. If that ends up helping it would be a good reason to get the BKD change POC merged upstream in lucene.
I'd start first by exploring the shuffle merge policy and benchmarking to assess any gains there. I definitely have reservations about forcing a specific asc desc segment order because inevitably a user will want to sort results and access the documents in the opposite direction. Shuffle merge is suppose to converge on the average for all search cases.

@nknize I did benchmarking for ShuffleMergePolicy, I think numbers are not promising. I take away two things from it,

Desc order sort is getting improved 3x as we can see below (but that is only when we do force merge to single segment, in real life scenario where customer do not do force merge, there is no notable difference in desc sort).
ASC sort is getting regressed by 3x.

Below are the results aI collected from http_logs workload on single shard with 3.7 G size.

Sort Type
hits = 1000

ASC
35

ASC (With Shufflemerge)
140

ASC (With Shufflemerge with forced merge to single segment)
450

Desc
525

Desc (With Shufflemerge)
450

Desc (With Shufflemerge with forced merge to single segment)
181

I think, this policy we can improve more within segment merge itself but still if we look at real production scenario, we will still end up having segments (group of segments) in asc order of timestamp, newer segments would be latest.
That's why I was thinking, since we aren't regressing anything in "Segment order sort" approach during query time (subReadersSorter) we can productize that first and I believe we still can make some improvements in segment merging mechanism for time serius based workloads.

I definitely have reservations about forcing a specific asc desc segment order because inevitably a user will want to sort results and access the documents in the opposite direction
I see the code right now, the way we create IndexWriter and DocumentReader from it only once upon boot up. I was thinking if we can make this sorting order based on sort type during query time, then your concern would be justified right ? let me dig into that more and find if we can have a way around for that....

@backslasht @reta ,
To your first question,

Echo'ing @reta question, how to determine from the workload if this optimization needs to be enabled or not.

I think we may not need to bind this segment ordering to only one type of workload (time series based). If data is uniformerly distributed across segments, order of traversing through segments wont matter. I did this exercises in nyc_txy workload on total_amount/tip_amount fields and I dont see any regression over queries there. This optimization we will only apply when query types has Sort and w.r.t its sort order we will sort segments by its min/max values.

What is the performance impact for other workloads where this optimization is not applicable, especially when the number of segments is high.
This I am still running and will probably have results in some time, will keep posted here.

@gashutos FYI, I'm reopening this issue because the implementation was reverted prior to the 2.8 release due to #7878.


Add command scaffolding and build scripts

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@20586c3). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main       #1   +/-   ##
=======================================
  Coverage        ?   75.51%
=======================================
  Files           ?        2
  Lines           ?       49
  Branches        ?        0
=======================================
  Hits            ?       37
  Misses          ?        9
  Partials        ?        3

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 20586c3...024bb5f. Read the comment docs.


Je/update dependencies

@tgoyne We have changed Core in the way that it is not allowed to create Table::create_object() on a table with primary key (you should call Table::create_object_with_primary_key()). This seems to create some problems in some migration tests. Would it be a solution to always remove the possible primary key before doing the migration?

Removing all primary keys at the start of a migration and then restoring them at the end is effectively what we do with core 5. I'm not sure that there's actually a good reason to be doing that, though. When adding a primary key to an existing type we can't enforce uniqueness until after the migration has run (as existing objects won't already have unique values if it's a new property), and when we first implemented primary keys the easiest way to achieve that was to just not enforce PK uniqueness during migrations at all.
I think the right thing to do is probably that if a table's primary key is being changed, remove it entirely before the migration and set it to the new pk after the migration, and otherwise don't touch it. There's probably some tests that'll need to be adjusted for this.

@tgoyne Are these changes ok?


JSON parser running on Content-Type: application/x-www-form-urlencoded requests

Expected Behavior
app.use(express.json()); should not parse requests with the Content-Type header set to application/x-www-form-urlencoded
Current Behavior
app.use(express.json()); parses requests with the Content-Type header set to application/x-www-form-urlencoded
Steps to Reproduce
servers.js:
const app = express();
app.use((req, res, next) => {
  console.log('BEFORE');
  console.log(req.body);
  next();
});

app.use(express.json());

app.use((req, res, next) => {
  console.log('AFTER');
  console.log(req.body);
  next();
});

Running:
$ curl -X POST \   http://localhost:3000/api/login_check \   -H 'Accept: application/json' \   -H 'Content-Type: application/x-www-form-urlencoded' \   -H 'Host: localhost:3000' \   -d 'email=my_email%40gmail.com&password=123456'
Outputs:
BEFORE
undefined
AFTER
{}

Context (Environment)
Express: 4.16.4
OS: Windows 10
Node: 8.11.1
Npm: 6.7.0
Detailed Description
When I add a parser for POST requests having a JSON body, it seems to also be executed when the server receives a application/x-www-form-urlencoded POST request, which I understand should not happen. After the parser, the req.body gets set to an empty object, whereas it should in my opinion not be modified by the JSON parser.
Am I doing something wrong in the way I use the JSON parser ?
Thanks in advance :)

It is not parsing that body at all, rather it is setting the empty object because it us not a json request.
From http://expressjs.com/en/4x/api.html

A new body object containing the parsed data is populated on the request object after the middleware (i.e. req.body), or an empty object ({}) if there was no body to parse, the Content-Type was not matched, or an error occurred.

Since the content type was not matched, it will set req.body to an empty object.


opera: 45.0.2552.898 -> 46.0.2597.39

updated to 46.0.2597.39
gtk2 dependency changed to gtk3
32-bit package is no longer supported

Upstream dropped 32bit?

Since 46 only 64-bit builds are provided: http://www.opera.com/download/index.dml/?os=linux-i386&list=all
Chromium base could be the reason: https://github.com/NixOS/nixpkgs/pull/26694
Vivaldi still offers both variants, though. So maybe Opera will support 32-bit again.


Add compiler-tester on CI

Add the era-compiler-tester as a submodule and add a CI workflow that runs the tester against our VM. For now it should only run the yul_instruction tests like so:
cargo run --verbose --features lambda_vm --release --bin compiler-tester -- --path  tests/solidity/simple/yul_instructions/ --target EraVM --disable-deployer --mode='Y+M3B3 0.8.26' --load-system-contracts system-contracts-stable-build

Closed by #74


segmentation fault on opening possibly corrupted jpeg file.

Problem
I was using OIIO for my application and I've noticed that OIIO causes my program to throw a segmentation fault when it was trying to open a certain jpeg file. GDB reports the segmentation fault comes from jpeg_alloc_quant_table () from libjpeg.so.8.
The stack looks like:
https://gist.github.com/tomansill/0cfbed5784e0467363da3765a0dc2420
I've investigated my code to see if it was causing OIIO to throw a segmentation fault but I cannot seem to find it. I tried to open the same image that caused my program to break, with oiiotool to see if the problem is limited to my application. It turns out that it is not limited to my application as it threw a segmentation fault also.
My terminal shows:
indigo% oiiotool 34535.jpg -resize 8x8 -o resized.jpg
zsh: segmentation fault (core dumped)  oiiotool 34535.jpg -resize 8x8 -o resized.jpg
Running GDB shows (with where command):
https://gist.github.com/tomansill/c50c940cacdab408ef43dbacac22e61a
The problem could be coming from libjpeg so I've decided to test for that also to confirm the source of the problem. I tried djpeg program that is reportedly to be using libjpeg. It did not crash but it reports that the jpeg file is corrupted.
My terminal shows:
indigo% djpeg -scale 1/2 34535.jpg > out.jpg
Corrupt JPEG data: 2 extraneous bytes before marker 0xdb
I was able to open the corrupted jpeg file in Nautilus File Manager and Eye of Gnome Image Viewer just fine.
Expected behavior:  No segmentation fault and it should either fix and continue silently or throw exception.
Actual behavior:  Segmentation fault
Steps to Reproduce

Obtain an corrupted jpeg file (I've attached a zip file containing the corrupted jpeg file, never mind the content of the file)
Use oiiotool on the file with commands similar to oiiotool file -resize 8x8 -o out
Observe the segmentation fault

Versions

OIIO branch/version: 1.7.13
OS:  Arch Linux x64
C++ compiler: Installed using pacman with precompiled package

Corrupted file download: corrupted_jpeg.zip

UPDATE:
I've been successfully reproduced the problem with latest OIIO 1.8.5 that was pulled from master branch of this repository and compiled with GCC 7.1.1

Good news, I have a fix for this old bug: #2073