Question: How do I use Migrations with Webpack?

I can't seem to figure out the correct way to pass migration instances instead of paths to the migrations configuration which is supported by umzug. I'm using webpack so I can't do it based on paths.
Do you have an example I can use?

I'm also currently getting:
Error: Cannot find module 'typescript'
Which I think is coming from ts-morph, even though I've done this:
import { MikroORM, ReflectMetadataProvider } from 'mikro-orm';

const orm = await MikroORM.init({
	//...
	metadataProvider: ReflectMetadataProvider,
	discovery: { disableDynamicFileAccess: true },
});

The "stack" if you could call it that is:
{
  "errorType": "Runtime.ImportModuleError",
  "errorMessage": "Error: Cannot find module 'typescript'\nRequire stack:\n- /var/task/index.js\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js",
  "trace": [
    "Runtime.ImportModuleError: Error: Cannot find module 'typescript'",
    "Require stack:",
    "- /var/task/index.js",
    "- /var/runtime/UserFunction.js",
    "- /var/runtime/index.js",
    "    at _loadUserApp (/var/runtime/UserFunction.js:100:13)",
    "    at Object.module.exports.load (/var/runtime/UserFunction.js:140:17)",
    "    at Object.<anonymous> (/var/runtime/index.js:43:30)",
    "    at Module._compile (internal/modules/cjs/loader.js:955:30)",
    "    at Object.Module._extensions..js (internal/modules/cjs/loader.js:991:10)",
    "    at Module.load (internal/modules/cjs/loader.js:811:32)",
    "    at Function.Module._load (internal/modules/cjs/loader.js:723:14)",
    "    at Function.Module.runMain (internal/modules/cjs/loader.js:1043:10)",
    "    at internal/main/run_main_module.js:17:11"
  ]
}

Of course /var/task/index.js is my webpack bundle, so I'm not really sure where this is coming from.

Hmm hard to tell, never used webpack with the ORM myself, maybe @jsprw will have an idea?

I can't seem to figure out the correct way to pass migration instances instead of paths to the migrations configuration which is supported by umzug. I'm using webpack so I can't do it based on paths.

I don't think this will be supported out of the box just because you can do this with umzug. I am using custom migration runner.

Error: Cannot find module 'typescript'

Just a guess, but make sure you also do not have useTsNode enabled in your package.json.
If you prepare a repro repository, I will be happy to take a look myself.

I've started simple with the repro. The current error is that the migrations directory doesn't exist.
https://github.com/thekevinbrown/mikro-orm-webpack-migrations
If this isn't supported in Mikro ORM that's fine. What I was hoping for was to get TypeORM style auto-generation of migrations when I change my entities, then to be able to bundle those up with webpack and run them in a lambda.
Cheers!

Hmm I am not sure if there is a way to get this working. You can pass migration instances to umzug, but you will never be able to use the up() method to migrate to the latest version. It works solely based on the file system, it looks for files in given directory that were not yet executed. In the webpack context, everything is one file, you do not know whether there is actually a new migration available or not. It least that is how I understand it...
The issue you linked is open still, and the solution with passing Migration class is not working for me, but I might have done something wrong.
To get around this, you would have to implement your own Migrator that will not require things but rather accept a list of migrations. The class is relatively small, and all its dependencies should be usable so you really only need to change the require logic Migrator.resolve method.
https://github.com/mikro-orm/mikro-orm/blob/master/lib/migrations/Migrator.ts
Going forward, I will be happy to make it protected or add another way to achieve webpack compatibility, if we find the correct way.

I was able to use the schema generator to integrate with my own migrations process. It generates migrations then manages an index.ts file so that we can always:
import * as Migrations from './migrations';

Object.values(Migrations);

Each migration has a sort key so they come out deterministically, then I built a super lightweight runner.
I'm not sure how much of this belongs in Mikro ORM vs in my own project in the long term, but that's working for me so far.

I provided a solution for that in this PR: https://github.com/sequelize/umzug/pull/199 . I just had the same problem and fixed it.
If you want a quick work around here is the key function to convert migrations: https://github.com/nicolas-van/umzug/blob/master/src/migrationsList.js

What about this, do you think we need some changes in the ORM to support this? Have you seen that recent issue in umzug repo?
I think it would be nice to have a working example repo for this use case, if not a direct support in the ORM.

Sorry for the delay.
I ended up using SchemaGenerator by itself and building my own lightweight runner and CLI that generates these migrations and adds them to an index.ts file because I couldn't get anything else working.
So the answer is no, I haven't dug too deeply into trying to get umzug working in this context, I've just built my own tooling around it.

I've ported our tooling over (which does not currently use Mikro's migrations support, but does work with webpack) into this repository: https://github.com/thekevinbrown/express-ts-webpack-example-app
@B4nan, what are the next steps here from your perspective? I can see three ways forward:

We might decide that supporting this use case in the migrations framework in Mikro isn't realistic. In this scenario I'd transfer the express-ts-webpack-example-app into the mikro-orm organisation and we'd document this on the migrations documentation.
Mikro might require some changes to support this use case. In this scenario, I'd be happy to help with any work I can, it'd be good to know what you're expecting as work that's needed here.
If Mikro's ready to go and I just need to leverage Mikro a bit differently to get this to work, let me know and I'll apply those changes to the example app repo, then I'll transfer it into the mikro-orm organisation.

Thanks!

Looking into that migrations thing, we definitely need to consolidate and reuse what is already here. That is why I suggested to extend (or reimplement) the Migrator class, but use the rest (but the ideal solution is to adjust it to accommodate the webpack problem). The resolve method there is now protected, so you can do whatever needed there instead of that hard require. Generating new migrations is definitely something to be reused too.
Ideally we should find a way to use that new migrations list feature (https://github.com/sequelize/umzug/pull/199).
Thinking about that index.ts file where you store the list, it could be a bit error prone (1. create a migration, 2. remove it, 3. index.ts keeps reference to it so it will break everything). Could we use a JSON file for that or do you really need a TS file with references to the migration classes? We can live with that, but it would have to be enabled by default and this possible problem with removing migrations would have to be noted in the docs.

Closing due to inactivity (open for PR tho, if you find a nice way to allow this).


getToken dont do nothing

var url_parts = url.parse(req.url, true);
		var query = url_parts.query;
		var json = 'dor';
		var code = query.code;

		console.log(code)

		response.writeHead(200, {"Content-Length": code.length});
		response.writeHead(200, {"Content-Type": "application/json"});
		response.writeHead(200, {"Access-Control-Allow-Origin": "*"});
		response.end(code.toString());

		oauth2Client.getToken(code, function(err, tokens) {
	      // set tokens to the client
	      // TODO: tokens should be set by OAuth2 client.
	      oauth2Client.setCredentials(tokens);
	      console.log(code);
	    });

does nothing at all

When i dived deep into it i found that my server cant do any https request but only http. any help?
server is ubuntu running from azure pack.

Can be closed. The issue was that the mtu on server was too high for google.


'set' object has no attribute 'append' error

Dear all
I first performed the run_test option of mantis and it worked fine. Then I set up python mantis run_mantis with test_sample.faa and it also worked fine. However, when I tried to run my own test samples (also .faa) of around 30,000 contig amino acid sequences, it gave me the following error:
Traceback (most recent call last):
  File "/mypath/miniconda3/envs/mantis_env/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/mypath/miniconda3/envs/mantis_env/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "mantis/source/Multiprocessing.py", line 579, in worker_interpret_output
    self.generate_integrated_output(output_annotation_tsv, interpreted_annotation_tsv)
  File "mantis/source/Metadata.py", line 281, in generate_integrated_output
    output_annotation = self.read_and_interpret_output_annotation(output_annotation_tsv)
  File "mantis/source/Metadata.py", line 193, in read_and_interpret_output_annotation
    ref_file_links = self.get_hit_links(links_to_get[ref_file], ref_file)
  File "mantis/source/Metadata.py", line 125, in get_hit_links
    self.get_common_links(hit, dict_hits[hit])
  File "mantis/source/Metadata.py", line 66, in get_common_links
    self.add_to_dict(res, 'pfam', pfam)
  File "mantis/source/Metadata.py", line 39, in add_to_dict
    dict_hits['link'][dict_key].append(i)
AttributeError: 'set' object has no attribute 'append'
Ran into an issue, check the log for details. Exitting!

I notice two things that are different between my .faa file and the test_sample.faa:

the fasta IDs for the test_sample.faa do not contain special characters, whereas my .faa file would have sequences containing IDs like that >seq1 # 4012 # 4944 # 1 # ID=1_6;partial=00;start_type=ATG;rbs_motif=None;rbs_spacer=None;gc_cont=0.666

my fasta sequences contain an asterisk at the end of the sequence, indicating a stop codon

Could either one of these two factors contributed to the error? If not, what could be giving this error? Thanks
Marcus

Hello @mhyleung
I've been refactoring and making some improvements and it seems I forgot to change that particular function.
In any case, please redownload Mantis and it should now work.
Please keep in mind sequence IDs such as >seq1 # 4012 # 4944 # 1 # ID=1_6;partial=00;start_type=ATG;rbs_motif=None;rbs_spacer=None;gc_cont=0.666 are converted to >seq1.
This is done to save memory and to avoid parsing errors.
Sorry for the inconvenience.
Regards,
Pedro

Hi Pedro
Thanks for the message. I have removed and reinstalled mantis. Actually same as last time, I ran the test, and it gave
Traceback (most recent call last):
  File "/scratch/marcusleung/tools/miniconda3/envs/mantis_env/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/scratch/marcusleung/tools/miniconda3/envs/mantis_env/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "mantis/__main__.py", line 241, in <module>
    run_mantis_test(target_path=add_slash(MANTIS_FOLDER + 'tests')+ 'test_sample.faa',
  File "mantis/source/MANTIS.py", line 104, in run_mantis_test
    mantis.run_mantis_test()
  File "mantis/source/utils.py", line 622, in wrapper
    res = f(self, *args, **kwargs)
  File "mantis/source/MANTIS.py", line 444, in run_mantis_test
    self.generate_translated_sample()
  File "mantis/source/MANTIS.py", line 398, in generate_translated_sample
    translation_tables = parse_translation_tables(ncbi_resources + 'gc.prt')
  File "mantis/source/utils.py", line 1168, in parse_translation_tables
    with open(ncbi_tables) as file:
FileNotFoundError: [Errno 2] No such file or directory: '/media/basecamp/Data2/scratch2/marcusleung/tools/mantis/Resources/NCBI/gc.prt'

So what I did, as I did last time, was to download gc.prt manually using wget ftp.ncbi.nih.gov/entrez/misc/data/gc.prt , and the test ran fine afterwards, but a different error came back after I do actual run:
Process Process-185:
Traceback (most recent call last):
  File "/scratch/marcusleung/tools/miniconda3/envs/mantis_env/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/scratch/marcusleung/tools/miniconda3/envs/mantis_env/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "mantis/source/Multiprocessing.py", line 579, in worker_interpret_output
    self.generate_integrated_output(output_annotation_tsv, interpreted_annotation_tsv)
  File "mantis/source/Metadata.py", line 281, in generate_integrated_output
    output_annotation = self.read_and_interpret_output_annotation(output_annotation_tsv)
  File "mantis/source/Metadata.py", line 193, in read_and_interpret_output_annotation
    ref_file_links = self.get_hit_links(links_to_get[ref_file], ref_file)
  File "mantis/source/Metadata.py", line 112, in get_hit_links
    self.get_link_compiled_metadata(dict_hits=dict_hits, ref_file_path=self.mantis_paths['pfam'] + 'metadata.tsv')
  File "mantis/source/Metadata.py", line 45, in get_link_compiled_metadata
    hit_info=cursor.fetch_metadata(hit)
  File "mantis/source/Metadata_SQLITE_Connector.py", line 148, in fetch_metadata
    res=self.convert_sql_to_dict(res_fetch)
  File "mantis/source/Metadata_SQLITE_Connector.py", line 133, in convert_sql_to_dict
    sql_result=sql_result[1:]
TypeError: 'NoneType' object is not subscriptable

Could this be related to the fact that somehow gc.prt was not set up properly during database setup? Thanks
Marcus

Hello Marcus,
Thanks for notifying me on the gc.prt issue. I fixed the link now so it should download fine during setup.
Regarding your second issue, it seems you are having issues with your metadata SQL database. I can't reproduce this error locally so I added some new code to check the database for errors.
Could you please download the new mantis version and run check_sql ? Please report the results after and I will do my best to fix it.
Regards,
Pedro

Thanks Pedro as always! I am going to set up my database first then run check_sql to see what the matter is. Cheers :)
M

Hi Pedro
Just wondering where the check_sql command would be? I am trying to run that now. Thanks
Marcus

Hello Marcus,
You would run that command instead of running the other ones like run_mantis or setup_databases.

Hello Marcus,
I forgot to add the option to run into when executing mantis (my bad). Could you please redownload the source code and try to run check_sql as you would with run_mantis or setup_databases?
I dont know if you did in the past, but no need to run setup_databases again, just update the __main__.py file and the files in the source folder.
Regards,
Pedro

Hi Pedro. I am running the check_sql command now. It appears that some of the verbose messages were in yellow and some in white. For example:
Checking /my_path/tools/mantis/References/NCBI/28211/metadata.db (this was in yellow)
Checking /my_path/tools/mantis/References/NCBI/43080/metadata.db (this was in yellow)
Checking /my_path/tools/mantis/References/NOG/4447/metadata.db (this was in yellow)
Creating SQL database /my_path/tools/mantis/References/NOG/583/metadata.db (this was in white)
Creating SQL database /my_path/tools/mantis/References/NOG/84406/metadata.db (this was in white)
and so forth...
Does that mean that during the setup database step some of these were skipped, and now the check_sql command was able to detect any missing databases and is now adding them back ?
The command is still running as I type, so I will get back to you as to whether everything is running smoothly...Thanks
Marcus

Hello Marcus,
Yes, the yellow lines are the output from check_sql, the white lines correspond to the creation of these SQL databases.  Since these are only created when you use the corresponding reference, you are seeing them now as you ran check_sql.
Anyway, keep me posted.
Regards,
Pedro

Hello Marcus,
Could you please let me know if this issue was resolved?
Regards,
Pedro

Hi Pedro
For some reason my chcek_SQL command is still running after 5 days! It has been very slow but it's running so I am just going to leave it running for now. I shall let you know when it is done. Thanks
Marcus

Hello Marcus,
That is a bit strange, it has never taken me that long. Is the command still displaying "checking ****/metadata.db"? But I guess depending on your system it's possible, since the check_sql is quite comprehensive (i.e., checks all metadata entries).
I have recently added indexation to the SQL databases so it should be a lot faster to fetch information from it.
If you'd like you could try to redownload the source code, remove all metadata.db files in your reference databases and restart the check_sql command. This will also result in faster mantis runs further down the line so it might be worth doing now.
Sorry for the inconvenience, this new method for metadata retrieval should be way better than the past one (txt parsing), but I still had to iron out a few kinks on how to make it efficient. Hopefully now everything should be working as it intended.
Regards,
Pedro

Hey Pedro
Alright. I will try again and let you know.
M

ok that was indeed magnitudes faster :). I have already set up the databases, and my check_SQL command seemed to be alright. I shall do a trial run of my small sample now. Keep you updated!
M

Hi Pedro
I believe the output ran alright, with the following output files?
.
├── consensus_annotation.tsv
├── integrated_annotation.tsv
├── Mantis.out
└── output_annotation.tsv

Cheers
Marcus

Hey Marcus,
Glad to hear that. Yes, those are the output files. There's also a few other output files you could generate, but this depends on what kind of analysis you want to do. Anyway, if you are curious please check the wiki where I describe those and also the default output files.
Regards,
Pedro


TAJO-1320: HBaseStorageManager need to support Zookeeper Client Port.

Currently, HBaseStorageManager just use Zookeeper Quorum host names. But I think that users may want to change their Zookeeper client port. Thus, we need to support that users can set the port. For reference, HBase defines the port to ZOOKEEPER_CLIENT_PORT in HBaseConstants.

Hi @blrunner, your patch looks good.
I left some minor comments.
Please consider them.

Thanks for your comment.
Those changes look good to me.
+1!


configure CMake

Required Info

Camera Model
{ D400 }

Firmware Version
(Open RealSense Viewer --> Click info)

Operating System & Version
{Win (10) }

Kernel Version (Linux Only)
(e.g. 4.14.13)

Platform
PC/Raspberry Pi/ NVIDIA Jetson / etc..

SDK Version
{ legacy / 2.. }

Language
{C }

Segment
{Robot/Smartphone/VR/AR/others }

Issue Description
An error occurred when configuring CMake.As follows:
CMake Error: Cannot open file for write: C:/Program Files/Realsense/librealsense-master/CMakeCache.txt.tmpa9ecf
CMake Error at C:/Program Files/CMake/share/cmake-3.20/Modules/CMakeDetermineSystem.cmake:203 (file):
file failed to open for writing (No such file or directory):
C:/Program Files/Realsense/librealsense-master/CMakeFiles/CMakeOutput.log

Call Stack (most recent call first):
CMakeLists.txt:3 (project)
CMake Error: Could not open file for write in copy operation C:/Program Files/Realsense/librealsense-master/CMakeFiles/3.20.2/CMakeSystem.cmake.tmp
CMake Error: : System Error: No such file or directory
CMake Error at C:/Program Files/CMake/share/cmake-3.20/Modules/CMakeDetermineSystem.cmake:215 (configure_file):
configure_file Problem configuring file
Call Stack (most recent call first):
CMakeLists.txt:3 (project)
Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.18363.
CMake Error at CMakeLists.txt:3 (project):
Failed to make directory:
C:/Program Files/Realsense/librealsense-master/CMakeFiles/3.20.2

Configuring incomplete, errors occurred!
CMake Error: Cannot open file for write: C:/Program Files/Realsense/librealsense-master/CMakeCache.txt.tmpec537
CMake Error: : System Error: Permission denied
CMake Error: Unable to open cache file for save. C:/Program Files/Realsense/librealsense-master/CMakeCache.txt
CMake Error: : System Error: Permission denied
CMake Error: : System Error: Permission denied
CMake Error: Unable to open cache file for save. C:/Program Files/Realsense/librealsense-master/CMakeCache.txt
CMake Error: : System Error: Permission denied
Why does this problem occur? How to solve it？Thank you!
And I don’t know how to do the first step of Building from Source. Is there a detailed explanation?

Hi @rain0902   As you are using Windows, you may not need to build librealsense with CMake from source as there is an automated installer file for Windows.  It can be found in the Assets file list at the bottom of the information listing for each SDK version on the Releases page.
https://github.com/IntelRealSense/librealsense/releases

If you do need to build librealsense on Windows from source code using CMake though then the guide in the link below can be followed.
https://github.com/IntelRealSense/librealsense/blob/master/doc/installation_windows.md
You should be able to skip the section of the guide about enabling metadata, as you can do so in the RealSense Viewer tool instead after the SDK has been installed.
In regard to the CMake error that you wrote about if you still require an answer to that question after reading the links above then could you provide information please about the steps that you took before the error occurred?  Thanks!

Hello, first of all thank you for your reply. I‘m sorry, I didn’t make it clear in the previous letter.
I want to controll the RealSense Cameras with Matlab R2020b in Windows 10. This is the link：
But I don't know how to do the first step of building from Source, which is "Download the Git repository. Opt in the Matlab wrapper for LibRs with cmake .. -D... -DBUILD_MATLAB_BINDINGS:BOOL=ON when configuring CMake.".  Is there a detailed explanation?Please.Thank you very much!

Hello, first of all thank you for your reply. I‘m sorry, I didn’t make it clear in the previous letter.
I want to controll the RealSense Cameras with Matlab R2020b in Windows 10. This is the link:
https://github.com/IntelRealSense/librealsense/tree/master/wrappers/matlab#getting-started-with-realsense-sdk20-for-matlab
But I don't know how to do the first step of building from Source, which is "Download the Git repository. Opt in the Matlab wrapper for LibRs with cmake .. -D... -DBUILD_MATLAB_BINDINGS:BOOL=ON when configuring CMake.".  Is there a detailed explanation?Please.Thank you very much!

Hi @rain0902   You may find it easier to install the MATLAB wrapper as part of the automated RealSense SDK installer file's installation process.

Go to the SDK Releases page and scroll down the version listings to the bottom of the SDK version that you want to install.

https://github.com/IntelRealSense/librealsense/releases

At the bottom of each version listing is an Assets link with a triangle beside it. Left-click on the triangle to expand open a file list.  Left-click on the file link that begins with the name Intel.RealSense.SDK-WIN10 to begin downloading the large installer file for the SDK at the bottom of your browser window.

If you want to keep using a particular version and do not want to upgrade to a newer one then simply select the version that you were already using, as the installer will overwrite the previous installation.

When the download is complete, left-click on the download at the bottom of the window to launch the downloaded installer file and run the installation process.

Left-click the Next button on the installer window and on its next page, check that Matlab developer package has a tick in the box beside it to ensure that the MATLAB wrapper will be installed as part of the overall SDK installation process.

If there is a tick beside it then click Next to continue the installation process and keep simply clicking Next until it is shortly completed.

Once the SDK is installed on your Windows computer, navigate to the directory location below to find a MATLAB wrapper folder called +realsense

C: > Program Files (x86) > Intel RealSense SDK 2.0 > matlab

Add the +realsense folder to the MATLAB search path using the guidance in the link below.

https://github.com/IntelRealSense/librealsense/issues/2400
Alternatively, you can copy and paste this folder to a location on your computer where MATLAB can find it.

Hello, thank you very much. I have completed these six steps.
But I still don’t know how to do the intermediate steps before the guidance in https://github.com/IntelRealSense/librealsense/issues/2400
In the other words, I haven't finished building the wrapper from sources, and I don't have a directory called +realsense with mex and matlab wrapper.
I found the link https://github.com/IntelRealSense/librealsense/tree/master/wrappers/matlab#getting-started-with-realsense-sdk20-for-matlab
But I don’t know how to do the first step of building from Source, which is "Download the Git repository. Opt in the Matlab wrapper for LibRs with cmake .. -D... -DBUILD_MATLAB_BINDINGS:BOOL=ON when configuring CMake.". Is there a detailed explanation?Please.Thank you very much!

Hello, thank you very much. I have completed these six steps.
But I still don’t know how to do the intermediate steps before the guidance in https://github.com/IntelRealSense/librealsense/issues/2400
In the other words, I haven't finished building the wrapper from sources, and I don't have a directory called +realsense with mex and matlab wrapper.
I found the link https://github.com/IntelRealSense/librealsense/tree/master/wrappers/matlab#getting-started-with-realsense-sdk20-for-matlab
But I don’t know how to do the first step of building from Source, which is "Download the Git repository. Opt in the Matlab wrapper for LibRs with cmake .. -D... -DBUILD_MATLAB_BINDINGS:BOOL=ON when configuring CMake.". Is there a detailed explanation?Please.Thank you very much!

You do not need to build the MATLAB wrapper from source code if you have installed it with the automated SDK installer.  The Getting Started instructions offer the choice of installing with source code or the Windows installer.  You do not need to perform both.
Are you able to find the SDK's folder location on your computer by right-clicking on the RealSense Viewer launch shortcut icon on your Windows desktop and selecting the Open file location option from the menu, please?

Yes, I can find it, and it is in C:\Program Files (x86)\Intel RealSense SDK 2.0\tools

Thanks very much for the update!


Support for Traces/Transactions via Opentelemetry

This adds support for Traces via Opentelemetry integration, handled by our custom Sentry.Opentelemetry.SpanProcessor which accumulates spans and turns them into hierarchical transactions that are then sent to Sentry.
For this to work the following deps must be added to an app:
{:opentelemetry, "~> 1.4"},
{:opentelemetry_api, "~> 1.3"}

Furthermore, Opentelemetry needs to be configured with our SpanProcessor:
config :opentelemetry, span_processor: {Sentry.Opentelemetry.SpanProcessor, []}

The rest will automagically work.
Supported opentelemetry packages
This initial implementation supports the following Opentelemetry packages:

opentelemetry_phoenix
opentelemetry_bandit
opentelemetry_ecto

Considerations

Should we add new deps as sentry deps instead of having people add them manually to their mix.exs?
Would it make sense to try to pre-configure opentelemetry automatically?
Which other opentelemetry_* packages do we want to support initially?
Should we support top-level ecto events that occur when ie Ecto tries to create migrations table, or checks migrations? (I think we should ignore them, FWIW)

Screenshots

@solnic can you rebase this one now that #797 has been merged? Gonna make it easier to review I think 😄

@solnic can you rebase this one now that #797 has been merged? Gonna make it easier to review I think 😄

@whatyouhide done!

@whatyouhide are you adding support for the otel propagator map from #820 I am failing to see where it is being done

What's the status of this PR? We would love to be start using otel with Sentry.

@danschultzer we need to finish a little rework in #817 - after it's merged into this one, we need to go through another round of reviews, assuming we're happy with the implementation and testing goes well, we should be able to wrap it up and release later this month.

@whatyouhide seems like now span_processor_test is failing on 1.13 - I'm looking into this!

I've tested this in staging enviroment and seen two issues:

Oban is not handled in build_transaction:

  Handler "Elixir.OpentelemetryOban.PluginHandler.plugin_stop" has failed and has been detached. Class=:error
  Reason=:function_clause
  Stacktrace=[
     {Sentry.OpenTelemetry.SpanProcessor, :build_transaction,
      [
         %Sentry.OpenTelemetry.SpanRecord{
           trace_id: "450fcd62516cbff48d61066e2db6af37",
            span_id: "996b1cea0c7231f4",
             tracestate: {:tracestate, []},
              parent_span_id: nil,
               name: "Elixir.Oban.Stager process",
                kind: :internal,
                 start_time: "2024-12-10T05:03:40.820Z",
                  end_time: "2024-12-10T05:03:40.820Z",
                   attributes: %{"oban.plugin" => Oban.Stager},
                    events: {:events, 128, 128, :infinity, 0, []},
                     links: {:links, 128, 128, :infinity, 0, []},
                      status: :undefined,
                       trace_flags: 1,
                        is_recording: false,
                         instrumentation_scope: {:instrumentation_scope, "opentelemetry_oban",
                          "1.1.1", :undefined},
                           origin: "opentelemetry_oban"
                            },
                             []
                              ], [file: ~c"lib/sentry/opentelemetry/span_processor.ex", line: 57]},
                               {Sentry.OpenTelemetry.SpanProcessor, :on_end, 2,
                                [file: ~c"lib/sentry/opentelemetry/span_processor.ex", line: 29]},
                                 {:otel_tracer_server, :"-on_end/1-fun-0-", 3,
                                  [file: ~c"/app/deps/opentelemetry/src/otel_tracer_server.erl", line: 195]},
                                   {:lists, :foldl, 3, [file: ~c"lists.erl", line: 2146]},
                                    {:otel_span, :end_span, 1, [file: ~c"src/otel_span.erl", line: 294]},
                                     {:otel_telemetry, :end_telemetry_span, 2,
                                      [file: ~c"src/otel_telemetry.erl", line: 52]},
                                       {:telemetry, :"-execute/3-fun-0-", 4,
                                        [file: ~c"/app/deps/telemetry/src/telemetry.erl", line: 167]},
                                         {:lists, :foreach_1, 2, [file: ~c"lists.erl", line: 2310]}
                                         ]

get_data_category fails as Sentry.Transaction is not handled:

GenServer {Sentry.Transport.SenderRegistry, 7} terminating
** (FunctionClauseError) no function clause matching in Sentry.Envelope.get_data_category/1
 (sentry 10.8.0) lib/sentry/envelope.ex:68: Sentry.Envelope.get_data_category(%Sentry.Transaction{event_id: "c17ec8f1f9094d58b52e28332ba9c68a", start_timestamp: "2024-12-10T05:16:11.079Z", timestamp: "2024-12-10T05:16:11.082Z", transaction: "app.repo.query", transaction_info: %{source: "component"}, status: nil, contexts: %{trace: %{data: %{"db.instance" => nil, "db.name" => nil, "db.system" => :postgresql, "db.type" => :sql, "db.url" => "ecto:", "decode_time_microseconds" => 3, "idle_time_microseconds" => 1250815, "query_time_microseconds" => 2032, "queue_time_microseconds" => 1014, "source" => nil, "total_time_microseconds" => 3051}, origin: "opentelemetry_ecto", description: "app.repo.query", op: "db.sql.ecto", span_id: "87d6d9b1c61ced34", trace_id: "119004179fa20ee3af70641c1e331508", parent_span_id: nil}}, request: nil, measurements: %{}, spans: [], type: "transaction"})
  (sentry 10.8.0) lib/sentry/client_report/sender.ex:36: anonymous fn/3 in Sentry.ClientReport.Sender.record_discarded_events/3
   (elixir 1.17.3) lib/enum.ex:987: Enum."-each/2-lists^foreach/1-0-"/2
    (sentry 10.8.0) lib/sentry/transport.ex:69: Sentry.Transport.post_envelope_with_retries/6
     (sentry 10.8.0) lib/sentry/transport.ex:32: Sentry.Transport.encode_and_post_envelope/3
      (sentry 10.8.0) lib/sentry/transport/sender.ex:67: Sentry.Transport.Sender.handle_cast/2
       (stdlib 6.1) gen_server.erl:2371: :gen_server.try_handle_cast/3
        (stdlib 6.1) gen_server.erl:2433: :gen_server.handle_msg/6
        Process Label: Sentry.Transport.Sender
        Last message: {:"$gen_cast", {:send, Sentry.HTTPClient.app, %Sentry.Transaction{event_id: "c17ec8f1f9094d58b52e28332ba9c68a", start_timestamp: "2024-12-10T05:16:11.079Z", timestamp: "2024-12-10T05:16:11.082Z", transaction: "app.repo.query", transaction_info: %{source: "component"}, status: nil, contexts: %{trace: %{data: %{"db.instance" => nil, "db.name" => nil, "db.system" => :postgresql, "db.type" => :sql, "db.url" => "ecto:", "decode_time_microseconds" => 3, "idle_time_microseconds" => 1250815, "query_time_microseconds" => 2032, "queue_time_microseconds" => 1014, "source" => nil, "total_time_microseconds" => 3051}, origin: "opentelemetry_ecto", description: "app.repo.query", op: "db.sql.ecto", span_id: "87d6d9b1c61ced34", trace_id: "119004179fa20ee3af70641c1e331508", parent_span_id: nil}}, request: nil, measurements: %{}, spans: [], type: "transaction"}}}

Testing this, I see that the sentry environment is not set for the traces.
Also it looks like all other SDK's makes the OTel integration be a separate package, should it be the same for the elixir sdk? At least all the otel modules needs to be wrapped in a ensure_loaded conditional as the otel deps are optional, but I think it does make sense that the sentry otel bridge its own dep?

I've tested this in a production enviroment and seen two issues:

@danschultzer thanks for testing it out so early, this is very helpful! I fixed both issues, but I consider Oban support still a WIP.

Also it looks like all other SDK's makes the OTel integration be a separate package, should it be the same for the elixir sdk? At least all the otel modules needs to be wrapped in a ensure_loaded conditional as the otel deps are optional, but I think it does make sense that the sentry otel bridge is a separate package?

Yes I think that would make sense. Thoughts @whatyouhide?

Sure, I've been testing this a lot more and I got several suggestions to it. I've forked your work @solnic and fixed some of the issues I had there along with things I think could be refactored. Let me push a draft PR so you can check it out, just need to clean it up a bit. Among other things, I think there's some considerations to be made on how much we want to manually normalize the attributes vs just deferring to otel.

@danschultzer I made it pull opentelemetry_oban from your fork for the time being, and made some adjustments. See recent commits:

844313d04098e7dd1186b7226243c4e908e01295 - this makes Oban processing traces/spans follow Sentry's/OTel conventions more closely
65f6b116d340d7c368285dd8b0f34eb67801a8f2 - this makes it skip Oban Stager traces as they produce a lot of noise. We may want to make that configurable though

@whatyouhide could you take another look over here?

@solnic @danschultzer can you folks let me know when this will be Ready to Review™? I havent been paying attention cause the rate of changes is too high but I think I would take a look before we take action on merging this.

@whatyouhide it's in a flux still as it's waiting for some upstream improvements in opentelemetry packages but I would really appreciate if you could review our updated approach to span storage. We still use genserver but now it uses ets, and there's a sweeping mechanism in place. This part could really use a review and I could address your feedback while we're waiting for other things.

I think a better approach here would be to split this PR into the data layer and then the OTel implementation layer. The data layer should be easy and "non controversial": Span struct, send_transaction, and whatnot. Thoughts?

@whatyouhide yes I agree, I'll split it up.
There's also a suggestion (somewhere in the comments here 😅) to actually separate otel stuff in a sentry-opentelemetry package. Would that make sense from the maintenance pov? We could keep it here but I'm not sure how that would affect our release pipeline (I suspect it should not be hard to configure though given that other SDKs use monorepo for multiple packages too).

I’m not sure we need sentry-opentelemetry if we make the opentelemetry_* deps optional. If we did, I'd rather put it in a separate repo.

I’m not sure we need sentry-opentelemetry if we make the opentelemetry_* deps optional. If we did, I'd rather put it in a separate repo.

FWIW all other Sentry SDK's are monorepos with individual libraries for instrumentation:
https://github.com/getsentry/sentry-ruby
https://github.com/getsentry/sentry-rust
https://github.com/getsentry/sentry-go
https://github.com/getsentry/sentry-javascript/tree/develop/packages

I've split this into #842 and #843 /cc @whatyouhide


fix SamplerBindingMap for non postprocess materials

This fixes #935
SamplerBindingMap was always adding the post-process samplers to the
map -- but matc doesn't generate those. This led the vulkan backend
to believe there was a post-process sampler bound, when there wasn't.
This is also technically a bug in the Vulkan backend because it should
have used the SamplerInterfaceBlock to determine which shaders are
actually used instead of blindly going through the whole list and
trusting SamplerBindingMap.
The fix is kind of a hack, but the whole SamplerBindingMap is kind of a
hack anyways.

it should have used the SamplerInterfaceBlock to determine which shaders are
actually used

To use the SamplerInterfaceBlock to determine which samplers are actually used by a given shader, the general procedure is to ask it for the string name of each sampler, then ask the program "hey are you using a sampler with this name?" ...  which Vulkan cannot do.

We believe there is a confusion. In GL, layout(binding=3) uniform sampler2D foo means that foo will always use texture unit 3. That's different from giving it a location (which is usually queried by name). We need to understand how location and binding are used in Vulkan. It might be different than in GL.


Scroll position reset where collapsible header normal

How to prevent reset when collapsible header in normal?

Hi @umardevid,
Thanks for your comment.
Sorry, I am not fully understand your question. Could you explain more?


Create Website with API Documentation from KDoc

It would be really great to have the website with a search ability from all documentation we already have in KDoc. Here is the link to get started: https://kotlin.github.io/dokka/1.4.32/user_guide/gradle/usage/

Great, thanks for the advice, we started Dokka integration, but it was not finished yet.

The documentation for the 0.2 version was added here https://jetbrains.github.io/KotlinDL/index.html


Add jni interface to use a binary hnsw index with faiss

Description
Add jni interface to use a binary hnsw index with faiss
SearchParameter is not supported with binary hnsw index in faiss library yet. Therefore, multi vector, pre filtering, and ef-search parameter passing won't work with binary index. Support of SearchParameter in binary index with faiss will be handled either by a PR in faiss repo or faiss custom patch in a separate PR.
Issues Resolved
N/A
Check List

[X] New functionality includes testing.

[X] All tests pass

[X] New functionality has been documented.

[X] New functionality has javadoc added

[X] Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@heemin32 lets fix the GH actions too.

@heemin32 lets fix the GH actions too.

Will do. So far, the error is coming from https://github.com/opensearch-project/k-NN/pull/1747/files#diff-892c5ae9158d35b9f5fa9b67d53d7373ca6d2bf04d1052add99c329d03645409R110 as index description is not passed to the method.

Refactored and added unit tests for createIndex method. For queryIndex, it needs another big change as current code shape is not test friendly. Hope, we can improve it incrementally outside of this PR.

Valgrind result
~/k-NN/jni$ valgrind --leak-check=yes ./bin/jni_test
==54535== Memcheck, a memory error detector
==54535== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==54535== Using Valgrind-3.22.0 and LibVEX; rerun with -h for copyright info
==54535== Command: ./bin/jni_test
==54535==
vex amd64->IR: unhandled instruction bytes: 0x62 0xF2 0xFD 0x8 0x7C 0xC0 0xC5 0xF9 0x7F 0x5
vex amd64->IR:   REX=0 REX.W=0 REX.R=0 REX.X=0 REX.B=0
vex amd64->IR:   VEX=0 VEX.L=0 VEX.nVVVV=0x0 ESC=NONE
vex amd64->IR:   PFX.66=0 PFX.F2=0 PFX.F3=0
==54535== valgrind: Unrecognised instruction at address 0x6a15c9e.
==54535==    at 0x6A15C9E: _GLOBAL__sub_I_distcomp_sparse_scalar_fast.cc (in /home/ubuntu/k-NN/jni/release/libopensearchknn_nmslib.so)
==54535==    by 0x400571E: call_init.part.0 (dl-init.c:74)
==54535==    by 0x4005823: call_init (dl-init.c:120)
==54535==    by 0x4005823: _dl_init (dl-init.c:121)
==54535==    by 0x401F59F: ??? (in /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2)
==54535== Your program just tried to execute an instruction that Valgrind
==54535== did not recognise.  There are two possible reasons for this.
==54535== 1. Your program has a bug and erroneously jumped to a non-code
==54535==    location.  If you are running Memcheck and you just saw a
==54535==    warning about a bad jump, it's probably your program's fault.
==54535== 2. The instruction is legitimate but Valgrind doesn't handle it,
==54535==    i.e. it's Valgrind's fault.  If you think this is the case or
==54535==    you are not sure, please let us know and we'll try to fix it.
==54535== Either way, Valgrind will now raise a SIGILL signal which will
==54535== probably kill your program.
==54535==
==54535== Process terminating with default action of signal 4 (SIGILL)
==54535==  Illegal opcode at address 0x6A15C9E
==54535==    at 0x6A15C9E: _GLOBAL__sub_I_distcomp_sparse_scalar_fast.cc (in /home/ubuntu/k-NN/jni/release/libopensearchknn_nmslib.so)
==54535==    by 0x400571E: call_init.part.0 (dl-init.c:74)
==54535==    by 0x4005823: call_init (dl-init.c:120)
==54535==    by 0x4005823: _dl_init (dl-init.c:121)
==54535==    by 0x401F59F: ??? (in /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2)
==54535==
==54535== HEAP SUMMARY:
==54535==     in use at exit: 83,256 bytes in 40 blocks
==54535==   total heap usage: 40 allocs, 0 frees, 83,256 bytes allocated
==54535==
==54535== LEAK SUMMARY:
==54535==    definitely lost: 0 bytes in 0 blocks
==54535==    indirectly lost: 0 bytes in 0 blocks
==54535==      possibly lost: 0 bytes in 0 blocks
==54535==    still reachable: 83,256 bytes in 40 blocks
==54535==         suppressed: 0 bytes in 0 blocks
==54535== Reachable blocks (those to which a pointer was found) are not shown.
==54535== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==54535==
==54535== For lists of detected and suppressed errors, rerun with: -s
==54535== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Illegal instruction (core dumped)


Update filters-helpers, play-ahc-ws, ... to 2.8.9

Updates

com.typesafe.play:filters-helpers
com.typesafe.play:play-ahc-ws
com.typesafe.play:play-akka-http-server
com.typesafe.play:play-docs
com.typesafe.play:play-guice
com.typesafe.play:play-jdbc
com.typesafe.play:play-jdbc-evolutions
com.typesafe.play:play-logback
com.typesafe.play:play-server
com.typesafe.play:play-test
com.typesafe.play:sbt-plugin

from 2.8.8 to 2.8.9.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.typesafe.play" } ]

labels: library-update, early-semver-patch, semver-spec-patch

Superseded by #78.


fix: circular dependencies rebuild panic

Related issue (if exists)

Summary

🤖 Generated by Copilot at 637cc9c
Improve HMR performance with parallelism and hash map optimization. Use rayon to parallelize module processing and only hash and chunk modules in the crates/rspack_core/src/compiler/hmr.rs file.

Walkthrough

🤖 Generated by Copilot at 637cc9c

Improve HMR performance by using parallel iterators from rayon crate (link)
Refactor module hash and chunk id map creation to iterate over chunk graph modules instead of module graph modules, avoiding unnecessary computations and lookups (link)

Can you add a test for it?

Can you add a test for it?
Look like already has test(packages/rspack/tests/hotCases/runtime/circular), but why not throw error..

packages/rspack/tests/hotCases/runtime/circular does not cover this case, I have added a new test case


JSDoc: inferred type from class instance does not respect optional constructor fields

🔎 Search Terms
Class, constructor, jsdoc, inferred, types
🕗 Version & Regression Information

This is the behavior in every version I tried, and I reviewed the FAQ for entries about derived class property initializers.

⏯ Playground Link
https://www.typescriptlang.org/play?ts=5.3.3&filetype=js#code/MYGwhgzhAECCB2BLAtmE0DeAoavoHoAqQnPaQ6AAQAcwAnMZTDaeRgUwH4AuaCAFzqJ4Ac2gBfcdABmidiAAmEUnkL4V0YAHt4AugFdg-LXQAUs+UoCUmDWX4ALRBAB0bZO2gBeGXMWv3dg1xLBCsbV1+aBBEHW9WdgB3OCRUEFMWQN4AcgAhLQAjbPErcJ0BaGpweCifeCSUlDQMkqwsIgoaekZMPWERKUDydWl9eCNY+GgRdn4AOQ5TQJtsMgiILRB2FxAtESWOUrCZ+cWYnTdDrBOFj1MqsBrLjxt8fD4HLX1FaEc6LWS-AAntRPOw6P86EA
💻 Code
class Animal {
    /**
     * @param {{ name?: string }} fields
     */
    constructor(fields) {
        this.name = fields.name
    }
}

const lion = new Animal({ name: 'Bob'})
const plant = new Animal({})

/** @param {string} name */
function getName(name) {
    console.log(name)
}

getName(lion.name)
getName(plant.name) // should throw type error

🙁 Actual behavior

My class instance const plant = new Animal({}) should have its plant.name property type inferred as undefined since it wasn't instantiated.
When passing plant.name as a parameter to my getName() function, it should expect its first parameter typed as string, but doesn't throw an error

🙂 Expected behavior
Class instances should have their public field types inferred from the constructor:
const lion = new Animal({ name: 'Bob'}) // `lion.name` should be type `string`
const plant = new Animal({}) // `plant.name` should be type `undefined`

Additional information about the issue
No response

This is working as intended.

My class instance const plant = new Animal({}) should have its plant.name property type inferred as undefined since it wasn't instantiated.

Types are based on the available type information, not runtime values. The type information available is this part: { name?: string } - so it's always an optional string. If you want to have different types depending on the type of the argument, then you need to use a generic.

When passing plant.name as a parameter to my getName() function, it should expect its first parameter typed as string, but doesn't throw an error

Because you have strictNullChecks disabled. Without this flag the type undefined is basically not existing. The type of name is string in this case.

Thanks for explaining. Moving forward I'll either use generics or type assertions where appropriate.


Escape slashes before config file generation

Hello.
I've found out that with RELX_REPLACE_OS_VARS=true exrm generates invalid runtime configuration files if ENV variables contain slashes.
In our case we've experienced application crash with error
could not start kernel pid (application_controller) (error in config file "/app/rel/application/running-config/sys.config.2.config" (none): no ending <dot> found)

because one of envirionment variables ended with slash.

Thanks!


[Question] New Relic slows down page load

Hi, sorry for creating an issue here, but I am also been playing around with rails 7 and new relic. Got the integration working but it seems like it slows down the page speed. I am not sure if this is caused by the New Relic browser agent or not, will love to hear from you if you faced similar problem in your setup.
I have generated a new Rails 7 app with tailwindcss and added Devise to it. After experimenting it on Heroku (free tier), it seems that with and without the New Relic ruby agent gem install show quite some significant different in page speed. Here are some examples:
With New Relic Gem Installed

Calling Devise Session#create

This is the Timings for the GET sign_in request after submitting the form with wrong Email and Password pair.

It feels like something is blocking the DOM to get painted

Without New Relic Gem Installed

Calling Devise Session#create

Timings

PS: Again, sorry for creating an issue in your demo repo, feel free to close it if you want. Thanks in advance and have a great day

Hey @marcushwz, thanks for raising this! I don't see the same lag when I deploy this sample app with or without on Render, but I can replicate it on the Heroku free tier. It might be something specific with what resources are allocated on Heroku at that level? In any case, I'm investigating internally to see if anyone has any advice. You may also want to duplicate this question in the GitHub repo for the gem itself at https://github.com/newrelic/newrelic-ruby-agent


[NFC] GenericSignatureImpl: Spell conformsToProtocol & getConformsTo in terms of requirements

Make it clear that only abstract conformances are of interest to these methods.

Huh, does git always mark adjacent changes as conflicting?

@swift-ci please smoke test and merge

You'll need to give this one more rebase after the const-ification patches.

@swift-ci please smoke test

@swift-ci please smoke test Linux

@swift-ci please clean smoke test OS X


Vr chat not on the software?

I open the software on my vr with the instructions to click on vr chat but vr chat is not there? Please help

What do you mean with VRChat is not there


Support for ChatGPT API?

Any plans to update this plugin to use the new official ChatGPT APIs?

@acneupane  it is using the chatgpt api since the release 😇

10x faster & cool UX :)


Update version

I think you should also update home_assistant/custom_components/manifest.json to match this version.


Allow creating indexes with PostgreSQL options (asc/desc, nulls first/last)

Is it possible to create descending index using EF core fluent api and Npgsql EF Core?
(e.g. like this here: https://www.postgresql.org/docs/9.6/static/indexes-ordering.html)

Not at the moment, the workaround is to create your index using raw SQL in your migrations.
This should be quite easy to implement will see about getting it in for 2.1.0. If everyone wants to take a stab, this is a good issue.


XamlAnimatedGif Memory leak in wp

there is Memory leak for arm build, if i use this code in pivotitem one and build for arm. then select pivotitem two there is memory leak .
in my test when  i select pivotitem two and StopAnimation() func use code is ok

Hi @FgoDt! Sorry for the delay. If possible, could you please provide a small project that demonstrates this issue?

Hi @JasterAtMicrosoft  project is here https://onedrive.live.com/redir?resid=2E6A88BF23516C0C!111&authkey=!AGvGuYL5pdjJ0z4&ithint=file%2Czip just buid for ARM there was memory leak

Thanks! I can repro the memory leak and have a theory about what is going on. I'll respond to this thread with further updates.

Fixed in v1.0.21


Use a own character class instead of the pure response objects

Currently the whole code works with the pure response objects. Every time something like p.json()['key1']['key2'] is done...
Make the code more object orientated and introduce a class Character.
This class takes the pull response from raider.io and creates a valid Character object which holds every necessary information about this player character.
e.g.:
class Character:
    def __init__(self, rio_response):
        self._iLvl = rio_response.json()['ilvl']
        self._rio_score = rio_response.json()['score']
        ...

You're right... Thats a lot better than the actual code. But my time to this project is very little. Feel free if u want to do that much time into it. If u need any help at any point, just tell me :)

I implemented a class Player which holds all relevant player/character information.
There is still some work to do because at some points there are still some player._data[...] calls.
In the future direct access to the self._data field should be prevented.


Changed body_test_motion to discard by angle rather than depth

Fixes #626.
Fixes #643.
This applies the change to PhysicsServer3D.body_test_motion that was suggested by @yosoyfreeman in #812, which effectively reverts the hack introduced by #559 and replaces its depth-based contact discard with an angle-based one instead.
Note that this also impacts PhysicsBody3D.move_and_collide as well as PhysicsBody3D.test_move and of course CharacterBody3D.move_and_slide.
This will possibly lead to a regression in ghost collisions for some scenarios, like the one seen in #529, but the downsides of #559 have proved to be too great to keep it around.

Thank you a lot for your work Mihe!

It turned out that this did not in fact fix #626, since apply_floor_snap still runs when walking across a perfectly parallel floor, which I probably should have caught before pushing this. It's not really an argument for reverting this though, since it was arguably worse before, but it's still somewhat unfortunate.
However, I am also slightly worried over the fact that this change results in zero slide collisions when a character is walking over a perfectly parallel floor like that. That's obviously to be expected given that this change discards any such planes, but I've seen a couple of examples of cases where people rely on these slide collisions to gather information about the surface they're walking on, which will now be more difficult.
It would have been nice if move_and_slide stored the wall/floor contacts explicitly rather than force people to rely on the slide collisions.

Hi Mihe!

It turned out that this did not in fact fix https://github.com/godot-jolt/godot-jolt/issues/626, since apply_floor_snap still runs when walking across a perfectly parallel floor, which I probably should have caught before pushing this. It's not really an argument for reverting this though, since it was arguably worse before, but it's still somewhat unfortunate.

It is unfortunate indeed, but to be expected. Move and slide is not capable of knowing is touching a floor unless you directly collide against one. This means that, even if it works perfectly, a call to apply_floor_snap will be done, as is the only way the system has to retrieve floor info. Its the same problem that appears while moving parallel to a wall, it will report nothing.

However, I am slightly worried over the fact that this change results in zero slide collisions when a character is walking over a perfectly parallel floor like that. That's obviously to be expected given that this change discards any such planes, and I would probably argue that such a movement shouldn't result in any slide collisions either, but I've seen a couple of examples where people rely on these slide collisions to gather information about the surface they're walking on, which will now be more difficult.

To fix this issue with walls, once i touched one i add a minimal "force" or velocity towards it. This is enough to make the character collide every frame with it without altering user input too much, so i can gather the data without skips.
Could this work in this scenarios? I know is a little of a hack, but seems less of a hack than what move and slide was actually doing. Maybe a small gravity when on floor would be enough for this cases.

It would have been nice if move_and_slide stored the wall/floor contacts explicitly, rather than force people to rely on the slide collisions (where they typically check the normals themselves), in which case apply_floor_snap could have stored its collision results there instead of them being discarded completely.

The problem is that move and slide does not have any way of knowing nothing that is not the first contact on the direction of the movement or apply floor snap. What is reported by is on wall or is on floor is basically what is reported in one of the slide collisions as the last floor or wall hit. They are just a quicker way of looking for it. I learning my way to c++, so i could start trying to implement some hacks... But they would be that, hacks.  Because of course we would want to rewrite it as soon as we would have better sweeping, normal data etc.

It is unfortunate indeed, but to be expected. Move and slide is not capable of knowing is touching a floor unless you directly collide against one. This means that, even if it works perfectly, a call to apply_floor_snap will be done, as is the only way the system has to retrieve floor info. Its the same problem that appears while moving parallel to a wall, it will report nothing.

This is however not how Godot Physics behaves, which seemingly never runs apply_floor_snap when just moving along a level surface like that, which bothers me quite a bit. I've spent a lot of time comparing the two implementations trying to figure out why that is, but have not been able to track down the exact cause.
I've always assumed it has something to do with the type of minimum contact depths that Godot Physics uses, where it will deliberately undershoot the recovery step, but if I try to replicate the same thing on my end I get a prohibitive amount of ghost collisions, to the point where it's almost impossible to walk up an inclined surface, hence why I took the opposite approach in #559 where I undershoot the final collision check step instead.

Could this work in this scenarios? I know is a little of a hack, but seems less of a hack than what move and slide was actually doing. Maybe a small gravity when on floor would be enough for this cases.

 In fact, since it powers move_and_collide as well it doesn't even necessarily have to involve a CharacterBody3D.

The problem is that move and slide does not have any way of knowing nothing that is not the first contact on the direction of the movement or apply floor snap. What is reported by is on wall or is on floor is basically what is reported in one of the slide collisions as the last floor or wall hit.

Yes, I know, but what I'm saying is that people currently rely on slide collisions in a way where they iterate through them all, check the normal of them, and then decide based on the angle of that normal whether that collision is a wall or a floor, and then use those accordingly. This is obviously a bit awkward/redundant since move_and_slide has already done the work (in _set_collision_direction) to decide which slide collisions map to what surface.
This is made worse by the fact that apply_floor_snap does not actually store its collision results as additional slide collisions, which is why we end up with zero slide collisions when walking along a level surface in Godot Jolt.
But if move_and_slide actually stored its collisions as floor_collisions and wall_collisions or something along those lines, then there's presumably no real reason why apply_floor_snap couldn't just add its collision results to floor_collisions and people could rely on those instead of inferring it from the slide collisions.
That obviously does nothing to fix the issue of wall collisions, but it would be an improvement at least.

Hi Mihe, i got some time free today and checked the code with what you told me in mind.
I agree that adding floor snap results as slide collision can be a solution. I could try to do that (Still learning c++ and godot codebase). However, i do think the reason they are not stored is because they are technically not slide collisions, in the sense that they do not trigger the sliding behavior or alter the velocity in any way. However, the responsibilities of checking the environment and reacting to it are already mixed. Since no breaking changes can be done by now, this would allow to at least provide more information to the user. It does not require additional parameters or API either, unlike adding a new floor, wall and ceiling categories. As i see it, calling a floor collision an slide collision is a minor hack compared to some of the other ones that are already there.
If you think this could improve things, i can look into it and try to do a PR. As i said, is probably the only one with some chance to get accepted right now and no breaking changes are required.

As i said, is probably the only one with some chance to get accepted right now and no breaking changes are required.

What do you mean by "get accepted right now"? As in Godot 4.3-beta1 or Godot 4 in general?
I would perhaps argue that adding more collisions to the list of slide collisions (that may or may not belong there) is an actual breaking change (albeit a small one) whereas adding new categories is not. I have a hard time imagining either one making it into Godot 4.3 though, given that we've entered the beta phase already, and Godot Physics doesn't seem to need the change.
If you're just talking about Godot 4 in general, I wouldn't shy away from adding to the API necessarily. It seems to me like adding those new lists/categories would be fairly simple to provide a reason for, seeing as how users work around it by checking the normal when using either physics engine.
I am however realizing now that even if these separate categories were to be added, we would have a hard time categorizing get_last_slide_collision(), since that particular collision (which is probably important to some people) may or may not be in any of these new lists. Adding some kind of is_floor() and is_wall() to KinematicCollision3D would have been a nice option instead, but unfortunately it's shared with move_and_collide, which has no notion of walls and floors, so that doesn't seem right.


Support multiple static asset dirs by allowing string arrays for config.kit.files.assets

Describe the problem
I'd like to get static assets from multiple locations. In this example the SvelteKit site is inside the site/ directory as part of a bigger project and I'd like to use static assets from assets/ and site/static/.
.
├── assets
│   ├── ...
└── site
    ├── src
    ├── static
    └── ...

Describe the proposed solution
config.kit.files.assets could support arrays:
// svelte.config.jg

export default {
  kit: {
    adapter: adapter(),

    files: {
      assets:[ `../assets`, `./static` ],
    },
  },
}

This currently raises

config.kit.files.assets should be a string, if specified

Alternatives considered
Create an alias of assets/ in site/static/. It works but feels hacky.
Importance
nice to have
Additional Information
No response

Would love to see this. I'm trying to exclude the static/img folder from the { files } in my service worker, because of the size.
I now do it with a workaround, but feels not very natural. Is there another solution for this problem?
Workaround: const cachedFiles = files.filter(file => !file.startsWith('/img');

Would also love to see this. It looks like vite only supports one public folder, so this might be a vite limitation?


Fail to create CRD on start

Hi,
As is, the operator can't create CRDs if not given some permissions.
Applying this makes it work (assuming you deploy on the default namespace) :
Maybe some extra documentation on the README.md file to tell permissions are needed would be nice ;-)
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: operator-kafka
rules:
- apiGroups:
  - krallistic.github.com
  resources:
  - "*"
  verbs:
  - "*"
- apiGroups:
  - ""
  resources:
  - pods
  - services
  - endpoints
  - persistentvolumeclaims
  - events
  - configmaps
  - secrets
  verbs:
  - "*"
- apiGroups:
  - apps
  resources:
  - deployments
  - daemonsets
  - replicasets
  - statefulsets
  verbs:
  - "*"
- apiGroups:
  - apiextensions.k8s.io
  resources:
  - customresourcedefinitions
  verbs:
  - '*'

---

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: default-account-operator-kafka
subjects:
- kind: ServiceAccount
  name: default
  namespace: default
roleRef:
  kind: ClusterRole
  name: operator-kafka
  apiGroup: rbac.authorization.k8s.io

Can you push it to this project!


Problem regarding Omni Explorer Blockchain

Hello Omni,
I transfer my usdt from my binance account to send it to my coins.ph wallet, but until now the status keep on confirming. Please solved this issue or return it back to my binance account.

Amount
249.60927651

Property
TetherUS (#31)

Sender
1FoWyxwPXuj4C6abqwhjDWdz6D4PZgYRjA

Recipient
37kHgDKawcBR7EcuwMspFbRHL9UMXLXgsU

Date/Time
6/8/2018 6:02:35 AM

In Block
526506

Status
84 CONFIRMATIONS

Bitcoin Fees
0.00002629 BTC

Omni Fees
0.00 OMNI

Type/Version
Type 0, Version 0

Raw Data
Click here for raw transaction...

How to transfer I don’t pls telll him I work for you pls tell whole method
explain my lost money Binance to coinbase $3500 lost money pls and i
working you also..
On Fri, Jun 8, 2018 at 10:53 AM Dhandee<EMAIL_ADDRESS>wrote:

Hello Omni,
I transfer my usdt from my binance account to send it to my coins.ph
wallet, but until now the status keep on confirming. Please solved this
issue or return it back to my binance account.
Amount<PHONE_NUMBER>1
Property TetherUS (#31 https://github.com/OmniLayer/omnicore/issues/31)
Sender 1FoWyxwPXuj4C6abqwhjDWdz6D4PZgYRjA
Recipient 37kHgDKawcBR7EcuwMspFbRHL9UMXLXgsU
Date/Time 6/8/2018 6:02:35 AM
In Block 526506
Status 84 CONFIRMATIONS
Bitcoin Fees 0.00002629 BTC
Omni Fees 0.00 OMNI
Type/Version Type 0, Version 0
Raw Data Click here for raw transaction...
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/OmniLayer/omnicore/issues/641, or mute the thread
https://github.com/notifications/unsubscribe-auth/AJ5-Q_L6YUvzkE0tqISab1-Ipo064_Ezks5t6pADgaJpZM4UgdAO
.

--
null

Hi there, please see:
https://github.com/OmniLayer/omnicore/wiki/I-sent-tokens-to-an-incompatible-wallet-such-as-Coinbase-or-GDAX.-Can-you-reverse-the-transaction%3F

I’m in this same position. I’m curious. If coinbase essentially received the money, is it just there in code that cannot be retained and they will not help you reverse it or do they actually receive it and keep it. Did they take the money?


Distributed modelling using HBVEC fails on PAVICS notebooks

Description
Using notebook "/tutorial-notebooks/hydro/Distributed_hydrological_modelling.ipynb" works flawlessly with the default GR4JCN model setup. Changing to HBVEC (and changing the parameter set to reflect an HBVEC model) fails due to lake HRUs that are not accepted in HBVEC.
What I Did

Changed the model instantiation from GR4JCN to HBVEC;
Changed the parameter set to an HBVEC one.

The resulting block of code within the cell (cell 6) looks like this now:
model_config = HBVEC(
    params=[0.059,4.07,2.001,0.034,0.099,0.5,3.4,38.3,0.46,0.06,2.27,4.8,0.583,0.045,0.87,18.94,2.03,0.44,0.67,1.14,1.02],
    #params=[0.529, -3.396, 407.29, 1.072, 16.9, 0.947],
    StartDate=dt.datetime(1998, 1, 1),
    EndDate=dt.datetime(2020, 12, 31),
    ObservationData=[qobs],
    Gauge=meteo_forcing_stations,
    GlobalParameter={"AVG_ANNUAL_RUNOFF": 40.65},
    **rvh,
)

This fails with the following traceback:
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[10], line 2
      1 # Prepare the model configuration
----> 2 model_config = HBVEC(
      3     params=[0.059,4.07,2.001,0.034,0.099,0.5,3.4,38.3,0.46,0.06,2.27,4.8,0.583,0.045,0.87,18.94,2.03,0.44,0.67,1.14,1.02],
      4     #params=[0.529, -3.396, 407.29, 1.072, 16.9, 0.947],
      5     StartDate=dt.datetime(1998, 1, 1),
      6     EndDate=dt.datetime(2020, 12, 31),
      7     ObservationData=[qobs],
      8     Gauge=meteo_forcing_stations,
      9     #GlobalParameter={"AVG_ANNUAL_RUNOFF": 40.65},
     10     **rvh,
     11 )
     13 # Run the model with the configuration we just built
     14 distributed_outputs = Emulator(model_config).run(overwrite=True)

File /opt/conda/envs/birdy/lib/python3.9/site-packages/ravenpy/config/emulators/hbvec.py:276, in HBVEC.__init__(self, **data)
    275 def __init__(self, **data):
--> 276     super().__init__(**data)
    278     if self.gauge:
    279         for gauge in self.gauge:

File /opt/conda/envs/birdy/lib/python3.9/site-packages/pydantic/main.py:341, in pydantic.main.BaseModel.__init__()

ValidationError: 11 validation errors for HBVEC
HRUs -> __root__ -> 45 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))
HRUs -> __root__ -> 65 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))
HRUs -> __root__ -> 66 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))
HRUs -> __root__ -> 68 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))
HRUs -> __root__ -> 84 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))
HRUs -> __root__ -> 85 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))
HRUs -> __root__ -> 86 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))
HRUs -> __root__ -> 96 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))
HRUs -> __root__ -> 146 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))
HRUs -> __root__ -> 152 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))
HRUs -> __root__ -> 164 -> hru_type
  unexpected value; permitted: 'land' (type=value_error.const; given=lake; permitted=('land',))

Expected Behavior:
I would expect the model to simply run and return a streamflow object.

My understanding is that we have a few options here:

Modify the HBVEC emulator to add Lake HRUs. This involves adding a water land use class, soil profile, etc;
Filter out the lake HRUs before instantiating the model configuration (done explicitly by the user);
Filter out the lake HRUs within the model instantiation (invisible to the user, or with a warning);

I'm not sure it's a good idea to modify the emulator config and keep the same name (1.).
Thoughts ?

Example of code that would filter out lake hrus and warn the user:
class HRUs(rc.HRUs):
    """HRUs command for GR4J.

    Pydantic is able to automatically detect if an HRU is Land or Lake if `hru_type` is provided.
    """

    root: Sequence[LandHRU]

    @field_validator("root", mode="before")
    @classmethod
    def skip_non_land_hrus(cls, values):
        """Filter out non-land HRUs."""
        hrus = [hru for hru in values if hru.hru_type == "land"]
        if len(hrus) != len(values):
            warn("Only land HRUs are supported for this emulator. Other types of HRUs, such as lakes, were "
                             "filtered out.")
        return hrus

I can apply the same logic to all emulators, using introspection to check what values of hru_type are recognized by the emulator. If the HRU values have a hru_type and it doesn't match the model, I would simply skip them from the config with a UserWarning.

I think option 3 is the best way forward. Users know that when using the RavenPy distributed model builder, it comes with a whole bunch of hypotheses and simplifications. A user warning would suffice I think!

In looking through the Raven manual v3.7 I found the following information that might be useful to treat lakes in HBV-EC:

Lakes in HBV
By default, the :LakeStorage state variable is SURFACE_WATER. For HBV-EC or HBV light emulation
this is modified to SOIL[2]. HBV treats all landscapes as if they have some presence of lakes, but does
not explicitly treat LAKE HRUs. If you would like to revise HBV to properly handle precipitation on a lake,
an additional command is required in the :HydrologicProcesses block:
:Flush RAVEN_DEFAULT [lake storage SV] [SURFACE_WATER]
:-->Conditional HRU_TYPE IS LAKE
Without this, the precipitation in the ’lake’ storage unit may steadily increase because standard mechanisms
for drainage (e.g., baseflow) are disabled for LAKE type HRUs.
In my opinion, skipping lake HRUs when they are small might be fine, but it could be problematic when they are meant to represent larger lakes. Alternatively (less preferable I think), since "HBV treats all landscapes as if they have some presence of lakes" using the :LakeStorage variable, could these lake HRUs be treated as land HRUs by the emulator?


Remove ModuleIdentityExtension from unnecessary module

Context
As a followup of https://github.com/gradle/gradle/pull/16463 , we removed ModuleIdentity extension dependency in build-update-utils. To avoid duplicate code, some common code was extracted.

@bot-gradle test this

OK, I've already triggered ReadyForMerge build for you.

@bot-gradle test and merge

Pre-tested commit build failed.

@bot-gradle test and merge


Upgrade pax stable python version

Description
The pax stable tests include optax as a dependency, which on 1/15 released a new stable version dependent on a newer python version. As a result, this PR updates the python version for the pax stable tests.
Tests
Will include link once the tests are done
Checklist
Before submitting this PR, please make sure (put X in square brackets):

[X] I have performed a self-review of my code.
[X] I have necessary comments in my code, particularly in hard-to-understand areas.
[X] I have run one-shot tests and provided workload links above if applicable.
[X] I have made or will make corresponding changes to the doc if needed.

The fix to the stable tests might require a new pax version, in which case I will hold this PR off for the time being.

The fix to the stable tests might require a new pax version, in which case I will hold this PR off for the time being.

new pax/praxis 1.3.1 is released, will open this PR, ready for review/merge


Allow executing queries even if not all of the variables are specified

Currently, if I try to execute the SQL query NPoco checks if all of the variables are specified, and if some of them are not - it throws the exception. I consider this behavior as overly defensive and here are the reasons why.

In case the query is malformed it will fail during the execution anyway, I see no point in preliminary failing.
If the variable is declared and assigned right inside the query the query would succeed without a need to specify its value externally, but NPoco demands it to be specified.
Example 1: declare @userid int = 1; select * from users where userid = @userid
In this case, there is a possible workaround - I still can specify a variable externally, passing some dummy value: db.Fetch<User>(sql, new {userid = 0});
But when the variable has some custom type (e.g. Table) there is no easy workaround:
Example 2:

declare @t table (id int)
insert into @t values
(1), (2), (3)
select * from users where userid in
(select id from @t)
order by userid

(The example is extremely simplified; normally I declare table variables only when it is really necessary). Instead, I have to create a temporary table, which is not always applicable.
4. And finally, the existing mechanism for parsing input parameters in SQL does not care about where this 'variable' is defined. Which means if I have an SQL where the @-sign is used inside the string or in the commented block the query will fail even though it is perfectly fine for RDMS.
Example 3:
/*Uncomment for debugging:
declare @userid int = 1 */
select * from users
-- where userid = @userid

Example 4:
select * from users where email like<EMAIL_ADDRESS>
Examples I provided may seem quite synthetic, but I try to use NPoco for getting data for reports using quite complicated SQL queries. Many of them use table variables and a whole bunch of ordinary variables; and for the sake of debugging, I have commented sections with examples of input parameters too. So I find it quite frustrating that NPoco fails to execute such queries.
The proposed solution simply removes the throwing of the exception if the variable (or something that looks like a variable) is not found among the input parameters, leaving the validation to DBMS parsing engine.

Hi! It's been almost 6 weeks without any response. Could you please approve or reject the PR so I could act accordingly?

Apologies, this seems like and ok change however it is a breaking change (and personally i wouldnt want it), so it would need to live behind a Database instance property that can be switched on or off.

I tried to do this "switchable" from the beginning but had some difficulties with this. Will try once more.

Ah, here it is.

There is a public class ParameterHelper with a static method ProcessParams that throws the exception. I can add a flag allowMissingParameters with default value = false so that existing behavior won't change unless the flag is explicitly set.
When this method is called from SqlExpression class we are able to set the flag value according to the setting specified in IDatabase instance which is available here.
But when this method is called from Sql class, which is public too, we cannot do the same as this class has no relation to IDatabase instance. The user can now call Sql.Builder.SQL and it throws the exception, and I see no good way to make this behavior optional not breaking the functionality and not introducing global variables controlling the Sql class behavior (which, in my opinion, is a greater evil than the breaking change).

I think all of these problems are solved by escaping.
Have you tried escaping the @ symbols by @@. This will leave them untouched,
declare @@t table (id int)
insert into @@t values
(1), (2), (3)
select * from users where userid in
(select id from @@t)
order by userid

I think all of these problems are solved by escaping.

Not all of them - consider select * from users where email =<EMAIL_ADDRESS>But I admit this is a rare case. Escaping may be used as a workaround indeed.

select * from users where email = 'user@@gmail.com'

select * from users where email = 'user@@gmail.com'
select * from users where email =<EMAIL_ADDRESS>Try to run both queries against the real data anywhere aside NPoco and compare the result. Again, it's a decent workaround for most of the cases, but it is really inconvenient to escape certain characters specially for NPoco.


Fixed grammar and markdown link

Quick edit in order to fix some typos.

Thanks but some of what you corrected is not compliant with what I wanted to say, so I can't accept your pull request, but I will use it to do the corrections.

Ok, for which sentence it got wrong ?

Things like "slots got empty", they are instead  "released", as the free of malloc. Sometimes grammar is not enough in the computer world.

Ah yes my bad I thought about malloc too is 'clear' ok?


Managed Identities on Storage accounts are incompatible with Data Flows when Soft Delete is enabled

I found this out the hard way, and spent countless hours trying to figure this out. It would be helpful if the documentation included some mention of the fact that if you're using Managed Identities for a StorageV2 account that has Soft Delete enabled for blob storage, this will not work with an ADF Data Flow.
You'll end up with an error message similar to this:
Job failed due to reason: at Sink 'ProcessedData': org.apache.hadoop.fs.FileAlreadyExistsException: HEAD https://<storageacctname>.dfs.core.windows.net/<dir>//?action=getAccessControl&timeout=90\nStatusCode=409\nStatusDescription=This endpoint does not support BlobStorageEvents or SoftDelete. Please disable these account features if you would like to use this endpoint.\nErrorCode=\nErrorMessage=.

Best place to document this is probably around here:
https://docs.microsoft.com/en-us/azure/data-factory/connector-azure-blob-storage#managed-identity
Hopefully this helps save someone else a lot of lost time.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: df7b56de-6c44-a6a7-7cb1-2d77025f17f5
Version Independent ID: fa99fc00-8d68-a1de-558e-0ef651548f34
Content: Copy and transform data in Azure Blob storage - Azure Data Factory
Content Source: articles/data-factory/connector-azure-blob-storage.md
Service: data-factory
GitHub Login: @linda33wj
Microsoft Alias: jingwang

@dpolivy - Thanks for your valuable inputs. I am looking into it and will get back to you shortly.

@dpolivy - I have seen that this is added to the documentation recently. Hence, closing the issue. Thanks for your contribution, please feel free to comment for further queries.


The library does not generate correct QR Codes for inputs that have more than 16 characters

With any input of 1-16 characters, the library correctly generates QR Codes. These QR Codes are consistent with other QR Code generation tools and read correctly in my tests (tested with Android Studio with an React Native using “expo-barcode-scanner” library).
However, for 17+ characters, the generated QR Code doesn't seem valid anymore.

I have also observed this issue.

Use this package qrcode.react

How to reproduce this issue? I've tried to generate 17+ length qr codes and they seem valid.

In my case work completely fine checkout: https://devtools-sabeerbikba.vercel.app/QrCodeGenerator
Source code: https://github.com/sabeerbikba/dev.tools/blob/main/src/pages/QRCodeGenrator.jsx


Kafka version update - indexing service

Description

https://issues.apache.org/jira/browse/KAFKA-7755

Since 2.1.0 Kafka clients are supporting multiple DNS resolved IP addresses if the first one fails. This change has been introduced by https://issues.apache.org/jira/browse/KAFKA-6863. However this DNS resolution is now performed only one time by the clients. This is not a problem if all brokers have fixed IP addresses, however this is definitely an issue when Kafka brokers are run on top of Kubernetes. Indeed, new Kubernetes pods will receive another IP address, so as soon as all brokers will have been restarted clients won't be able to reconnect to any broker.

Currently only way to force re-resolution on the druid is side is by terminating kafka supervisors and resubmitting them, if you run a cluster ingesting from kafka on k8s

Fix versions are available 2.1.1 and above, this PR updates it to 2.2.1

I think we've been seeing this occasionally. FWIW this led to the symptom of negative offset lag being reported — just mentioning this in case anyone else is searching for that.

Actually, is there any chance of backporting this (or the smaller version that is 2.1.1) to 0.16? I guess 0.16.1 is about to be released...  This is a regression in Druid 0.15.

(Note: a workaround for k8s users like us is to create a k8s Service per Kafka pod, and ensure that Kafka's advertised.listeners advertises the service hostname like kafka-0.NAMESPACE.svc.cluster.local:PORT instead of the pod hostname (which looks like kafka-0.kafka.NAMESPACE.svc.cluster.local with an extra copy of the statefulset name).)

Actually, is there any chance of backporting this (or the smaller version that is 2.1.1) to 0.16? I guess 0.16.1 is about to be released... This is a regression in Druid 0.15.

@glasser We'll be doing an 0.17.0 release very soon, so this would be included with that


Use stable ID generation in HTML5

The HTML5 output should use stable @headers and @id generation instead of the current one that is based on generate-id() which is not stable. Using a stable approach would allow e.g. DITA-OT site to have smaller change sets when @headers would not change between every conversion.

@eerohele do you have time for a review?

Added XSpec tests and copied ID related functions from XHTML to HTML5. This made tests simpler and in the future when we remove the dependency to XHTML this will be done in any case.

The stable IDs could maybe be made even more stable by using topic_id + closest_ancestor_element_id + counter_in_that_element. What do you think? Then changes to topic content would be limited to closest ancestor element and the counter scope would no longer be the whole topic.

@eerohele do you have time for a review?

I'm a bit late, but looks good to me, and a big :+1: for this change. I diff HTML outputs quite often when developing plugins and this change makes regression testing much more straightforward.
Pretty much the only (very minor) thing is that we could call dita-ot:get-closest-topic($element) instead of doing  $element/ancestor-or-self::*[contains(@class, ' topic/topic ')][1], but that'll require rejigging the XSpec tests a bit, because that function isn't currently available in the XSpec tests.


Verify download checksum

Is your feature request related to a problem? Please describe.
Downloads should be verified against at least their checksums (currently sha256) before execution.
Describe the solution you'd like
Compare the downloaded file to the checksum. If verification fails, retry some number of times, then fail the job.
Describe alternatives you've considered
chtf, which validates the checksums:
https://github.com/Yleisradio/chtf/blob/c34f6265b47dda812cc1936871ed79d250933ed1/chtf/__chtf_terraform-install.sh#L202-L212
tfenv, which validates checksum and GPG signature:
https://github.com/tfutils/tfenv/blob/6d16fa586a991b94b1dcb6dd7e4597b24b8b2a1e/libexec/tfenv-install#L148-L160
terraform-exec : tfinstall, which also doesn't validate the download.
Additional context
Currently, we're using a Docker image for Debian Buster and installing terraform. Addressing this issue would be a step toward allowing us to migrate to this Orb, allowing for a simpler CircleCI customer config.

Typically we recommend people use a relevant or custom Docker image with the tools required pre-installed, as it helps reduce overhead of having to install things on-the-fly. Install steps aren't encouraged but are added to help in the case it might really be necessary (e.g using one of our machine or macos images).
I agree we should checksum stuff we download, and will definitely look at other orbs to see which ones we should incorporate this idea into. However installing things during runtime in our Docker executor is discouraged :)


Normative: Exclude "standard" and "search" from CollationsOfLocale

Address https://github.com/tc39/proposal-intl-locale-info/issues/31
This PR is logical because in https://tc39.es/ecma402/#sec-intl-collator-internal-slots
The values "standard" and "search" must not be used as elements in any [[SortLocaleData]].[[<locale>]].[[co]] and [[SearchLocaleData]].[[<locale>]].[[co]] list.

@anba

If we want to exclude "standard" and "search", we should also amend this description a bit:

[...] descending preference of those in common use in the locale for string comparison.

Because "standard" and "search" are certainly the two most preferred collation types for string comparisons.

If we want to exclude "standard" and "search", we should also amend this description a bit:

[...] descending preference of those in common use in the locale for string comparison.

Because "standard" and "search" are certainly the two most preferred collation types for string comparisons.

@anba  I totally understand your point in https://github.com/tc39/proposal-intl-locale-info/issues/33 but I just think it will be easier if I can address that issue separately in a different PR and focusing on first ensure that two value are clearly excluded.

I made a mistake while merging the "unique" and "lower case" should be kept.

TG2/ECMA402 2021-06-03 agree with this change.

2021-06-03 discussion: https://github.com/tc39/ecma402/blob/master/meetings/notes-2021-06-03.md#normative-exclude-standard-and-search-from-collationsoflocale


feat: move to klog/v2

klog/v1 is not supported anymore, move to a recent version of klog.

Thank you!


Add permissions to the example action definition

These are the minimum permissions needed to run Infracost. I use these on my own workflows.

This seems like a good addition, we've seen other users hit this enough times that we added this section to the readme but I'll leave it with @aliscott to review/decide.

Yeah, I PR'd this because I was also confused as to what I needed and eventually found that part of the readme. I think it would be a smoother new user experience if it was simply in the example action to begin with.

This makes a lot of sense to me, thanks @Octogonapus! I think we should include this in all the examples so they are consistent.
Test failures are unrelated, so no action needed on those.

I think we should include this in all the examples so they are consistent.

Will do.


“memo” versus “memoize”

Early bikeshedding. No matter what we decide in #2, we will need to decide whether we would rather call the new metafunction “memo” or “memoize”. I’m fine with either.

ime "memoize" constantly tries to be autocorrected to "memorize", so i'd vote "memo".

I kind of like it spelled out. But, I think that's mostly because I see and use it in its long-form more often than its short form, so that's what I'm comfortable with.
But, readability-wise, I could see people seeing the function for the first time, mis-reading it as "memorization" (really easy to do), and start trying to google for what the memorize function is.
I know some might vote for "memo" because it's a bit shorter, though I don't think that should be a strong concern - I would assume that most usages of memoization would require some extra options to be passed in, like customizing the caching strategy or configuring how it decides if two keys are equal - the extra baggage required to do all of this would certainly shadow the verbosity of the couple of extra "ize" characters, making them insignificant. But, perhaps I'm wrong about how often additional arguments will be necessary.

Oh also, British people might try to use “memoise” :-p

I too prefer "memoize". Memoization is a common technique with a known name. I think the British version uses the same name instead of "memoisation". And it is easy to associate "memoize" with the technique instead of memo. "memo" is commonly known an abbreviation of "memorandum".
Extending what @Andrew-Cottrell said, it would be easy to search what memoize is instead of memo, where you will mostly get results of papers and post-it's
But hey, "memorize" isn't a bad name since the idea is to "memorize" the value! :P


Big breaking changes in rest protocol (/api/v1 --> /api/v2)

Circle CI failed, but this is not related to my modification
(docker pull failed due to some problem between CircleCI and DockerHub)

Pulled, thanks!


CCI task "push_qa" crashes for Unlocked package with no namespace

Describe the bug
cci task run push_qa --orgs orgs/qa_orgs.txt --org devhub --metadata_package_id 0337S000000XXXXXXX
crashes when the package is an Unlocked package with no namespace.
It seems to expect that the namespace is returned by the query :
SELECT Id, name, namespaceprefix FROM MetadataPackage WHERE Id = '0337S000000PKO8QAO'
In other calls to the same "run push_qa' command, the following query is called :

Error message is :
Error: 'NoneType' object has no attribute 'split'

### Reproduction steps

1. Create an unlocked package and promote it
2. Install this package in an org
3. Declare this org as a global org in CCI services
4. Create a file qa_orgs.txt containing the orgid of this org
5. Run the command
```cci task run push_qa --orgs qa_orgs.txt --org <devhub org name> --metadata_package_id <package 033 id>```

### Your CumulusCI and Python versions

% cci --version
CumulusCI version: 3.69.0 (/Users/rupert/.local/bin/cci)
Python version: 3.10.2 (/Users/rupert/.local/pipx/venvs/cumulusci/bin/python)

### Operating System

MacOS 11.7

### Windows environment

_No response_

### CumulusCI installation method

pipx

### Error Gist

https://gist.github.com/cb6db108ca037c674e27a13dfd6a64d8

### Additional information

This bug probably also applies to the push_sandbox task, and maybe others ?

OK, it seems that, by default, the Packaging Push API is not activated :
https://help.salesforce.com/s/articleView?id=000385292&type=1
It would be nice to catch the exception above and display a user-friendly error message telling the user to get this API activated.


Backport 16094 to v1.14 branch

on non-XIP system, loading address shouldn't be located on FLASH memory region.
on non-XIP system, SRAM is the default region, and relocated .data
section and .bss section of SRAM shouldn't be inserted between
_image_rom_start and _image_rom_end, because the memory region between
_image_rom_start and _image_rom_end will construct the mpu ro region.
Also for the newly added memory region on non-XIP system, the
relocated .text secition and .rodata section should also be mpu aligned.
Fixes #16090.

Please update the title to what this PR is fixing, add a bug describing the bug you are fixing and then label the PR as Backport.

The body of the PR is very dense and difficult to read. Please restructure it a bit.


Do not assume that fclose can be converted to int()(FILE)

Do not assume that fclose can be converted to int(* )(FILE*) because some compilers use a special calling convention for stdlib functions like fclose.
This change is necessary to make pugixml successfully compile as part of our game project where we use vectorcall as default calling convention on x64. The function fclose is explicitly marked as cdecl which makes the function incompatible to the function pointer type int(* )(FILE*).

Thanks, merged!


FIX: optimizations and small bug fix

Description of changes:
Some optimizations, PEP8 corrections.
Bug fix when removing missing labels. Old code was not working, when the labels are pd.Series with index different than 0, 1, 2, ....
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Job PR-373-1 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-373/1/index.html


fix(tools): force ligth mode in composer devtools

Motivation / Background
Composer devtools aren't usable with dark mode system preference and implementing proper dark mode support doesn't seem like an easy task, so a quickfix suggestion here.
Here's how dark mode looks:

These changes remove dark mode for the whole app while devtools is open, right? I wonder if there's a way to just scope them to devtools component. Perhaps @thure has an idea there. Would nesting a ThemeProvider with hardcoded light mode help?

Yeah, correct.
Nesting ThemeProvider was my first try, but it didn't work and I decided not to spend too much time trying to understand why. If I just missed smth then yeah, that'd be a better solution.

I think a solution like this is risky and likely to encourage further color token confusion in future feature work.
Instead, devtools should be refactored to support dark mode. It’s much easier now to choose color tokens using the new semantic tokens, and finding tokens that need replacing in devtools is also not difficult.
If this has become important, I can try to squeeze it in, cc @zhenyasav.

ok, closing the PR, will just keep it as a local change


Fix connection object

I was asked by @dothebart  to find the cause why each
request creates a new connection to the db api.
So I identified AikidoSession as the culprit as this
class loose the scope to the requests session object
which leads to a connection close.
Furthermore I've cleaned up the connection.py
code a little bit. But as the current API method naming
doesn't fit pyhon pep specs, it's still not very clean
an pythonic.
If functionality of Aikido or the former JSONHook
is still needed or wanted:
I thinks this should be better implemented by using requests
hook method,  subclassing requests session and return objects
or decorators.

Current coverage is 82.81% (diff: 85.71%)

Merging #46 into master will decrease coverage by 0.16%

@@             master        #46   diff @@
==========================================
  Files            12         12
  Lines          1768       1722    -46
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
- Hits           1467       1426    -41
+ Misses          301        296     -5
  Partials          0          0

Powered by Codecov. Last update 5d112cd...c86fedd

Thank you for narrowing down the search. The true culprit was the with statement inside AikidoSession that caused the connection to be closed after each request.
However I can't merge your pull request as it is because of risks of unintended consequences :

AikidoSession is needed. It centralizes requests, authentication procedures, logs, error handeling. I agree that the design can be surprising since the standard way of achieving the same result would be by inheriting from Session. However, because of the way inheritance works in python, it is very easy to over write important attributes of the base class and end up struggling with hellish bugs. For this reason I usually elect to use this deflecting pattern to wrap complex classes that I don't maintain. I find this architecture with two distinct processing levels way easier to debug.

username, password default value is None and not "", because ArangoDB accepts empty strings.

The naming conventions are a bit different of what PEP suggests, but they are consistent throughout pyArango, and also with Arango's javascript interface. Furthermore, I personally find that white spaces around =s and camelCase improves code readability. Changing the naming conventions at this point is dangerous.

But thanks a lot for the hint, I was able to fix the issue thanks to you.

@tariqdaouda can you cherry-pick c86fedd6611da7b292c8a162f64218d8515d0284 from this pr? its a fix to the debian graph example.