3D Non Max Suppression feature request

Following the first thread in tensorflow/tensorflow.
Describe the feature and the current behavior/state.
TensorFlow already incorporates a native Non Max Suppression algorithm for 2D bounding boxes. It is used for instance for objet detection tasks, usually after a anchor box generation step as it is observed in YOLO. Another remarkable use exists within the Mask R-CNN algorithm: many anchor boxes are generated (~200,000-300,000 boxes). After a step to reduce this number to 6,000 using some classification results, tf.image.non_max_suppression is called to sort these 6,0000 best anchors and finally obtain 2,000 regions of interest.
The feature would consist of a Non Max Suppression algorithm for three dimensional bounding boxes. This 3D Non Max Suppression would almost have the same parameters than the existing 2D implementation, called with
tf.image.non_max_suppression(
    boxes, scores, max_output_size, iou_threshold=0.5,
    score_threshold=float('-inf'), name=None
)

with boxes containing no more the 4 coordinates of the two points that define a 2D bounding box, but the 6 coordinates of the two points defining a 3D bounding box. The other parameters would stay the same. The heart of this addition would be to adapt the IOU (Intersection Over Union) function within the Non Max Suppression C file to a three dimensional IOU evaluation.
Relevant information

Are you willing to contribute it (yes/no): Yes although I've almost no experience in C++
If you wish to contribute, then read the requirements for new contributions in CONTRIBUTING.md
Are you willing to maintain it going forward? (yes/no): Yes but it will not be necessary
Is there a relevant academic paper? (if so, where): No
Does the relavent academic paper exceed 50 citations? (yes/no): No
Is there already an implementation in another framework? (if so, where): Some tried to implement it in PyTorch (see benefice part)
Was it part of tf.contrib? (if so, where): No

Which API type would this fall under (layer, metric, optimizer, etc.)
It falls under the image processing API part (tf.image). The feature would add a Non Max Suppression algorithm tf.image.non_max_suppression_3d to TensorFlow for 3D bounding boxes following the structure of the existing 2D implementation. It will add three files to tensorflow/tensorflow/core/kernels/image/ : non_max_suppression_3d_op.cc, non_max_suppression_3d_op.cu.cc and non_max_suppression_3d_op.h.
Who will benefit with this feature?
This addon will greatly benefit to the increasing community of 3D object detection or 3D instance segmentation. The demand is increasing. I give two examples: this stackoverflow thread (alas removed by its author now), and this implementation in PyTorch that possesses a not working 3D implementation of the NMS (the implementation author have been asked many times to correct it). It actually exists some implementations in Python language but the speed performance given by a C implementation would be crucial when handling 3D data.
Any other info.

Can you check the function we have in Tensorflow 3d?
Probably they could be interested to have a high performance implementation of this op.
It was presented one months ago: https://ai.googleblog.com/2021/02/3d-scene-understanding-with-tensorflow.html

@bhack I've implemented 3D versions of Non Max Suppression and Crop And Resize algorithms, run basic tests with them, and used them successfully within a deep network. Before following the Contributing guidelines, I believe it is still up to you to decide if these algorithms should be included in tensorflow-addons as this thread is still marked as "ecosystem-review". Do you think it is relevant for tensorflow/addons?

I am monitoring your last comment at https://github.com/google-research/google-research/issues/660

It could be interesting to have a learnable 3d nms like:
https://github.com/abhi1kumar/groomed_nms/blob/main/lib/groomed_nms.py

TensorFlow Addons is transitioning to a minimal maintenance and release mode. New features will not be added to this repository. For more information, please see our public messaging on this decision:
TensorFlow Addons Wind Down
Please consider sending feature requests / contributions to other repositories in the TF community with a similar charters to TFA:
Keras
Keras-CV
Keras-NLP

3D Non Max Suppression and 3D Crop And Resize: https://github.com/gdavid57/3d-nms-car-custom-op


Update index.js

Catch error instead of crash when renameSync() function fails.

Hi, may be it's better to make commit message more readable, not just update index.js.

Ok! You are right sorry :)


netutils: introduce libwebsockets support

Summary
Introduce libwebsockets support
Impact
None
Testing
Pass CI.
Local build with CONFIG_NETUTILS_LIBWEBSOCKETS=y pass

@wangyingdong1 please feel free to do that. I got started with this, but some project priorities changed, so I stoped working on this. You can either reuse results of my work, so do all from scratch by your own.

@pkarashchenko the update version(https://github.com/apache/nuttx-apps/pull/2733) is merged, let's close this pr.


Vera/ENG-402 banyan-cli Stable Compilation

Description
Removed all features and got library compiling on stable. Updated CI to test for the same.
Link to issue
https://linear.app/banyan/issue/ENG-402/banyan-cli-stable-compilation
Type of change

[x] Refactor (non-breaking change that updates existing functionality)

looks good 👍


[bug] Rust panic in tauri_plugin_fs_extra from crash reporting in our installbase

Describe the bug
We've recently integrated Bugsnag for crash reporting in our app phcode.io. This report summarizes a recurring crash issue observed in macOS.
Overview

Platform Affected: macOS only
Deployment Date: About 2 days ago, 12-Sep-2024
Total Sessions Since Deployment: 13,000 sessions
Total Crashes Reported: 173 incidents of the same crash type

Crash Details

Crash Type:  rust panic, probably at app start itself.
macOS only
Function Causing Crash: tauri_plugin_fs_extra::system_time_to_ms. The crashes are pointing to this single function in all stacks. See stack below.
We do not capture or report the full file paths in the crash stack for privacy(potentially involving usernames in the paths). Sorry for not having more info on file paths.

Reproduction
This is crash reporting from our installbase.
This looks like the offending code: https://github.com/tauri-apps/tauri-plugin-fs-extra/blob/v1/src/lib.rs#L80
Expected behavior
no crash
Full tauri info output
N/A production crashlitics

Stack trace
Rust_Panic
Unknown panic message
Unknown panic message
Sept 12th 2024, 22:29:13 GMT+5:30

STACKTRACE

Rust_Panic Unknown panic message
     (path redacted for privacy) phoenix_code_ide::bugsnag::handle::hf6fa730ab2ba5fdc
     (path redacted for privacy) phoenix_code_ide::main::{{closure}}::h63ef96d811aaafb2
     (path redacted for privacy) std::panicking::rust_panic_with_hook::h09e8a656f11e82b2
     (path redacted for privacy) std::panicking::begin_panic_handler::{{closure}}::h1230eb3cc91b241c
     (path redacted for privacy) std::sys::backtrace::__rust_end_short_backtrace::hc3491307aceda2c2
     (path redacted for privacy) _rust_begin_unwind
     (path redacted for privacy) core::panicking::panic_fmt::ha4b80a05b9fff47a
     (path redacted for privacy) core::result::unwrap_failed::h441932a0bca0dd7f
     (path redacted for privacy) tauri_plugin_fs_extra::system_time_to_ms::h397a5ccb275f176e
     (path redacted for privacy) <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll::h37c3e63198041c1c
     (path redacted for privacy) <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll::ha3a0836894dd9ab6
     (path redacted for privacy) tauri::hooks::InvokeResolver<R>::respond_async_serialized::{{closure}}::h3f627abb3f53c634
     (path redacted for privacy) tokio::runtime::task::core::Core<T,S>::poll::ha10d84077f79d4ae
     (path redacted for privacy) tokio::runtime::task::harness::Harness<T,S>::poll::h149b233a188d48fd
     (path redacted for privacy) tokio::runtime::scheduler::multi_thread::worker::Context::run_task::h9aa34f3a4661eb13
     (path redacted for privacy) tokio::runtime::scheduler::multi_thread::worker::Context::run::h787319f1d7277b47
     (path redacted for privacy) tokio::runtime::context::runtime::enter_runtime::h50bae69fda092374
     (path redacted for privacy) tokio::runtime::scheduler::multi_thread::worker::run::hcc941a10239ee947
     (path redacted for privacy) tokio::runtime::task::core::Core<T,S>::poll::h16d0024f82d9c144
     (path redacted for privacy) tokio::runtime::task::harness::Harness<T,S>::poll::hfda8362637abae26
     (path redacted for privacy) tokio::runtime::blocking::pool::Inner::run::hb8b3a7ae99ef937a
     (path redacted for privacy) std::sys::backtrace::__rust_begin_short_backtrace::h25d22e0eed28a485
     (path redacted for privacy) core::ops::function::FnOnce::call_once{{vtable.shim}}::h79d92fea277cd232
     (path redacted for privacy) std::sys::pal::unix::thread::Thread::new::thread_start::h1bd1b9c95010bf71
     (path redacted for privacy) __pthread_joiner_wake

Additional context

Le chatgpt O1 says:
It appears that you're encountering a panic in the system_time_to_ms function of the tauri-plugin-fs-extra crate. The panic is likely caused by unwrapping a Result without handling the potential Err case when converting a SystemTime to milliseconds since the UNIX epoch.
Here's the relevant portion of the code from line 80 in lib.rs:
fn system_time_to_ms(time: SystemTime) -> u128 {
    time.duration_since(SystemTime::UNIX_EPOCH).unwrap().as_millis()
}

Issue:

The duration_since method returns a Result<Duration, SystemTimeError>.
If time is before the UNIX_EPOCH, duration_since will return an Err.
Calling .unwrap() on an Err value causes a panic.

Solution:
Modify the function to handle the Result properly instead of unwrapping it. You have two main options:

Handle the Error Inside the Function
Return a default value (e.g., 0) or handle the error appropriately within the function.
fn system_time_to_ms(time: SystemTime) -> u128 {
    match time.duration_since(SystemTime::UNIX_EPOCH) {
        Ok(duration) => duration.as_millis(),
        Err(_) => {
            // Handle the error (e.g., return 0 or log the error)
            0
        }
    }
}

Propagate the Error to the Caller
Change the function signature to return a Result<u128, SystemTimeError> so that the caller can handle the error.
use std::time::{SystemTime, SystemTimeError};

fn system_time_to_ms(time: SystemTime) -> Result<u128, SystemTimeError> {
    time.duration_since(SystemTime::UNIX_EPOCH).map(|duration| duration.as_millis())
}

Usage:
match system_time_to_ms(some_system_time) {
    Ok(ms) => {
        // Use the milliseconds
        println!("Milliseconds since UNIX_EPOCH: {}", ms);
    }
    Err(e) => {
        // Handle the error
        eprintln!("Error converting time: {:?}", e);
    }
}

Recommendation:
Option 2 is generally better practice as it doesn't hide potential errors and allows the caller to decide how to handle them.
Explanation:

duration_since Method: Calculates the duration between the given SystemTime and UNIX_EPOCH.
Error Handling: By using match or Result propagation, you ensure that your program can handle times before the UNIX epoch without panicking.
as_millis Method: Converts the Duration to milliseconds, returning a u128.

Additional Steps:

Update the Function Signature (if propagating the error):
Ensure all places where system_time_to_ms is called are updated to handle the Result.

Test with Different Times:
Test the function with SystemTime::now() and with times before the UNIX epoch to ensure it behaves as expected.
// Time after UNIX_EPOCH
let now = SystemTime::now();
println!("{:?}", system_time_to_ms(now));

// Time before UNIX_EPOCH
let before_epoch = UNIX_EPOCH - Duration::from_secs(1000);
println!("{:?}", system_time_to_ms(before_epoch));

Why This Error Occurred:

SystemTime Before UNIX_EPOCH: If any file or system time you're working with is dated before January 1, 1970 (the UNIX epoch), duration_since will fail.
Possible Causes: Incorrect system clock, files with incorrect timestamps, or using UNIX_EPOCH - duration without handling.

Preventing Future Errors:

Validate Inputs: Ensure that the SystemTime values you're working with are valid and expected.
Robust Error Handling: Always handle Result types properly to prevent panics.
Logging: Implement logging for error cases to aid in debugging.

Example with Error Logging:
use std::time::{SystemTime, SystemTimeError};

fn system_time_to_ms(time: SystemTime) -> Result<u128, SystemTimeError> {
    match time.duration_since(SystemTime::UNIX_EPOCH) {
        Ok(duration) => Ok(duration.as_millis()),
        Err(e) => {
            eprintln!("Error converting time: {:?}", e);
            Err(e)
        }
    }
}

Conclusion:
By properly handling the Result from duration_since, you can prevent the panic and make your code more robust. Choose the error handling strategy that best fits your application's needs.

Note: Always ensure that you're handling time-related functions carefully, especially when dealing with files or systems that might have timestamps before the UNIX epoch.

Should be fixed with the linked pr above (non default branch so no auto closing...)


Show more info when uploading data to Azure

It costs like 20-30 minutes to upload weatherdata folder in my apartment, and cifar 10 even more.
In that case, we need to show some progress to let the user know we are uploading data, so as to improve their using experience.
@briacht  any thoughts?

here is the message struct from AZCopy
        public override string MessageType => "Progress";

        public string ErrorMsg { get; }

        public string JobStatus { get; }

        public int TotalTransfers { get; }

        public int TransfersCompleted { get; }

        public int TransfersFailed { get; }

        public int TransfersSkipped { get; }

        // includes bytes sent in retries (i.e. has double counting, if there are retries) and in failed transfers
        public double BytesOverWire { get; }

        public double TotalBytesTransferred { get; }

        public double TotalBytesEnumerated { get; }

        public double TotalBytesExpected { get; }

        public float PercentComplete { get; }

        public List<FailedTransfer> FailedTransfers { get; }

        public List<FailedTransfer> SkippedTransfers { get; }
    }

In Progress section, under Status

If user has not uploaded dataset before:
Uploading data to Azure (33.33%...)
If user re-trains with same dataset (without making changes) or if they delete images from dataset:
Data already uploaded. Skipping data upload.
If user re-trains with same dataset after adding more images to local dataset:
Uploading new images to dataset in Azure (33.33%...)

@briacht I just find out (Thanks @JakeRadMSFT ) that we can get the # of skip file while uploading). Do we want to show the skip file either?
I'd prefer do that since it might still take a  long time even we do skip, and before that progress won't be updated, which will be always zero


onTap event is pretty slow

onTap event takes about 2-4 seconds to fire.

sorry about the picture it's my dumb cat.

Hi @YassinEldeeb
Thank you for your issue.
Try to run the app in release mode and see if the issue still appears.
If you want to change the time interval between tap and double-tap, you can use doubleTapInterval prop which I added in v0.2.5

when I published my app to the Google Play Store it was 2x faster and it wasn't a problem anymore, but doubleTapInterval seems a good configuration if I faced any problems.
Thank you for clarifying it's not a performance issue but just the time to determine it's a tap or a Double Tap.


Confirm merge from repo_sync_working_branch to master to sync with https://github.com/MicrosoftDocs/windows-insider-pr (branch live)

Pull request opened by Docs to confirm syncing from https://github.com/MicrosoftDocs/windows-insider-pr (branch live).

@lizap @DaniHalfin - PR is ready for final review. Thank you.


[SPARK-28075][SQL] Enhance TRIM function

What changes were proposed in this pull request?
This pr add support TRIM(BOTH/LEADING/TRAILING FROM str) format.
PostgreSQL, Vertica, MySQL, Teradata, Oracle and DB2 support this format. Hive, Presto and SQL Server does not support this feature.
PostgreSQL:
postgres=# select substr(version(), 0, 16), trim(BOTH from '    SparkSQL   '), trim(LEADING FROM '    SparkSQL   '), trim(TRAILING FROM '    SparkSQL   ');
     substr      |  btrim   |    ltrim    |    rtrim
-----------------+----------+-------------+--------------
 PostgreSQL 11.3 | SparkSQL | SparkSQL    |     SparkSQL
(1 row)

Vertica:
dbadmin=> select version(), trim(BOTH from '    SparkSQL   '), trim(LEADING FROM '    SparkSQL   '), trim(TRAILING FROM '    SparkSQL   ');
              version               |  btrim   |    ltrim    |    rtrim
------------------------------------+----------+-------------+--------------
 Vertica Analytic Database v9.1.1-0 | SparkSQL | SparkSQL    |     SparkSQL
(1 row)

MySQL:
mysql> select version(), trim(BOTH from '    SparkSQL   '), trim(LEADING FROM '    SparkSQL   '), trim(TRAILING FROM '    SparkSQL   ');
+-----------+-----------------------------------+--------------------------------------+---------------------------------------+
| version() | trim(BOTH from '    SparkSQL   ') | trim(LEADING FROM '    SparkSQL   ') | trim(TRAILING FROM '    SparkSQL   ') |
+-----------+-----------------------------------+--------------------------------------+---------------------------------------+
| 5.7.26    | SparkSQL                          | SparkSQL                             |     SparkSQL                          |
+-----------+-----------------------------------+--------------------------------------+---------------------------------------+
1 row in set (0.01 sec)

Teradata:

Oracle:

DB2:

How was this patch tested?
unit tests

Test build #106577 has started for PR 24891 at commit a30fa00.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/11815/
Test PASSed.

Test build #106577 has finished for PR 24891 at commit a30fa00.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/106577/
Test PASSed.

Interesting, it looks like Spark SQL already tries to support this? is this more of a fix to the implementation?

Yes. It is a fix to the implementation.

OK, can you clarify the PR / JIRA? it sounds like this already works to some degree. It'd be good to know what isn't working and what you're fixing here.

Thank you @srowen I updated the PR description.

It seems reasonable to support trimming the default of just whitespace, though I'm surprised not all SQL engines support it. Any opinion, say @cloud-fan or @gatorsmile

thanks, merging to master!


crash in search and pick after update 2.0.6k

Describe the bug
Crash of the toolbox

before the 2.0.6k update it worked as intended.
To Reproduce
Steps to reproduce the behavior:

Go to search and pick
Click on  search
See error

Version Info:

ToyBox Version: 1.4.19

same bug&error code

this is what happens if you try to "rest selected"

the rest all function is working

I get the same error as you after 2.0.6k. Also, Infinite abilities are not working.

@ccmedavid please make a new issue for that.

I'm also unable to reproduce the arcane reservoir one.


A domain you all should enjoy.

If it detects it is a mediawiki wiki, it should go to Special:AllPages
Ex: https://apple.fandom.com/wiki/Special:AllPages

In virtually all cases, Special:AllPages can be discovered from the homepage on a recursive crawl. On standard MediaWiki installations, there's a link to Special:SpecialPages in the sidebar, which in turn links to AllPages (among various other things). On heavily modified ones, like Fandom, it can be a bit less obvious, but there is a direct link to AllPages in the right sidebar on the homepage. Over the years, I haven't come across a single wiki where this addition would've been necessary. So I'm not fundamentally opposed to this, but the benefit is marginal to inexistent.


clarification for the relative coordinates of the control points

For ThinPlateSpline2, two of the arguments are: source, and target and it is written in the readme they are Relative coordinates of the source points (source) and Relative coordinates of the target points (target). In the example given, source points are
s_ = np.array([ # source position
  [-0.5, -0.5],
  [0.5, -0.5],
  [-0.5, 0.5],
  [0.5, 0.5]])

and target points are :
t_ = np.array([ # target position
  [-0.3, -0.3],
  [0.3, -0.3],
  [-0.3, 0.3],
  [0.3, 0.3]])

If we assume the (0,0) is center of the image, then each source point is traveling towards the center (e.g., for first control point from (-0.5,-0.5) to (-0.3,-0.3)); hence I expected the resulted image will be shrinked, but apparently it is not true and it is actually the other way around. Am I missing something? what is the coordinate system for the image?
Any help is appreciated

Yeah, I agree that it can mislead users.
The second version is implemented based on the first version.
In the first version, coord samples pixel pointed by vector from coord position.
In the second version, target is pointed by source with an implicit vector map.
Therefore, I named it source and target, not target and source.

Thank you for your explanation, I really appreciate your prompt reply


Notion Enhancer builds

Adds support for building "pre-enhanced" packages via Notion Enhancer

@dragonwocky got this working 🎉  are you alright if I merge and add notion-enhanced to the repositories?

btw should be doable to make pre-enhanced packages for windows/mac :) mac might be annoying to install because of gatekeeper / notarization

@davidbailey00 go ahead, this looks good.


Correction in Readme.md

Thanks for your great work on IRLS.
There might be some small errors in Readme.md.
Original: In this implementation, we use svd to solve peudo inverse of singular matrices. The svd in torch is somehow less numerically stable than that in tensorflow, so we use a slight regularization in the valina IRLS in pytorch.
Correction:

peuso → pseudo
so we use a slight regularization in the valinda IRLS in pytorch (or in tensorflow?)
what does valina mean here?

Best Wishes!

@GuokaiLiu Sorry to reply sooooooo late and thanks for your correction and comment. I haven't noticed the issue here.
The pseudo inverse is  calculated manually both in pytorch and tf. In pytorch, we need to add a small regularization term to make it numerically stable. It is not needed in tf. You can check the source code.
"valina" means the it is the simplest IRLS.


Resolve merge and add panel back to morphir homepage

Hi @ronaldssebalamu 👋🏻
Great to see activity happening on this PR.
Let me know if you need help merging add-contributing-companies into main and raising a morphir PR.
James.

Hey @James McLeod @.***>
Sure! I would love to move forward with the merging side of things.
Previously, I had signed an EasyCLA agreement as an individual contributor.
This time around, I got flagged after raising a PR from my branch prompting
me to sign again.
Do you have any idea how I could rectify this on my end?
Kind regards,
Ronald.
On Mon, Jan 29, 2024 at 1:27 AM James McLeod @.***>
wrote:

Hi @ronaldssebalamu https://github.com/ronaldssebalamu 👋🏻
Great to see activity happening on this PR.
Let me know if you need help merging add-contributing-companies into main
and raising a morphir PR.
James.
—
Reply to this email directly, view it on GitHub
https://github.com/ronaldssebalamu/morphir/pull/5#issuecomment-1913742240,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A6IXM7ADKH4KEBGIBRRBYS3YQ3GGDAVCNFSM6AAAAAA5QWXEQKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJTG42DEMRUGA
.
You are receiving this because you were mentioned.Message ID:
@.***>


Added Rainbow Animation Buttons

Fixes Issue🛠

Closes #487
Description👨💻

I have added a new set of Animated buttons with Rainbow hover effects to the Buttons section.
Type of change📄

[] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)

How this has been tested✅
I have opened the HTML file using different browser sizes to ensure that the buttons are rendered nicely and are responsive to the size of the screen.
Checklist✅

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have added demonstration in the form of GIF/video file
[x] I am an Open Source Contributor

Screenshots/GIF📷
https://github.com/Rakesh9100/Beautiify/assets/98112914/ce9baebc-d0c2-49d9-bff5-7484c320d640

@Rakesh9100 Kindly review my PR for #487 under JWOC

Merged, Congrats on getting your PR merged to this repo under JWOC 🎉🎉 @zul132


Improved documentation for in-tree QAT OpenSSL Engine installation an…

For in-tree installation and usage of the QAT OpenSSL Engine, the documentation omitted a few steps in example 2 (the  parameter '-with-qat_hw_dir`' was not needed) and the need for developer headers.

Thanks for the PR. Can you please add signed-off  with your name and Email to the commit please ?

Done.

Thank you for the contribution. The changes is now part of v1.3.0 release.


Issue when return string

Hi,
when I try to print some text value , it return optional(" ----text   ")
for example if I have this xml:
<root>
...
<catalog>
<book>Bob</book>
<book>John</book>
 <book>Mark</book>
</catalog>
...
</root>
when I try to print    xml["root"]["catalog"]["book"][1]["author"].element?.text
the output is   optional( "              John           ")   instade of   "John".
Any idea?

@VitoPietrapertosa It looks like your XML has spaces, so the parser is doing what it should. If you want to trim the spaces, you can do that yourself. Have a look here for a Swift 2 solution, but keep in mind this will likely change a bit for Swift 3.
Regarding the Optional() bit, this is because element?.text is going to return a Swift optional. Best practice is to make sure it exists with an "if let" statement, but if you are 100% sure that it will be there, you can use forced unwrapping with an '!' character. See here for more details on Swift Optional chaining.

@VitoPietrapertosa FYI, I'm going to go ahead and close this issue - if you're still having problems, feel free to re-open and post some additional details. Thanks!


[Apple Notes] Importing locked notes

In order to import my locked notes I had to remove the locks from all of the files. It would be great if Importer could ask if I want to ignore the locks and import them anyway.

The locked notes are encrypted so while I wouldn’t say this is impossible (perhaps the credentials could be obtained somehow) it’s probably not an easy u see taking technically..
I believe other Apple Notes decoders have been able to figure out notes with separate passwords but not necessarily ones that use the users’s credentials.

it's probably not an easy task technically..

Ok. It seemed important at the time, but looking back I seem to recall that the locked notes in my case were less than 10% of the total number of notes. I can see that it could be more important for some people, perhaps using Notes in a corporate situation.


get_cell_autotile_coord gives the same information if the tile is at the position 0, 0 in the tileset or the cell doesn't have autotilling

Godot version:
3.2.1.stable.official
OS/device including version:
Windows10
Issue description:
The GDscript function get_cell_autotile_coord() returns a zero vector when the cell doesn't have autotilling. See documentation: https://docs.godotengine.org/en/stable/classes/class_tilemap.html#class-tilemap-method-get-cell-autotile-coord
What the function returns is a vector2 containing the coordinates of the tile in the tileset. But the first tile in the tileset is at the position (0, 0). So it's exactly the same as if the cell doesn't have autotilling.
So the function can return misleading information.
Steps to reproduce:
Execute the minimal project linked. You'll see that the cell at the coordinates (0, 0) get the same result with the get_cell_autotile_coord function as the cell (4, 0) even if it's an empty cell - so obviously no autotiling.
Minimal reproduction project:
Issue_TileMap.zip
I took the liberty to check the tile_map.cpp - if I may add a small contribution to the project.
Maybe the constant INVALID_CELL could be returned is both coordinates of the vector2 to handle this case:

Actually the problem is that in several places default empty values for autotile coordinates are set to (0,0) and I'm not sure why.
Your correction would make function return (-1,-1) if cell is empty, but if there is a tile, which is not autotile - it will still return (0,0).

I agree that this can be seen as miss information.
At the same time this I don't see a scenario where you would use get_cell_autotile_coord without get_cell, can you think of one?
The documentation does say that it will return a zero vector

Returns the coordinate (subtile column and row) of the autotile variation in the tileset. Returns a zero vector if the cell doesn't have autotiling.

I agree that this can be seen as miss information.
At the same time this I don't see a scenario where you would use get_cell_autotile_coord without get_cell, can you think of one?

For example if you use few tilemaps to separate different types of objects and one of them have single autotile in tileset. So you could use only get_cell_autotile_coord to determine if cell is empty or have tile and which autotile it is.
It may not be common use though..

@groud What do you think of this?

I don't mind changing it. While returning something like Vector2(INVALID_CELL, INVALID_CELL), does not make much sense, as the vector is not made of two cell IDs, we might change it to return Vector2(-1, -1).
In any case that does not change the fact the user should first check if the tile is an autotile first,  but I agree it might be simpler to simply check if the returned result is Vector2(-1, -1).


chore(release): v0.8.4 🎉

:robot: I have created a release beep boop
0.8.4 (2023-09-09)
Bug Fixes

domain: contactMethod.channel -> contactMethod.type (5b7e4a7)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/whodisio/whodis-react/releases/tag/v0.8.4 :sunflower:


Pipenv has unversioned dependency of Pipfile

The vendor folder contains a copy of pipfile.
But it looks like the pipfile repo is outdated, for example https://github.com/pypa/pipfile/blob/master/pipfile/api.py#L129-L141 doesn't appear in the vendor folder version.
This makes things difficult:

Any commit to vendor/pipfile needs to be mirrored in the other repository
Tests seem to reside in the pipenv repository, and not the pipfile repository

Is there any reason why we don't just reference pipfile as a dependency in pipenv?

pipfile isn't finalized yet.


Question about builds

I would be grateful if you could enlighten me about the builds. I feel lost, because I see version numbers like "#476" and "#314", as well as "94df3fc". I would like to know what the real version number is, and it would be nice if this was made more visible and simplified.

The latest mainline build can always be downloaded from the links in the README. If you don't need anything else, then that's all you need to know.
Automatic builds are made both for all branches in the mainline repository (i.e. this one, the latest base version of the port), as well as for any Pull Requests submitted to it (e.g. the one at the top is for this PR).
To identify which builds are for PRs and which are for the mainline repo, look at the little blue tag on the right of the build. If it says port, that's the base version. If it says something else, that's someone's PR. You can test these if you want.
The # number is the sequential number of the build. It just counts up for all the builds ever made.
The other hex number is the ID of the commit that triggered the build. This is basically the ID of the source code revision that the build was made from. The latest commit ID is displayed on the front page for the repository (here on the right):

You can use this ID to figure out whether you have the latest version or not. To figure out what exact version you have, you can look at the commit log.

In the link you gave me, there is no version number indicated, and when I look in the port, there is nothing specific indicated. Frankly, I don't think it's well explained, it's too confusing. You don't use the same methods as developers, who use for example v1.0 or v1.10. I can't find myself in your builds and there is no clear indication.

In the link you gave me, there is no version number indicated, and when I look in the port, there is nothing specific indicated. Frankly, I don't think it's well explained, it's too confusing. You don't use the same methods as developers, who use for example v1.0 or v1.10. I can't find myself in your builds and there is no clear indication.

I think the build code is in the title screen, no?

There are no actual version numbers because there are no releases. I don't make them because changes are still frequent and there's no roadmap of any kind. I'm not sure why you need them, since you already have the ability to check whether you have the latest build or not, and all builds are uniquely identified for debugging purposes.
If you want to identify which build you have, the commit ID is displayed on the startup screen under "Variant":

@fgsfdsfgs
I created https://github.com/fgsfdsfgs/perfect_dark/pull/320 to possibly assist with this.
You're welcome to decline it if not interested.

That PR has been merged and now there's an automatically updated release. Automatic builds also now include executables for PAL and JPN.


Add summary to ASP.NET 5 Intro topic

@danroth27  me and @MikeWasson are banning summaries. If you have something useful to say, why is it not in the article? If it's not useful, why is it in the summary. 90%+ of the summaries we have are liabilities, they don't contribute to the article. We should be opening issues - "Remove SPAM summary".


Investigate CI for Kotlin Multiplatform project

have started investigating building the KMP project via CircleCi & Bitrise

looks like both services have free tiers for open-source projects
will be contacting them about getting the free pricing
will be looking at a circleci trial to evaluate their MacOS build


Forward and reverse mode differentiation at the same time

Hey,
is it possible to do forward mode and reverse mode differentiation at the same time in JAX for different sets of parameters, e.g., hyperparameter and network weights, respectively?
Example:
Neural Network with a scalar hyperparameter, e.g., the learning rate.
The goal would be then to train the network using standard reverse mode differentiation, while simultaneously accumulating the gradient of the scalar parameter in forward mode (reverse-mode would require to store all intermediate weight states (= similar to backprop through time)).
Unfortunately i haven't found a way to do this in JAX yet.
Thanks a lot for your help.
Heiner

Can you give a small, self-contained, concrete example of what you want to compute? I'm a little hazy from your description, but if you made the example concrete then I think we might be able to make progress.

Thanks, i will try to present a simple example to see if what i want to do is possible at all before going to the more complicated problem described before.
import jax.numpy as np
from jax import random

key = random.PRNGKey(0)

key, W_key, b_key = random.split(key, 3)
W = random.normal(W_key, (3,))
b = random.normal(b_key, ())

a = 2.0

def predict(W, b, a, inputs):
    return sigmoid(np.dot(inputs, W - a) + b)

# Build a toy dataset.
inputs = np.array([[0.52, 1.12,  0.77],
                   [0.88, -1.08, 0.15],
                   [0.52, 0.06, -1.30],
                   [0.74, -2.49, 1.39]])
targets = np.array([True, True, False, True])

# Training loss is the negative log-likelihood of the training examples.
def loss(W, b, a):
    preds = predict(W, b, a, inputs)
    label_probs = preds * targets + (1 - preds) * (1 - targets)
    return -np.sum(np.log(label_probs))

# TODO
# calculate gradient w.r.t. 'a' by forward-mode
# at the same time: calculate gradients of 'W' and 'b' by reverse mode

# why: e.g. want to update 'a' after a whole epoch of updating 'W' and 'b'


Crash after injection

I'm getting the following error after I trigger the nut-ups node with an injection. After this error Node-RED crashes and I need to restart it.
Error: spawn upsc ENOENT
at _errnoException (util.js:1031:13)
at Process.ChildProcess._handle.onexit (internal/child_process.js:201:19)
at onErrorNT (internal/child_process.js:389:16)
at _combinedTickCallback (internal/process/next_tick.js:138:11)
at process._tickDomainCallback (internal/process/next_tick.js:218:9)

Well, do you have the upsc binary from NUT installed and accessible on $PATH?

Hi ! Same problem here on my Raspberry PI. The upsc binary is working well under shell and accessible on $PATH.
Any help is welcome !

myfreeweb you're right. I'm in Mac and I've discovered that upsc command isn't available.
I'll try to resolve installing upsc if I find an available version. Tnx

Yeah you'll need to run something like brew install nut (with Homebrew)

Hi myfreeweb !
Why did you close the issue ?
I've NUT installed and working on a Raspberry PI .. and got the same issue .. Node-RED crashes ..
Please see my post above ..
Thanks

@moose75 because this is not the same issue, you should've created a new one.
I think I know what's happening in your stack trace. You're using an old version. Well, the latest one on npm probably :D try git master instead. Actually, I'll release a version right now

ok @myfreeweb, sorry about not opening a new issue .. I was thinking it was the same problem.
I'll upgrade asap to 1.0.2 and let you know . Thx

@myfreeweb 1.0.2 works like a charm !
Thanks a lot for your support


Prune dependencies

Ran depcheck and  deptry to try and get rid of some unused dependencies. Need to QA to make sure nothing broke

Going to close this--but we should probably start automating something here


gfsh unable to destroy region that has an hyphen in its name

Added check for a hyphen in region name.  This has to be wrapped in quotes before searching.
Added test to verify creation and deletion of regions with a hyphen or underscore in the name.

+1


Incorrect API Endpoint for Get-RubrikVappSnapshot in Get-RubrikApiData.ps1

Tested with CDM 5.3, the included endpoint currently reads:
'/api/internal/vcd/vapp/snapshot/{id}'
Should be:
'/api/internal/vcd/vapp/{id}/snapshot'

Thanks, this is something we'll include in our 5.3 release!

Thanks @scryb3,
I have taken a look at this cmdlet and it seems that it can indeed be improved upon. I will start work on an improved version of this cmdlet.

Thanks @scryb3,
I have taken a look at this cmdlet and it seems that it can indeed be improved upon. I will start work on an improved version of this cmdlet.


Allow to check if a package is installed by pip3

Right now you usually have two Python versions installed and both exist in your path.
This PR allows you to check if a package is installed by pip into a Python 3 interpreter.

Thanks.

Released as v2.74.0.


chore: test bundler module resolution

📚 Description
This is a 'canary' pull request. It aims to ensure that the ecosystem is ready for a shift in the default Nuxt TypeScript module resolution from 'Legacy' to 'Bundler': https://github.com/nuxt/nuxt/pull/24837. You'll need to run your build + type checking tests to verify that things still work; I'm hoping that CI will do that for us so we get an immediate indication of whether there's anything to investigate further here.
'Bundler' module resolution is recommended by Vue and by Vite, but unfortunately there are still many packages that do not have the correct entries in their package.json.
This might include this module or packages that are used by this module. You can see, for example: https://arethetypeswrong.github.io/?p=nuxt-cookie-control. The initial red X for node10 needs to be fixed in https://github.com/nuxt/module-builder; you can ignore that for now.
If there are any errors reported in CI for this PR, it would be good to fix them for the sake of any users of this module who enable the Bundler module resolution. Feel free to ping me or others on the Nuxt Discord if you need any help resolving any issues discovered by this PR - and feel free to check out https://github.com/nuxt/nuxt/pull/24837 for any feedback or comments from others affected.
📝 Checklist

[ ] All commits follow the Conventional Commit format
[ ] The PR's title follows the Conventional Commit format

Thank you for assisting in the ecosystem! :heart:

:tada: This PR is included in version 7.2.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Kronos crate not being pulled in

cant find crate for kronos_api when building with latest rustc and cargo.
I'm assuming cargo used to pull in sub-modules and init them, but doesn't now, or setting for that changed?

I think https://github.com/bjz/gl-rs/pull/278 fixed this?

If I clone the project, and then init and update submodule, it works as expected, but including this a dependency in a project with gl_generator, and compiling with cargo, it fails.
gfx_gl is failing as well when using it with the build.rs.
When building with the 1.0a release of rustc and cargo, it was working, but just updated today and it is failing with:
rustc 1.0.0-nightly (ed530d7a3 2015-01-16 22:41:16 +0000)
cargo 0.0.1-pre-nightly (8154255 2015-01-14 19:14:12 +0000)

The problem doesn't come from gl-rs, but from the crates that uses it. You need to add a build-dependency to khronos_api, like this: https://github.com/tomaka/glium/pull/307/files

That does work.  Thanks.


Required protoc version is confusing

The readme states to install a protobuf compiler of version 3.15 or greater. However, when I install protoc as instructed at http://google.github.io/proto-lens/installing-protoc.html, and afterward install cargo component, I get the following error:
error: failed to run custom build command for `warg-transparency v0.1.0 (https://github.com/bytecodealliance/registry#9d9e4b17)`

Caused by:
  process didn't exit successfully: `/tmp/cargo-installGXyW0h/release/build/warg-transparency-964a17dd4eb3e10f/build-script-build` (exit status: 101)
  --- stderr
  thread 'main' panicked at 'Building this crate requires a version of protoc (libprotoc) >=15.0, found: libprotoc 3.15.8

  Please install a suitable version of protobuf (see https://github.com/protocolbuffers/protobuf for downloads and instructions as well as https://protobuf.dev/support/version-support/ for info on protobuf's versioning schema)', cargo/git/checkouts/registry-c434769d228bc60a/9d9e4b1/crates/transparency/build.rs:57:9
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

So the error message tells me to install protoc of version >= 15.
I was able to overcome this error, by downloading a protoc binary from https://github.com/protocolbuffers/protobuf/releases and move that to my bin folder instead.

The error message in question comes from the build script of warg-protocol, which isn't in this repository.
It appears the script was updated recently to accommodate changes to protoc versions and we're now using the latest out of the registry repo.
I'm going to close this issue for now but if you continue to run into problems, please open an issue in the https://github.com/bytecodealliance/registry repo.


Implement dynamic faiman model and function to fit this to measurements

[x] Closes #1877
[x] I am familiar with the contributing guidelines
[x] Tests added
[x] Updates entries in docs/sphinx/source/reference for API changes.
[x] Adds description and name entries in the appropriate "what's new" file in docs/sphinx/source/whatsnew for all changes. Includes link to the GitHub Issue with :issue:`num` or this Pull Request with :pull:`num`. Includes contributor name and/or GitHub username (link with :ghuser:`user`).
[x] New code is fully documented. Includes numpydoc compliant docstrings, examples, and comments where necessary.
[x] Pull request is nearly complete and ready for detailed review.
[ ] Maintainer: Appropriate GitHub Labels (including remote-data) and Milestone are assigned to the Pull Request and linked Issue.

@pvlib/pvlib-maintainer This PR request is ready for review.

If general commentary is helpful, I support this premise of this additional because 1) transient (dynamic) temperature modeling is important for sub-hourly modeling and  2) I think tools to fit temperature models to observations are valuable for monitoring in ongoing plant operation (e.g., to make sure back of module temperature measurements don't drift over time due to sensor detachment).

This extension of faiman smooths the inputs, and prilliman smooths the output, both are intended to account for thermal inertia. Is that right?

Fundamentally correct.  The main difference is that my focus is on extracting model parameters, and being able to simulate better is a nice added benefit.

My reaction is that we may want to build this into faiman as a kwarg-activated option, along with the extension in faiman_rad. Thoughts?

I see both faiman_rad and faiman_dyn as baby-steps toward more advanced modeling of PV operating temperature.  As such their formulations (APIs) are also no more complex than they need to be, in the hope that more people will try them and like them.   I have more comprehensive models that pull them together, but those are even further from being published.
Never-the-less, there may be some kwarg-activated option that fits the bill.

Here is some more specific feedback:

in the gallery example, I would add units to the x-axis label in the first plot (e.g., "Thermal inertia (minutes)").
I like it! I ran an example on EPRI's SSRC data, where there was a known back of module temperature sensor issue right after commissioning. A fit Faiman model appears to match observations better than sapm_module, so I think could do a better job of detecting partial sensor detachment. A full gist is here https://gist.github.com/williamhobbs/7ae27e1d5202e58d526314cd53439643, and here's the main output:

I think sapm_cell would have fit better than sapm_module in this case, but it isn't clear to me why.

Great to see it worked out of the box, @williamhobbs! Do the stats include night-time values?

Do the stats include night-time values?

No, I filtered on POA > 200 W/m^2 (chosen arbitrarily). Nighttime matched regular Faiman and overestimated relative to measured:

@adriesse, I just remembered that the EPRI SSRC dataset* has downwelling longwave/IR irradiance, which could be useful for testing faiman_rad. It looks like tmod_sample_data_subset.csv is new for this PR. Maybe the EPRI dataset would be more generally useful for testing/demonstrating temperature models? I could check with EPRI on license/copyright/etc. for including a subset of the data in pvlib, if that's of interest to the pvlib team. Alternatively, maybe there's data from Sandia/NREL/NIST/etc. that has everything as well.
*Southeastern Solar Research Center, Birmingham, AL, operated ~2014-2020, down-sampled dataset here https://dashboards.epri.com/ssrc.
You can get the EPRI SSRC data with something like:
# Import SSRC data, based on https://github.com/pvcaptest/pvcaptest/issues/89#issue-1816276179
import pandas as pd
import requests
from io import BytesIO

# (takes about 2 minutes with my internet connection)
# Import EPRI SSRC data. Data descriptions are at:
# https://github.com/epri-dev/PV-Subhourly-Clipping/blob/main/Data%20Channel%20Descriptions.csv
try:
    df_temp = pd.read_pickle('ssrc_all_df_temp.pkl')
except:
    urls = ['https://github.com/epri-dev/PV-Subhourly-Clipping/raw/main/SSRC01.zip',
            'https://github.com/epri-dev/PV-Subhourly-Clipping/raw/main/SSRC02.zip',
            'https://github.com/epri-dev/PV-Subhourly-Clipping/raw/main/SSRC03.zip',
            'https://github.com/epri-dev/PV-Subhourly-Clipping/raw/main/SSRC04.zip',
            'https://github.com/epri-dev/PV-Subhourly-Clipping/raw/main/SSRC05.zip',
            'https://github.com/epri-dev/PV-Subhourly-Clipping/raw/main/SSRC06.zip',
            'https://github.com/epri-dev/PV-Subhourly-Clipping/raw/main/SSRC07.zip',
            'https://github.com/epri-dev/PV-Subhourly-Clipping/raw/main/SSRC08.zip',
            'https://github.com/epri-dev/PV-Subhourly-Clipping/raw/main/SSRC09.zip',
            'https://github.com/epri-dev/PV-Subhourly-Clipping/raw/main/SSRC10.zip',]
    dfs = [pd.read_csv(BytesIO(requests.get(url).content),compression='zip', header=0, sep=',', quotechar='"', index_col=0) for url in urls]
    df_temp = dfs[0].join(dfs[1:])
    # Convert timestamp to datetime (takes about 1 minute)
    df_temp.index = pd.to_datetime(df_temp.index)
    df_temp.to_pickle('ssrc_all_df_temp.pkl')

# keep relevant columns
# SAT_Temp_PV_Module_10 is known bad in 2014, see https://github.com/pvlib/pvanalytics/issues/143
df_ssrc = df_temp[[
    'SAT_AC_Power',
    'SAT_Irradiance_POA_South',
    'SAT_Irradiance_POA_North',
    'SAT_Temp_PV_Module_5',
    'SAT_Temp_PV_Module_10',
    'WS_Temp_Air_Outdoor_Amb',
    'WS_Wind_Speed_Avg',
    'WS_Irradiance_Global_Horiz',
    'WS_Irradiance_NDLW_GR3Shaded']].copy()

df = df_ssrc[['SAT_Irradiance_POA_South',
             'WS_Wind_Speed_Avg',
             'WS_Temp_Air_Outdoor_Amb',
             'SAT_Temp_PV_Module_5',
             'SAT_Temp_PV_Module_10',
             'WS_Irradiance_NDLW_GR3Shaded']].copy()
df.rename(columns={'SAT_Irradiance_POA_South': 'poa_global',
                   'WS_Wind_Speed_Avg': 'wind_speed',
                   'WS_Temp_Air_Outdoor_Amb': 'temp_air',
                   'SAT_Temp_PV_Module_5': 'temp_pv',
                   'SAT_Temp_PV_Module_10': 'temp_pv2',
                   'WS_Irradiance_NDLW_GR3Shaded': 'ir_down'},
                   inplace=True)

Do the stats include night-time values?

No, I filtered on POA > 200 W/m^2 (chosen arbitrarily)

Yes, I should have seen that in the code.  From my more attentive re-reading of the code, it appears that 4-day averages (resampled, not rolling) are the basis of the statistics. That's perhaps a bit unusual and probably the reason for the relatively small deviations in all cases.

Edit to clarify my intention: It looks like tmod_sample_data_subset.csv is new for this PR. If a new measured timeseries dataset is going to be added to pvlib, perhaps it could be useful for demonstrating more than this set of functions. For example, if it included downwelling longwave, it could be used faiman_rad. If it included GHI/DNI/DHI in addition to POA, it could be used for demonstrating many irradiance models. I know of at least one dataset from EPRI that might be a good fit.
@adriesse, I just remembered that the EPRI SSRC dataset* has downwelling longwave/IR irradiance, which could be useful for testing faiman_rad. It looks like tmod_sample_data_subset.csv is new for this PR. Maybe the EPRI dataset would be more generally useful for testing/demonstrating temperature models? I could check with EPRI on license/copyright/etc. for including a subset of the data in pvlib, if that's of interest to the pvlib team. Alternatively, maybe there's data from Sandia/NREL/NIST/etc. that has everything as well.

tmod_sample_data_subset.csv is a subset of the data found here: https://datahub.duramat.org/dataset/module-temperature
which does indeed include down-welling long-wave, and the notebook found at that location also demonstrates faiman_rad toward the end.
I think there is an objective to avoid large data files within pvlib in order to keep it small.

Maybe the EPRI dataset would be more generally useful for testing/demonstrating temperature models?

In my opinion, yes!  I didn't see an easy way of getting it through the dashboard though.  Maybe you could clean up and prepackage something into an hdf or netcdf file?

Maybe you could clean up and prepackage something into an hdf or netcdf file?

I've asked @dfregosi and team at EPRI to assign a license (e.g., CC-BY-4.0) to their dataset (https://github.com/epri-dev/PV-Subhourly-Clipping) so that there are no issues with using, modifying, and redistributing the data.
Would it be worthwhile to start a GitHub Discussion on what portion(s) of that dataset to include, ideal file format, etc.?

I just started that dataset discussion here https://github.com/pvlib/pvlib-python/discussions/1889.
On the dataset currently included in this PR, I think some sort of citation of the DuraMAT DataHub source may be needed. See the "publication and non-endorsement" section here https://datahub.duramat.org/terms. Maybe it's already included and I just missed it.

On the dataset currently included in this PR, I think some sort of citation of the DuraMAT DataHub source may be needed. See the "publication and non-endorsement" section here https://datahub.duramat.org/terms. Maybe it's already included and I just missed it.

This is a detail I'm not sure about given that I created the larger data set also (on behalf of and using data from Sandia).  But I'm sure by the end of this code review it will be acknowledged as it should be!

So now we can get the discussion back to the essence of this PR: faiman_dyn and fit_faiman_dyn!

This PR is currently blocked.  See #1898.


Bump-up Torch to version 2.5.1 from 2.4.1

Description
Bump-up Torch to version 2.5.1 from 2.4.1
Type of change

[ ] Refactor
[ ] New feature
[ ] Bug fix
[ ] CVE fix
[ ] Optimization
[ ] Documentation Update
[ ] Configuration Update
[x] Bump-up dependent library
[ ] Bump-up library or tool used for development (does not change the final image)
[ ] CI configuration change
[ ] Konflux configuration change

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 93.48%. Comparing base (010acff) to head (ffb9a46).
Report is 13 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1960      +/-   ##
==========================================
- Coverage   96.84%   93.48%   -3.37%
==========================================
  Files          69       69
  Lines        2883     2884       +1
==========================================
- Hits         2792     2696      -96
- Misses         91      188      +97

see 8 files with indirect coverage changes

/hold

/test 4.15-e2e-ols-cluster
/test 4.17-e2e-ols-cluster

/override "Red Hat Konflux / ols-enterprise-contract / lightspeed-service"

/unhold

/approve

/override "ci/prow/4.16-e2e-ols-cluster"


confidence score

Hi, how can I find the confidence score or how can I calculate it?
Thank you.

Hi, the code below contains the classification score (i.e., likelihood of target object being present) for every spatial location in the search region. These classification scores can be interpreted as confidence scores, and they are in the range 0 and 1.
https://github.com/goutamyg/SMAT/blob/main/lib/test/tracker/mobilevitv2_track.py#L107

Thank you very much.
@goutamyg

We will release the TensorRT and ONNX-based inference scripts by the end of this month

Thank you very much.
@goutamyg


Release 0.2.0

[ ]  Update readme: Small and Pragmatic and Flexible async http server. Fast to compile.
[ ] update crates description to above
[ ] changelog
[x] remove clippy from stable ci

701e0a94df280db63443a7420ada275a1c3cdcbf remove clippy from stable

414e19be8e12e83a2611582763d84f65877e8464 updates descriptions.
f16c59c6c2d231b10659e2368fd3d6fb36da06e2 basic docs, some cleanup on unused pub types and remove unwrap macros.

a53e0efc1a56806b94d2f50f931c1da20606c021

bd1528191ce3f6a994000b3216a651671b35a739 v0.2.0


Fix serialization of enum values to Lionweb JSON (was throwing CCE)

I am afraid this would be covered by a PR on LionWeb Java (https://github.com/LionWeb-io/lionweb-java/pull/137) and work I am doing in branch https://github.com/Strumenta/kolasu/tree/bugfix/enumDeserialization .
I would wait for the PR on LionWeb Java, then verify tests still pass and combine this with my PR.
Sorry for overlapping work

Ok cool!


Cannot import cell2location: Failed in nopython mode pipeline (step: native lowering)

[x] I have confirmed this bug exists on the latest version of cell2location. See https://github.com/BayraktarLab/cell2location#installation
[x] I follow the instructions from the scvi-tools tutorial.

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.
Minimal code sample (that we can run without your data, using public data)
This issue came up quite suddenly and it worked normally before.  No library-related changes have been made.
# Your code here
import cell2location

[Paste the error output produced by the above code here]
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/errors.py:823, in new_error_context(fmt_, *args, **kwargs)
    822 try:
--> 823     yield
    824 except NumbaError as e:

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/lowering.py:293, in BaseLower.lower_block(self, block)
    291     with new_error_context('lowering "{inst}" at {loc}', inst=inst,
    292                            loc=self.loc, errcls_=defaulterrcls):
--> 293         self.lower_inst(inst)
    294 self.post_block(block)

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/lowering.py:438, in Lower.lower_inst(self, inst)
    437 ty = self.typeof(inst.target.name)
--> 438 val = self.lower_assign(ty, inst)
    439 argidx = None

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/lowering.py:624, in Lower.lower_assign(self, ty, inst)
    623 elif isinstance(value, ir.Expr):
--> 624     return self.lower_expr(ty, value)
    626 elif isinstance(value, ir.Var):

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/lowering.py:1159, in Lower.lower_expr(self, resty, expr)
   1158 elif expr.op == 'call':
-> 1159     res = self.lower_call(resty, expr)
   1160     return res

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/lowering.py:889, in Lower.lower_call(self, resty, expr)
    888 else:
--> 889     res = self._lower_call_normal(fnty, expr, signature)
    891 # If lowering the call returned None, interpret that as returning dummy
    892 # value if the return type of the function is void, otherwise there is
    893 # a problem

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/lowering.py:1130, in Lower._lower_call_normal(self, fnty, expr, signature)
   1128     argvals = [the_self] + list(argvals)
-> 1130 res = impl(self.builder, argvals, self.loc)
   1131 return res

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/base.py:1201, in _wrap_impl.__call__(self, builder, args, loc)
   1200 def __call__(self, builder, args, loc=None):
-> 1201     res = self._imp(self._context, builder, self._sig, args, loc=loc)
   1202     self._context.add_linking_libs(getattr(self, 'libs', ()))

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/base.py:1231, in _wrap_missing_loc.__call__.<locals>.wrapper(*args, **kwargs)
   1230 kwargs.pop('loc')     # drop unused loc
-> 1231 return fn(*args, **kwargs)

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/cpython/rangeobj.py:40, in make_range_impl.<locals>.range1_impl(context, builder, sig, args)
     39 state.start = context.get_constant(int_type, 0)
---> 40 state.stop = stop
     41 state.step = context.get_constant(int_type, 1)

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/cgutils.py:164, in _StructProxy.__setattr__(self, field, value)
    163     return super(_StructProxy, self).__setattr__(field, value)
--> 164 self[self._datamodel.get_field_position(field)] = value

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/cgutils.py:188, in _StructProxy.__setitem__(self, index, value)
    187     else:
--> 188         raise TypeError("Invalid store of {value.type} to "
    189                         "{ptr.type.pointee} in "
    190                         "{self._datamodel} "
    191                         "(trying to write member #{index})"
    192                         .format(value=value, ptr=ptr, self=self,
    193                                 index=index))
    194 self._builder.store(value, ptr)

TypeError: Invalid store of i64 to i32 in <numba.core.datamodel.models.RangeModel object at 0x7fd85018ed30> (trying to write member #1)

During handling of the above exception, another exception occurred:

LoweringError                             Traceback (most recent call last)
Input In [2], in <cell line: 1>()
----> 1 import cell2location

File /data/xy/anaconda3/lib/python3.8/site-packages/cell2location/__init__.py:5, in <module>
      2 from pyro.distributions.transforms import SoftplusTransform
      3 from torch.distributions import biject_to, transform_to
----> 5 from .run_c2l import run_cell2location
      6 from .run_colocation import run_colocation
      7 from .run_regression import run_regression

File /data/xy/anaconda3/lib/python3.8/site-packages/cell2location/run_c2l.py:18, in <module>
     15 import scipy
     16 import theano
---> 18 import cell2location.models as models
     19 import cell2location.plt as c2lpl
     20 from cell2location.cluster_averages import compute_cluster_averages, select_features

File /data/xy/anaconda3/lib/python3.8/site-packages/cell2location/models/__init__.py:1, in <module>
----> 1 from ._cell2location_model import Cell2location
      2 from ._cell2location_module import (
      3     LocationModelLinearDependentWMultiExperimentLocationBackgroundNormLevelGeneAlphaPyroModel,
      4 )
      5 from .downstream import CoLocatedGroupsSklearnNMF

File /data/xy/anaconda3/lib/python3.8/site-packages/cell2location/models/_cell2location_model.py:21, in <module>
     19 from cell2location.models.base._pyro_base_loc_module import Cell2locationBaseModule
     20 from cell2location.models.base._pyro_mixin import PltExportMixin, QuantileMixin
---> 21 from cell2location.utils import select_slide
     24 class Cell2location(QuantileMixin, PyroSampleMixin, PyroSviTrainMixin, PltExportMixin, BaseModelClass):
     25     r"""
     26     Cell2location model. User-end model class. See Module class for description of the model (incl. math).
     27
   (...)
     42     >>>
     43     """

File /data/xy/anaconda3/lib/python3.8/site-packages/cell2location/utils/__init__.py:5, in <module>
      1 import sys
      3 import numpy as np
----> 5 from ._spatial_knn import spatial_neighbours, sum_neighbours
      6 from .filtering import filter_genes
      9 def select_slide(adata, s, batch_key="sample"):

File /data/xy/anaconda3/lib/python3.8/site-packages/cell2location/utils/_spatial_knn.py:5, in <module>
      3 from scipy.spatial import cKDTree
      4 from sklearn.neighbors import KDTree
----> 5 from umap.umap_ import fuzzy_simplicial_set
      8 def get_sparse_matrix_from_indices_distances_umap(knn_indices, knn_dists, n_obs, n_neighbors):
      9     """
     10     Copied out of scanpy.neighbors
     11     """

File /data/xy/anaconda3/lib/python3.8/site-packages/umap/__init__.py:1, in <module>
----> 1 from .umap_ import UMAP
      3 # Workaround: https://github.com/numba/numba/issues/3341
      4 import numba

File /data/xy/anaconda3/lib/python3.8/site-packages/umap/umap_.py:54, in <module>
     52 from umap.spectral import spectral_layout
     53 from umap.utils import deheap_sort, submatrix
---> 54 from umap.layouts import (
     55     optimize_layout_euclidean,
     56     optimize_layout_generic,
     57     optimize_layout_inverse,
     58 )
     60 try:
     61     # Use pynndescent, if installed (python 3 only)
     62     from pynndescent import NNDescent

File /data/xy/anaconda3/lib/python3.8/site-packages/umap/layouts.py:39, in <module>
     25     else:
     26         return val
     29 @numba.njit(
     30     "f4(f4[::1],f4[::1])",
     31     fastmath=True,
     32     cache=True,
     33     locals={
     34         "result": numba.types.float32,
     35         "diff": numba.types.float32,
     36         "dim": numba.types.int32,
     37     },
     38 )
---> 39 def rdist(x, y):
     40     """Reduced Euclidean distance.
     41
     42     Parameters
   (...)
     49     The squared euclidean distance between x and y
     50     """
     51     result = 0.0

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/decorators.py:219, in _jit.<locals>.wrapper(func)
    217     with typeinfer.register_dispatcher(disp):
    218         for sig in sigs:
--> 219             disp.compile(sig)
    220         disp.disable_compile()
    221 return disp

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/dispatcher.py:965, in Dispatcher.compile(self, sig)
    963 with ev.trigger_event("numba:compile", data=ev_details):
    964     try:
--> 965         cres = self._compiler.compile(args, return_type)
    966     except errors.ForceLiteralArg as e:
    967         def folded(args, kws):

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/dispatcher.py:125, in _FunctionCompiler.compile(self, args, return_type)
    124 def compile(self, args, return_type):
--> 125     status, retval = self._compile_cached(args, return_type)
    126     if status:
    127         return retval

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/dispatcher.py:139, in _FunctionCompiler._compile_cached(self, args, return_type)
    136     pass
    138 try:
--> 139     retval = self._compile_core(args, return_type)
    140 except errors.TypingError as e:
    141     self._failed_cache[key] = e

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/dispatcher.py:152, in _FunctionCompiler._compile_core(self, args, return_type)
    149 flags = self._customize_flags(flags)
    151 impl = self._get_implementation(args, {})
--> 152 cres = compiler.compile_extra(self.targetdescr.typing_context,
    153                               self.targetdescr.target_context,
    154                               impl,
    155                               args=args, return_type=return_type,
    156                               flags=flags, locals=self.locals,
    157                               pipeline_class=self.pipeline_class)
    158 # Check typing error if object mode is used
    159 if cres.typing_error is not None and not flags.enable_pyobject:

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/compiler.py:693, in compile_extra(typingctx, targetctx, func, args, return_type, flags, locals, library, pipeline_class)
    669 """Compiler entry point
    670
    671 Parameter
   (...)
    689     compiler pipeline
    690 """
    691 pipeline = pipeline_class(typingctx, targetctx, library,
    692                           args, return_type, flags, locals)
--> 693 return pipeline.compile_extra(func)

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/compiler.py:429, in CompilerBase.compile_extra(self, func)
    427 self.state.lifted = ()
    428 self.state.lifted_from = None
--> 429 return self._compile_bytecode()

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/compiler.py:497, in CompilerBase._compile_bytecode(self)
    493 """
    494 Populate and run pipeline for bytecode input
    495 """
    496 assert self.state.func_ir is None
--> 497 return self._compile_core()

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/compiler.py:476, in CompilerBase._compile_core(self)
    474         self.state.status.fail_reason = e
    475         if is_final_pipeline:
--> 476             raise e
    477 else:
    478     raise CompilerError("All available pipelines exhausted")

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/compiler.py:463, in CompilerBase._compile_core(self)
    461 res = None
    462 try:
--> 463     pm.run(self.state)
    464     if self.state.cr is not None:
    465         break

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/compiler_machinery.py:353, in PassManager.run(self, state)
    350 msg = "Failed in %s mode pipeline (step: %s)" % \
    351     (self.pipeline_name, pass_desc)
    352 patched_exception = self._patch_error(msg, e)
--> 353 raise patched_exception

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/compiler_machinery.py:341, in PassManager.run(self, state)
    339 pass_inst = _pass_registry.get(pss).pass_inst
    340 if isinstance(pass_inst, CompilerPass):
--> 341     self._runPass(idx, pass_inst, state)
    342 else:
    343     raise BaseException("Legacy pass in use")

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/compiler_lock.py:35, in _CompilerLock.__call__.<locals>._acquire_compile_lock(*args, **kwargs)
     32 @functools.wraps(func)
     33 def _acquire_compile_lock(*args, **kwargs):
     34     with self:
---> 35         return func(*args, **kwargs)

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/compiler_machinery.py:296, in PassManager._runPass(self, index, pss, internal_state)
    294     mutated |= check(pss.run_initialization, internal_state)
    295 with SimpleTimer() as pass_time:
--> 296     mutated |= check(pss.run_pass, internal_state)
    297 with SimpleTimer() as finalize_time:
    298     mutated |= check(pss.run_finalizer, internal_state)

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/compiler_machinery.py:269, in PassManager._runPass.<locals>.check(func, compiler_state)
    268 def check(func, compiler_state):
--> 269     mangled = func(compiler_state)
    270     if mangled not in (True, False):
    271         msg = ("CompilerPass implementations should return True/False. "
    272                "CompilerPass with name '%s' did not.")

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/typed_passes.py:394, in NativeLowering.run_pass(self, state)
    391 with targetctx.push_code_library(library):
    392     lower = lowering.Lower(targetctx, library, fndesc, interp,
    393                            metadata=metadata)
--> 394     lower.lower()
    395     if not flags.no_cpython_wrapper:
    396         lower.create_cpython_wrapper(flags.release_gil)

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/lowering.py:196, in BaseLower.lower(self)
    194 if self.generator_info is None:
    195     self.genlower = None
--> 196     self.lower_normal_function(self.fndesc)
    197 else:
    198     self.genlower = self.GeneratorLower(self)

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/lowering.py:250, in BaseLower.lower_normal_function(self, fndesc)
    248 # Init argument values
    249 self.extract_function_arguments()
--> 250 entry_block_tail = self.lower_function_body()
    252 # Close tail of entry block, do not emit debug metadata else the
    253 # unconditional jump gets associated with the metadata from the function
    254 # body end.
    255 with debuginfo.suspend_emission(self.builder):

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/lowering.py:279, in BaseLower.lower_function_body(self)
    277     bb = self.blkmap[offset]
    278     self.builder.position_at_end(bb)
--> 279     self.lower_block(block)
    280 self.post_lower()
    281 return entry_block_tail

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/lowering.py:293, in BaseLower.lower_block(self, block)
    290     defaulterrcls = partial(LoweringError, loc=self.loc)
    291     with new_error_context('lowering "{inst}" at {loc}', inst=inst,
    292                            loc=self.loc, errcls_=defaulterrcls):
--> 293         self.lower_inst(inst)
    294 self.post_block(block)

File /data/xy/anaconda3/lib/python3.8/contextlib.py:131, in _GeneratorContextManager.__exit__(self, type, value, traceback)
    129     value = type()
    130 try:
--> 131     self.gen.throw(type, value, traceback)
    132 except StopIteration as exc:
    133     # Suppress StopIteration *unless* it's the same exception that
    134     # was passed to throw().  This prevents a StopIteration
    135     # raised inside the "with" statement from being suppressed.
    136     return exc is not value

File /data/xy/anaconda3/lib/python3.8/site-packages/numba/core/errors.py:837, in new_error_context(fmt_, *args, **kwargs)
    835     else:
    836         tb = None
--> 837     raise newerr.with_traceback(tb)
    838 elif use_new_style_errors():
    839     raise e

LoweringError: Failed in nopython mode pipeline (step: native lowering)
Invalid store of i64 to i32 in <numba.core.datamodel.models.RangeModel object at 0x7fd85018ed30> (trying to write member #1)

File "../../../../anaconda3/lib/python3.8/site-packages/umap/layouts.py", line 53:
def rdist(x, y):
    <source elided>
    dim = x.shape[0]
    for i in range(dim):
    ^

During: lowering "$20call_function.7 = call $16load_global.5(dim, func=$16load_global.5, args=[Var(dim, layouts.py:52)], kws=(), vararg=None, target=None)" at /data/xy/anaconda3/lib/python3.8/site-packages/umap/layouts.py (53)

Hi @98xy
We have just updated cell2location to the latest scvi-tools version (0.15.3+). Please try re-creating a clean conda environment. The error you observe looks like umap installation version issues. Given that cell2location passes GitHub tests, the problem with UMAP can be specific to your platform.

Hi @98xy
We have just updated cell2location to the latest scvi-tools version (0.15.3+). Please try re-creating a clean conda environment. The error you observe looks like umap installation version issues. Given that cell2location passes GitHub tests, the problem with UMAP can be specific to your platform.

Hi @vitkl
Thanks for your help! I have re-created a clean conda environment but the error is still exist.

Hi again
This error comes from package version mismatch. Maybe your conda
environments don't work correctly.
Do you use export PYTHONNOUSERSITE="someletters" as described in the
installation section? Maybe you need to add this every time you open
terminal, before activating conda environment, before creating conda
environment. I have to do that our compute cluster.
On Sun, 17 Apr 2022, 09:38 Yin Xu, @.***> wrote:

Hi @98xy https://github.com/98xy
We have just updated cell2location to the latest scvi-tools version
(0.15.3+). Please try re-creating a clean conda environment. The error you
observe looks like umap installation version issues. Given that
cell2location passes GitHub tests, the problem with UMAP can be specific to
your platform.
Hi @vitkl https://github.com/vitkl
Thanks for your help! I have re-created a clean conda environment but the
error is still exist.
—
Reply to this email directly, view it on GitHub
https://github.com/BayraktarLab/cell2location/issues/142#issuecomment-1100832168,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFMFTV6WUE3Y4ZT7FEGFTKLVFPEW5ANCNFSM5S4NY2DQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

Hi again:
Thanks very much for your help! After export PYTHONNOUSERSITE="someletters", this error removed.
And sorry to disturb you again, another mistake comes now… Could you give me some advice for solving it? Thanks sincerely again!
AttributeError: module 'jaxlib.xla_extension' has no attribute 'CpuDevice'
details :
--------------------------------------------------------------------------- AttributeError                            Traceback (most recent call last) Input In [2], in <cell line: 14>()      11 import matplotlib.pyplot as plt      12 import matplotlib as mpl ---> 14 import cell2location      15 import scvi      17 from matplotlib import rcParams File /data/xy/anaconda3/envs/cell2loc_env/lib/python3.9/site-packages/cell2location/init.py:5, in <module>       2 from pyro.distributions.transforms import SoftplusTransform       3 from torch.distributions import biject_to, transform_to ----> 5 from .run_c2l import run_cell2location       6 from .run_colocation import run_colocation       7 from .run_regression import run_regression File /data/xy/anaconda3/envs/cell2loc_env/lib/python3.9/site-packages/cell2location/run_c2l.py:18, in <module>      15 import scipy      16 import theano ---> 18 import cell2location.models as models      19 import cell2location.plt as c2lpl      20 from cell2location.cluster_averages import compute_cluster_averages, select_features File /data/xy/anaconda3/envs/cell2loc_env/lib/python3.9/site-packages/cell2location/models/init.py:1, in <module> ----> 1 from ._cell2location_model import Cell2location       2 from ._cell2location_module import (       3     LocationModelLinearDependentWMultiExperimentLocationBackgroundNormLevelGeneAlphaPyroModel,       4 )       5 from .downstream import CoLocatedGroupsSklearnNMF File /data/xy/anaconda3/envs/cell2loc_env/lib/python3.9/site-packages/cell2location/models/_cell2location_model.py:10, in <module>       8 from pyro import clear_param_store       9 from pyro.nn import PyroModule ---> 10 from scvi import REGISTRY_KEYS      11 from scvi.data import AnnDataManager      12 from scvi.data.fields import (      13     CategoricalJointObsField,      14     CategoricalObsField,    (...)      17     NumericalObsField,      18 ) File /data/xy/anaconda3/envs/cell2loc_env/lib/python3.9/site-packages/scvi/init.py:10, in <module>       7 from ._settings import settings       9 # this import needs to come after prior imports to prevent circular import ---> 10 from . import data, model, external, utils      12 # https://github.com/python-poetry/poetry/pull/2366#issuecomment-652418094      13 # https://github.com/python-poetry/poetry/issues/144#issuecomment-623927302      14 try: File /data/xy/anaconda3/envs/cell2loc_env/lib/python3.9/site-packages/scvi/model/init.py:6, in <module>       4 from ._condscvi import CondSCVI       5 from ._destvi import DestVI ----> 6 from ._jaxscvi import JaxSCVI       7 from ._linear_scvi import LinearSCVI       8 from ._multivi import MULTIVI File /data/xy/anaconda3/envs/cell2loc_env/lib/python3.9/site-packages/scvi/model/_jaxscvi.py:7, in <module>       5 import jax.numpy as jnp       6 import numpy as np ----> 7 import optax       8 import pandas as pd       9 import tqdm File /data/xy/anaconda3/envs/cell2loc_env/lib/python3.9/site-packages/optax/init.py:17, in <module>       1 # Copyright 2019 DeepMind Technologies Limited. All Rights Reserved.       2 #       3 # Licensed under the Apache License, Version 2.0 (the "License");    (...)      13 # limitations under the License.      14 # ==============================================================================      15 """Optax: composable gradient processing and optimization, in JAX.""" ---> 17 from optax import experimental      18 from optax._src.alias import adabelief      19 from optax._src.alias import adafactor File /data/xy/anaconda3/envs/cell2loc_env/lib/python3.9/site-packages/optax/experimental/init.py:20, in <module>       1 # Copyright 2021 DeepMind Technologies Limited. All Rights Reserved.       2 #       3 # Licensed under the Apache License, Version 2.0 (the "License");    (...)      13 # limitations under the License.      14 # ==============================================================================      15 """Experimental features in Optax.      16       17 Features may be removed or modified at any time.      18 """ ---> 20 from optax._src.experimental.complex_valued import split_real_and_imaginary      21 from optax._src.experimental.complex_valued import SplitRealAndImaginaryState File /data/xy/anaconda3/envs/cell2loc_env/lib/python3.9/site-packages/optax/_src/experimental/complex_valued.py:32, in <module>      15 """Complex-valued optimization.      16       17 When using split_real_and_imaginary to wrap an optimizer, we split the complex    (...)      27 See details at https://github.com/deepmind/optax/issues/196      28 """      30 from typing import NamedTuple, Union ---> 32 import chex      33 import jax      34 import jax.numpy as jnp File /data/xy/anaconda3/envs/cell2loc_env/lib/python3.9/site-packages/chex/init.py:17, in <module>       1 # Copyright 2020 DeepMind Technologies Limited. All Rights Reserved.       2 #       3 # Licensed under the Apache License, Version 2.0 (the "License");    (...)      13 # limitations under the License.      14 # ==============================================================================      15 """Chex: Testing made fun, in JAX!""" ---> 17 from chex._src.asserts import assert_axis_dimension      18 from chex._src.asserts import assert_axis_dimension_comparator      19 from chex._src.asserts import assert_axis_dimension_gt File /data/xy/anaconda3/envs/cell2loc_env/lib/python3.9/site-packages/chex/_src/asserts.py:26, in <module>      23 import unittest      24 from unittest import mock ---> 26 from chex._src import asserts_internal as _ai      27 from chex._src import pytypes      28 import jax File /data/xy/anaconda3/envs/cell2loc_env/lib/python3.9/site-packages/chex/_src/asserts_internal.py:32, in <module>      29 from typing import Any, Sequence, Union, Callable, Optional, Set, Tuple, Type      31 from absl import logging ---> 32 from chex._src import pytypes      33 import jax      34 import jax.numpy as jnp File /data/xy/anaconda3/envs/cell2loc_env/lib/python3.9/site-packages/chex/_src/pytypes.py:40, in <module>      37 PyTreeDef = type(jax.tree_structure(None))      38 Shape = Tuple[int, ...] ---> 40 CpuDevice = jax.lib.xla_extension.CpuDevice      41 GpuDevice = jax.lib.xla_extension.GpuDevice      42 TpuDevice = jax.lib.xla_extension.TpuDevice AttributeError: module 'jaxlib.xla_extension' has no attribute 'CpuDevice'
------------------ 原始邮件 ------------------
发件人:                                                                                "BayraktarLab/cell2location"                                                                                @.>;
发送时间: 2022年4月17日(星期天) 下午5:47
@.>;
<EMAIL_ADDRESS>主题: Re: [BayraktarLab/cell2location] Cannot import cell2location: Failed in nopython mode pipeline (step: native lowering) (Issue #142)
Hi again
This error comes from package version mismatch. Maybe your conda
environments don't work correctly.
Do you use export PYTHONNOUSERSITE="someletters" as described in the
installation section? Maybe you need to add this every time you open
terminal, before activating conda environment, before creating conda
environment. I have to do that our compute cluster.
On Sun, 17 Apr 2022, 09:38 Yin Xu, @.***> wrote:
> Hi @98xy <https://github.com/98xy>
>
> We have just updated cell2location to the latest scvi-tools version
> (0.15.3+). Please try re-creating a clean conda environment. The error you
> observe looks like umap installation version issues. Given that
> cell2location passes GitHub tests, the problem with UMAP can be specific to
> your platform.
>
> Hi @vitkl <https://github.com/vitkl>
> Thanks for your help! I have re-created a clean conda environment but the
> error is still exist.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/BayraktarLab/cell2location/issues/142#issuecomment-1100832168>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AFMFTV6WUE3Y4ZT7FEGFTKLVFPEW5ANCNFSM5S4NY2DQ>
> .
> You are receiving this because you were mentioned.Message ID:
> @.***>
>
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>

This is a temporary error with JAX (scvi-tools dependency) - as discussed here: https://github.com/BayraktarLab/cell2location/pull/144
I think you can solve this by installing oldex jax and jaxlib:
pip install jax>=0.3,<=0.3.6 jaxlib>=0.1.65,<=0.3.5

Thanks sincerely again for your help! The error has been solved now :)
------------------ 原始邮件 ------------------
发件人:                                                                                "BayraktarLab/cell2location"                                                                                @.>;
发送时间: 2022年4月17日(星期天) 晚上6:38
@.>;
<EMAIL_ADDRESS>主题: Re: [BayraktarLab/cell2location] Cannot import cell2location: Failed in nopython mode pipeline (step: native lowering) (Issue #142)
This is a temporary error with JAX (scvi-tools dependency) - as discussed here: #144
I think you can solve this by installing oldex jax and jaxlib:
pip install jax>=0.3,<=0.3.6 jaxlib>=0.1.65,<=0.3.5
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>

This does not seem to be a problem now as well. If you see similar issues with JAX in the future - could be good to try installing the previous version or wait for a few days until the bug is fixed.


bump runc to c6e4a1ebeb1a72b529c6f1b6ee2b1ae5b868b14f

Signed-off-by: Antonio Murdaca<EMAIL_ADDRESS>
- What I did
bump runc to c6e4a1ebeb1a72b529c6f1b6ee2b1ae5b868b14f so it contains https://github.com/opencontainers/runc/pull/1651 which is a fix to a weird systemd bug which we're hitting in e2e_rhel
- How I did it
vim
- How to verify it
- Description for the changelog

@mrunalp PTAL

/test all
/test ami

The next ami built daily will pick this change so no reason to tag the ami after this

@mrunalp @rhatdan PTAL

LGTM. will wait for that last test to pass.


Drop CI docs build?

The original purpose of the CI docs build was to ensure that ReadTheDocs would be able to build. I've enabled ReadTheDocs PR builds.
Okay to remove the Circle CI jobs? Or is there a good reason to keep them?

I guess that's fine. 👍

Offloading whatever we can from Circle is fine by me


Version 2 ideas of what ml can do

Great work with these final updates. This is ready to merge.
I noticed the algorithm name parsing didn't work in the EGFR dataset because the dataset is called tps-egfr. I fixed that in f4f8ad1, but we could have future problems if an algorithm name ever contains -. Before the correction, the EGFR plots looked like this.


Provide APIs to help application developers test their Histogram-using code

I have an application that uses Histogram.  I want to write some tests that assert that my application is putting values into the right buckets of that histogram.
To read values out of the Histogram it seems that need to either:

look at the Histogram's private _buckets attribute, or
call Histogram.collectand look at the resulting Metric objects which are documented as "intended only for internal use" and then look at its Samples which aren't documented at all.

Neither of these seems particularly satisfactory.  It would be very helpful if client_python provided some way to read bucket counts and the total sum from a Histogram, using documented APIs clearly marked as intended for external use (perhaps only in test suites).

I think using the Metric objects would probably be fine for this case, I believe that documentation for internal use is for creating custom collectors you should use the more specifically typed collectors instead.
That said, I think adding some testing utilities would be nice. The Go client has a package of test utilities that could be a good reference<EMAIL_ADDRESS>


Allow users to inspect a cell or feature to read more information

stac ipyleaflet version: for release 0.3.0
Python version: python >= 3.9 (some of the widgets are incompatible at v 3.8)
Reference column item 5 in the MAAP-ipyleaflet-DevRoadmap

Context
After a user adds layers to display data on a map, they might want to inspect that data to learn more about:

a cell - the smallest unit within a raster dataset - each with a specific dimension (n*n resolution), and typically representing a single value (unless the value is cleverly encoded)
a feature - a single point, line or polygon within that dataset - it can carry a lot of attributes, that typically means they are assigned to that feature (i.e. one lake within a dataset of lakes in N America)

Problem
Currently users can pan around the map and zoom in/out, but they cannot select a cell or feature to learn more. This doesn't provide enough information for them.
Ideas

New widget: inspection/info widget

User adds data --> User selects a cell --> User opens inspect widget --> See results
User adds data --> User opens inspect widget --> User selects a cell --> See results

Adding to Drawing widget: Could reframe the drawing one to include more info?

User adds data --> User opens draw widget --> User selects a cell or draws box --> See results
User adds data --> User selects a cell --> User opens draw widget (would need to rename) --> See results

Here's an example of an app that shows highlighting to identify a cell.

Usually the inspection tool is associated with a single cell or point location.. and it's assumed that you've either selected a single layer you want results for, or that we would query all visible datasets and present all of those results
Example where you click and everything is presented to you: https://gis-cityofkennesaw.hub.arcgis.com/apps/5db59e85c2fc4a1c80b73df0ef525c83/explore

Example where you select which layer you are identifying (the i for identify tool is activated): https://maps.durhamnc.gov/gomaps/

Toolbar example derived from QGIS & other desktop GIS applications (this became the default in many web maps but can be overkill for non-GIS users)

Acceptance Criteria

[ ] Design mockup as reference
[ ] Implementation of a way for users to inspect data based on a single point or cell selection

Separate info (i) widget could be added as a button above the map, alongside other widget buttons.

an option for an inspect widget as a way for users to indicate that they'd like to activate selection mode. I don't know what data about the cell we have to show but just some early thoughts here.  https://www.figma.com/file/2gezwwtJdUoBc5SD2pB86W/stac_ipyleaflet---Release-0.2?type=design&node-id=28-6811&mode=design&t=uQzNYeWOAZO5nGNB-4

Addressing the multiple layers issue:

First get it using for the most top layer and then possibly create a fast follow issue to grab data for all the layers
If not to much of a push to tackle in this issue, go ahead and tackle
https://developmentseed.slack.com/archives/C05FRGCA12Q/p1690206719180059?thread_ts=1690204309.207899&cid=C05FRGCA12Q

UI Update July 27, 2023

After meeting with @emmalu @sandrahoang686 @wildintellect, I'm showcasing the latest design mockups as reference. The mockup here is only intended to address the immediate issue which is allowing users to select a single point (not area) on the map in order to see a corresponding value for all selected layers.

Overall changes to the DRAW widget

Rename the draw widget to Interact - other ideas were coordinates, explore, design, inspect...this can be debated so I'm just recommended interact for now.
This widget includes two features: Select Point (new default - this issue) + Select Area (existing Draw widget)
In future users may want to also see charts and other analysis so we're creating a bit of room to scale through a widget, but the UI organization within the widget could be improved in the future.

Select Point

Upon selecting the point button, the map panning becomes a pointer and the users are prompted with a text message to Select a point feature on the map to view its values
All of the currently selected layers are also displayed as a list with greyed out values
 Direct link to Figma
Users can then select any point on the map. Though they are clicking on the top layer, there could be multiple layers with different opacities displayed so we'd want the location regardless of the layer.
Based on the point, the values for each selected layer will be displayed
If a user selects a new point, the values will change

Select Area (Existing Draw widget)

The current draw button is now renamed to be Area
If users click on the Area button, they are able to draw a box (area of interest) on the map
Once they draw an area of interest, they can see the coordinates displayed as text

Making a note that we still need to include info from an identified vector feature within the Interact widget. Adding a vector layer can be done by following either the geopandas or geojson sample notebook .
cc @wildintellect @sandrahoang686 @heidimok


Error in count of TodoMVC example

I'd love to send a PR for this one since I'm just getting my feet wet but it might take me a bit so I'll defer to people smarter :)
To reproduce:

Load TodoMVC example
Add a few items to your todo list. Note that the counter properly increments (mine is at 4)
Mark one item as complete. Note that the counter properly decrements (mine is at 3)
Toggle the view to show only completed tasks. Note the counter still correctly shows the number of items left on your list. (mine is at 3)
Click "clear completed." You should still have the correct number of items displaying. (mine is at 3)
Go back to the "all" view. Add a new item to the list.

Expected result:

The list should grow in length with the item I just added at the top. The counter of "items left" should increment by one

Actual result

My item is added to the top but the last item on the list disappears. The counter of "items left" has incremented by one but only three items are visible.
Adding new items after this will grow the list as expected but the Todo that disappeared will remain gone. The counter of "items left" still accounts for the missing Todo.

There's some additional funk on the counter that occurs if you remove an item from the list while it's in this incremented state. I can file more of a bug report if needed but I have a hunch that resolving this problem will resolve the latter issue.

This is happening because the newly added task has the same id as the previous task. This is because the todos reducer is simply assigning state.length as id when a new task is added.
I think the solution is to keep some increment-only part (like nextId) in the state or just use big UUIDs.

Thanks a lot! Should be fixed in 1.0.1.


Use GitHub actions for continuous integration (CI)

Bye bye Travis CI, you've served us well.

Duplicate of #58


Added preview images

Description
I added preview image in some projects like quote generator,Bubble Burst Game and Lorem Ipsum Generator
Before

After

Fixes # (62)

Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented on my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules

@priyakumari02

@priyakumari02

I deleted

@priyakumari02 do this asap

@priyakumari02 also delete the images used in dice challenge

Sorry
and I am unable to delete the images , it is not showing any option of delete file in dice images. How should I do it?

Sorry and I am unable to delete the images , it is not showing any option of delete file in dice images. How should I do it?

make a new PR and make small small changes