gradient computation fails in graph mode with numpy_function

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary): binary (conda-forge)
TensorFlow version (use command below): 'v2.3.0-rc2-23-gb36436b087', 'v2.4.0-49-g85c8b2a817f'
Python version: 3.7.7
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version: 10.1
GPU model and memory: Titan RTX 24GB

Describe the current behavior
I'm trying to use a function implemented in cython in a loss function. The (int64) output of this function is used as index for tf.gather. When computing the loss in eager execution mode gradients are computed fine, but when run in graph mode, an exception is raised.
Describe the expected behavior
I would expect the same result in graph execution mode.
Standalone code to reproduce the issue
A tiny sample where the cython function simply implements argmin:
import tensorflow as tf
import numpy as np
import assign

model = tf.keras.applications.ResNet50(False, weights=None, input_shape=(224, 224, 3))
data = tf.random.normal((8,224,224,3), 128, 64)

def f(x):
	y = model(x)
	idx = tf.numpy_function(assign.assign_objects_to_cells, (-y,), np.int64)
	#idx = tf.argmin(-y, -1)
	#idx = tf.numpy_function(np.argmin, (-y, -1), np.int64)
	values = tf.gather(y, idx, batch_dims=3)
	return tf.reduce_sum(values)

def grad(x):
	with tf.GradientTape() as tape:
		y = f(x)
	return tape.gradient(y, model.trainable_variables)
tf_grad = tf.function(grad)

print('eager execution')
g = grad(data)
print('graph execution')
g = tf_grad(data)

And the assign.pyx:
import numpy as np
cimport numpy as np
cimport cython

@cython.boundscheck(False) # turn off bounds-checking for entire function
@cython.wraparound(False)  # turn off negative index wrapping for entire function
cdef void _assign_objects_to_cells(float[:,:,:,:] error, np.int64_t[:,:,:] argmax) nogil:
	cdef int N = error.shape[0], H = error.shape[1], W = error.shape[2], C = error.shape[3]
	cdef int y, x, c, best_id
	cdef float best, e

	for n in range(N):
		for y in range(H):
			for x in range(W):
				best_id = 0
				best = error[n,y,x,0]
				for c in range(1, C):
					e = error[n,y,x,c]
					if e<best:
						best = e
						best_id = c
				argmax[n,y,x] = best_id

def assign_objects_to_cells(np.ndarray[np.float32_t, ndim=4] error not None):
	cdef int N = error.shape[0], H = error.shape[1], W = error.shape[2], C = error.shape[3]
	out = np.empty((N,H,W), np.int64)
	_assign_objects_to_cells(error, out)

	return out

Other info / logs
Eager execution runs fine, but graph execution fails:
eager execution
graph execution
Traceback (most recent call last):

  File "<ipython-input-1-eafa53e19dc9>", line 25, in <module>
    g = tf_grad(data)

  File "d:\conda\envs\tf2\lib\site-packages\tensorflow\python\eager\def_function.py", line 780, in __call__
    result = self._call(*args, **kwds)

  File "d:\conda\envs\tf2\lib\site-packages\tensorflow\python\eager\def_function.py", line 823, in _call
    self._initialize(args, kwds, add_initializers_to=initializers)

  File "d:\conda\envs\tf2\lib\site-packages\tensorflow\python\eager\def_function.py", line 697, in _initialize
    *args, **kwds))

  File "d:\conda\envs\tf2\lib\site-packages\tensorflow\python\eager\function.py", line 2855, in _get_concrete_function_internal_garbage_collected
    graph_function, _, _ = self._maybe_define_function(args, kwargs)

  File "d:\conda\envs\tf2\lib\site-packages\tensorflow\python\eager\function.py", line 3213, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)

  File "d:\conda\envs\tf2\lib\site-packages\tensorflow\python\eager\function.py", line 3075, in _create_graph_function
    capture_by_value=self._capture_by_value),

  File "d:\conda\envs\tf2\lib\site-packages\tensorflow\python\framework\func_graph.py", line 986, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)

  File "d:\conda\envs\tf2\lib\site-packages\tensorflow\python\eager\def_function.py", line 600, in wrapped_fn
    return weak_wrapped_fn().__wrapped__(*args, **kwds)

  File "d:\conda\envs\tf2\lib\site-packages\tensorflow\python\framework\func_graph.py", line 973, in wrapper
    raise e.ag_error_metadata.to_exception(e)

TypeError: in user code:

    <ipython-input-1-eafa53e19dc9>:19 grad  *
        return tape.gradient(y, model.trainable_variables)
    d:\conda\envs\tf2\lib\site-packages\tensorflow\python\eager\backprop.py:1073 gradient  **
        unconnected_gradients=unconnected_gradients)
    d:\conda\envs\tf2\lib\site-packages\tensorflow\python\eager\imperative_grad.py:77 imperative_grad
        compat.as_str(unconnected_gradients.value))
    d:\conda\envs\tf2\lib\site-packages\tensorflow\python\eager\backprop.py:162 _gradient_function
        return grad_fn(mock_op, *out_grads)
    d:\conda\envs\tf2\lib\site-packages\tensorflow\python\ops\array_grad.py:678 _GatherV2Grad
        batch_dims, params_shape[axis])
    d:\conda\envs\tf2\lib\site-packages\tensorflow\python\ops\array_grad.py:603 _BatchGatherGrad
        indices = _GetBatchIndices(params_shape, indices, batch_dims)
    d:\conda\envs\tf2\lib\site-packages\tensorflow\python\ops\array_grad.py:582 _GetBatchIndices
        [1] * (dim - 1) + [dim_value] + [1] * (indices_ndims - dim), axis=0)

    TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'

Note, that both eager and graph modes work if I replace tf.numpy_function with tf.argmin and the results are the same (up to rounding errors). Graph execution also fails when using np.argmin.

@akkiss
I have tried in colab with TF -GPU version 2.4 and i am seeing the below error message.(AttributeError: module 'assign' has no attribute 'assign_objects_to_cells').Please, find the gist here.
Please, share colab link or simple standalone code with supporting files to reproduce the issue in our environment.It helps us in localizing the issue faster. Thanks!

Assign is not a library but a cython function (the "And the assign.pyx:" part). This is not required anyway, the problem appears with np.argmin as well. Unfortunately I cannot make my colab public, but this code reproduced the problem:
!pip install tensorflow-gpu==2.4.0

import tensorflow as tf
import numpy as np
#import assign

model = tf.keras.applications.ResNet50(False, weights=None, input_shape=(224, 224, 3))
data = tf.random.normal((8,224,224,3), 128, 64)

def f(x):
	y = model(x)
	#idx = tf.numpy_function(assign.assign_objects_to_cells, (-y,), np.int64)
	#idx = tf.argmin(-y, -1)
	idx = tf.numpy_function(np.argmin, (-y, -1), np.int64)
	values = tf.gather(y, idx, batch_dims=3)
	return tf.reduce_sum(values)

def grad(x):
	with tf.GradientTape() as tape:
		y = f(x)
	return tape.gradient(y, model.trainable_variables)
tf_grad = tf.function(grad)

print('eager execution')
g = grad(data)
print('graph execution')
g = tf_grad(data)

The problem is that tf.numpy_functions returns completely undefined static shape; i.e.,
  print(idx.shape)

returns (8, 7, 7) in Eager mode, but `' in Graph mode (i.e., not even the rank of the result is known). This behaviour is expected (it would be difficult to perform some static analysis of the resulting shape).
What you can do is to provide the information about the output shape (and then check at runtime that it holds). You can do so quite easily by using the following just after idx = tf.numpy_function(np.argmin, (-y, -1), np.int64):
  idx = tf.ensure_shape(idx, y.shape[:-1])

The idx has the same value, but it has the same static shape as y, apart from the last dimension (which is eradicated by the np.argmin); the tf.ensure_shape even checks at runtime that the dynamic shape matches.

BTW, the gist with the added line: https://colab.research.google.com/drive/1iOwlwiETazXwkHt0SvNgZP29KOXKDjQg?usp=sharing

OMG, so obvious, how did I not think of this... Thanks @foxik for the solution.
If someone ever reads this might be happy to hear this works also with variable tensor shapes:
idx = tf.ensure_shape(idx, 3*[None])


HAPI-141 HNO table

@davidmegginson I've updated this with the changes in document 14.
I await answers to some questions here: https://humanitarian.atlassian.net/browse/HAPI-326, but in the meantime, I have included gender_description, sector_name, population_group_description and population_status_description in the humanitarian needs view and made population_status (ie. inneed, affected) mandatory in the humanitarian needs table. I can easily make changes to these.

@davidmegginson I've changed population_status _code to nullable=True and removed gender_description, population_group_description and population_status_description.
I have changed gender_code to String(1), but left reference_period* as DateTime due to what it says in the SQLAlchemy docs:
The rudimental types have “CamelCase” names such as String, Numeric, Integer, and DateTime. All of the immediate subclasses of TypeEngine are “CamelCase” types. The “CamelCase” types are to the greatest degree possible database agnostic"

Thank you. Please go ahead and merge.


Training metrics are being accumulated and not reset on train end by default

🐛 Bug
Using self.log inside training_step accumulates the metrics, which are never reset before the validation step starts, resulting in validation results that correspond to the union of the training and validation datasets.
To Reproduce
class BoringModel(LightningModule):
    def __init__(
            self,
            size: Tuple[int, int],
            num_classes: int,
    ) -> None:

        super().__init__()

        self.save_hyperparameters(logger=False)

        self.size = size
        self.num_classes = num_classes

        ...

        self.segmentation_accuracy = Accuracy(num_classes=num_classes)
        self.segmentation_iou = JaccardIndex(num_classes)

    def calculate_loss(self, batch: BatchType) -> Dict[str, Union[Tensor, Metric]]:
        ...

        loss = loss_segmentation + loss_depth

        self.segmentation_accuracy(output_segmentation, segmentation)
        self.segmentation_iou(output_segmentation, segmentation)

        metrics = {
            'Loss': loss,
            'Loss Segmentation': loss_segmentation,
            'Loss Depth': loss_depth,
            'Segmentation Accuracy': self.segmentation_accuracy,
            'Segmentation IoU': self.segmentation_iou,
        }

        return metrics

    def training_step(self, batch: BatchType, *args) -> Tensor:
        metrics = self.calculate_loss(batch)

        for name, value in metrics.items():
            self.log(f'Train {name}', value)

        loss = metrics['Loss']

        return loss

    def validation_step(self, batch: BatchType, *args) -> None:
        metrics = self.calculate_loss(batch)

        for name, value in metrics.items():
            self.log(f'Validation {name}', value)

    def test_step(self, batch: BatchType, *args) -> None:
        metrics = self.calculate_loss(batch)

        for name, value in metrics.items():
            self.log(f'Test {name}', value)

    def on_validation_start(self) -> None:
        super().on_validation_start()

        num_samples1 = (
            self.segmentation_accuracy.tp +
            self.segmentation_accuracy.fp +
            self.segmentation_accuracy.tn +
            self.segmentation_accuracy.fn
        ) / self.size[0] / self.size[1] / self.num_classes
        num_samples2 = torch.sum(self.segmentation_iou.confmat) / self.size[0] / self.size[1]
        print('Validation start', num_samples1, num_samples2)

    def on_validation_end(self) -> None:
        num_samples1 = (
            self.segmentation_accuracy.tp +
            self.segmentation_accuracy.fp +
            self.segmentation_accuracy.tn +
            self.segmentation_accuracy.fn
        ) / self.size[0] / self.size[1] / self.num_classes
        num_samples2 = torch.sum(self.segmentation_iou.confmat) / self.size[0] / self.size[1]
        print('Validation end', num_samples1, num_samples2)

        super().on_validation_end()

Here's how I am training the model:
logger = MLFlowLogger()

trainer = Trainer(
    max_epochs=max_num_epochs,
    logger=logger,
    enable_progress_bar=False,
    log_every_n_steps=utils.get_num_steps(dataset_train, batch_size),
    gpus=torch.cuda.device_count(),
    callbacks=[
        EarlyStopping(
            monitor='Validation Segmentation IoU',
            mode='max',
            patience=patience,
        ),
        ModelCheckpoint(
            monitor='Validation Segmentation IoU',
            mode='max',
        ),
    ],
)

trainer.fit(
    model=model,
    train_dataloaders=DataLoader(
        dataset=dataset_train,
        batch_size=batch_size,
        shuffle=True,
        num_workers=4,
    ),
    val_dataloaders=DataLoader(
        dataset=dataset_validation,
        batch_size=batch_size,
        num_workers=4,
    ),
)

Expected behavior
The training dataset has 2975 samples and the validation dataset has 500 samples. The validation metrics should be computed on 500 samples but I am getting 3475 (2975 + 500). Here's the output from running this code:
Validation start tensor(0., device='cuda:0') tensor(0., device='cuda:0')
Validation end tensor(20., device='cuda:0') tensor(20., device='cuda:0')
Validation start tensor(2975., device='cuda:0') tensor(2975., device='cuda:0')
Validation end tensor(3475., device='cuda:0') tensor(3475., device='cuda:0')
Validation start tensor(2975., device='cuda:0') tensor(2975., device='cuda:0')
Validation end tensor(3475., device='cuda:0') tensor(3475., device='cuda:0')
Validation start tensor(2975., device='cuda:0') tensor(2975., device='cuda:0')
Validation end tensor(3475., device='cuda:0') tensor(3475., device='cuda:0')
Validation start tensor(2975., device='cuda:0') tensor(2975., device='cuda:0')
Validation end tensor(3475., device='cuda:0') tensor(3475., device='cuda:0')
Validation start tensor(2975., device='cuda:0') tensor(2975., device='cuda:0')
Validation end tensor(3475., device='cuda:0') tensor(3475., device='cuda:0')
Validation start tensor(2975., device='cuda:0') tensor(2975., device='cuda:0')
Validation end tensor(3475., device='cuda:0') tensor(3475., device='cuda:0')

Environment

PyTorch Lightning Version: 1.5.9
PyTorch Version: 1.10.1
Python version: 3.9.6
OS: Linux
CUDA/cuDNN version: 11.3
GPU models and configuration: Tested on diverse GPUs (V100, A100, Titan X).
How you installed PyTorch: pip

Additional context
Setting on_epoch=False when logging on training_step seems to not have any effect: the metrics are still aggregated and they do not reset before validation starts.

hey @adeandrade !
you shouldn't use the same metric instance for both training and validation since they contain states which might differ for both.
check out the note here: https://torchmetrics.readthedocs.io/en/latest/pages/overview.html

hey @adeandrade !
you shouldn't use the same metric instance for both training and validation since they contain states which might differ for both.
check out the note here: https://torchmetrics.readthedocs.io/en/latest/pages/overview.html

Thank you @rohitgr7. I'll use this approach then. Though It would simplify things if we could reuse metrics across stages. Why shouldn't we just reuse the metric and reset the state before validation starts? Is there a use case where the state needs to be kept across epochs?

in Lightning, we support multiple validations within a single training epoch, so for consistency and in a general case, the validation loop runs right after the last training batch but the training loop isn't completed yet, thus the metrics associated with it aren't reset either when validation loop starts. This is not the only reason why it recommends using different instances but since you are using modular ones, it stores states internally, so ideally keeping them separated ensures they don't mix up and computes independently.


[To rel/1.1] [IOTDB-5717] Fix incorrect result when querying with limit push-downing & order by time desc

see IOTDB-5717.
Before:

After:

see #9434


[deliver] Can't upload tvOS Apple TV 4k screenshots

New Issue Checklist

[x] Updated fastlane to the latest version
[x] I read the Contribution Guidelines
[x] I read docs.fastlane.tools
[x] I searched for existing GitHub issues

Issue Description
expected behavior:
Upload Screenshots to AppStore Connect
happening:
Unsupported Screen size
ERROR [2018-12-17 16:39:44.89]: Unsupported screen size [3840, 2160] for path './screenshots/tvOS/de-DE/Apple TV 4K-0-TV.png'

my lane:

  lane :tvOSUploadScreenshots do
    deliver(skip_binary_upload: true,
                 skip_metadata: true,
                      platform: "appletvos",
              screenshots_path: "./screenshots/tvOS")
  end

Complete output when running fastlane, including the stack trace and command used

[✔] 🚀
[16:39:16]: fastlane detected a Gemfile in the current directory
[16:39:16]: however it seems like you don't use `bundle exec`
[16:39:16]: to launch fastlane faster, please use
[16:39:16]:
[16:39:16]: $ bundle exec fastlane tvOSUploadScreenshots --capture_output
[16:39:16]:
[16:39:16]: Get started using a Gemfile for fastlane https://docs.fastlane.tools/getting-started/ios/setup/#use-a-gemfile
DEBUG [2018-12-17 16:39:17.63]: Checking if there are any plugins that should be loaded...
DEBUG [2018-12-17 16:39:17.72]: Loading 'fastlane-plugin-appicon' plugin
INFO [2018-12-17 16:39:17.77]: gem 'fastlane-plugin-appicon' is already installed
DEBUG [2018-12-17 16:39:17.79]: Loading 'fastlane-plugin-versioning' plugin
INFO [2018-12-17 16:39:17.79]: gem 'fastlane-plugin-versioning' is already installed
DEBUG [2018-12-17 16:39:17.82]: Loading 'fastlane-plugin-google_drive' plugin
INFO [2018-12-17 16:39:17.82]: gem 'fastlane-plugin-google_drive' is already installed
INFO [2018-12-17 16:39:18.13]: ------------------------------
INFO [2018-12-17 16:39:18.13]: --- Step: default_platform ---
INFO [2018-12-17 16:39:18.13]: ------------------------------
INFO [2018-12-17 16:39:18.14]: Driving the lane 'ios tvOSUploadScreenshots' 🚀
INFO [2018-12-17 16:39:18.14]: ---------------------------
INFO [2018-12-17 16:39:18.14]: --- Step: xcode_install ---
INFO [2018-12-17 16:39:18.14]: ---------------------------
INFO [2018-12-17 16:39:18.14]: gem 'xcode-install' is already installed
INFO [2018-12-17 16:39:18.68]: Xcode 10.1 is already installed ✨
INFO [2018-12-17 16:39:19.06]: Using Xcode 10.1 on path '/Applications/Xcode.app'
INFO [2018-12-17 16:39:19.19]: ---------------------
INFO [2018-12-17 16:39:19.19]: --- Step: deliver ---
INFO [2018-12-17 16:39:19.19]: ---------------------
INFO [2018-12-17 16:39:19.23]: Successfully loaded '/Users/mariohahn/Documents/work/fastlane/Deliverfile' 📄
INFO [2018-12-17 16:39:19.23]: Login to App Store Connect
INFO [2018-12-17 16:39:21.77]: Login successful
INFO [2018-12-17 16:39:44.89]: Currently not running on CI system, skipping keychain cleanup
WARN [2018-12-17 16:39:44.89]: Lane Context:
INFO [2018-12-17 16:39:44.89]: {:DEFAULT_PLATFORM=>:ios, :PLATFORM_NAME=>:ios, :LANE_NAME=>"ios tvOSUploadScreenshots", :XCODE_INSTALL_XCODE_PATH=>#}
ERROR [2018-12-17 16:39:44.89]: Unsupported screen size [3840, 2160] for path './screenshots/tvOS/de-DE/Apple TV 4K-0-TV.png'
INFO [2018-12-17 16:39:44.89]: Successfully generated documentation at path '/Users/mariohahn/Documents/work/fastlane/README.md'
DEBUG [2018-12-17 16:39:44.90]: All plugins are up to date
ERROR [2018-12-17 16:39:44.90]: fastlane finished with errors
DEBUG [2018-12-17 16:39:44.90]: All plugins are up to date

Environment

[✔] 🚀

Apple TV is indeed only listed as
https://github.com/fastlane/fastlane/blob/cdfdc2e5d7e46008c56018ec46274d2d12ed5745/deliver/lib/deliver/app_screenshot.rb#L269-L271
I don't have experience with Apple TV, do you upload those 4K screenshots additionally to the normal ones or just the 4K ones that are then downscaled by Apple?

I only use both Screenshots, 4K und normal, is it just adding the Size here?

Reference: https://help.apple.com/app-store-connect/#/devd274dd925


AH: Filter Custom Data Download By Vintage Year

Definition of Ready

[x] Does this meet the INVEST criteria?
[x] Does this have a clearly defined user?
[x] Does this have an estimate?
[x] Does this have acceptance criteria?
[x] Does this have sufficient context (wireframes, business rules, etc.)?
[x] Is this traceable to a feature, epic, or PI?

User Story
As a CAMPD custom data download user I want to enter filter criteria for vintage year so that my dataset includes data for my desired vintage year(s)
Acceptance Criteria
Given that a user has navigated to the custom data download and selected allowance holdings
When the filter criteria page is displayed
Then the user has the option to select vintage year
Given that a user has selected vintage year
When filter panel is displayed
Then the user can enter one, multiple, or a range of  vintages years
Given a user has entered filter criteria for vintage year
When the user applies the filter
Then the system will ensure that the data is valid
Given that a user has selected vintage year
When the criteria is applied
Then the system ensures the criteria is in the YYYY format
And displays error message FV-102 if not
Given that a user has selected vintage year
When the criteria is applied
Then the system ensures the criteria is between 1995 or greater
And displays error message FV-105 if not
Given a validation has failed
When the error message is displayed
Then the error message is displayed in accordance with the USWDS validation component
Given that a user has applied filter criteria for vintage year
When the criteria is applied
Then the system displays a filter tag
Given that a user has applied filter criteria for vintage year
When any vintage year criteria is cleared
Then the system clears the criteria from the filter
Context
Error Message Repo
Wireframes
Functionality
Vintage Year
Text field. Allows ranges and individual years (YYYY-YYYY, YYYY, YYYY)
Format: YYYY
Optional
Valid date range: 1995 or greater
Multi select (multiple 2019,2020, range 2017-2020, and indiv. Years 2015)
**Breadcrumb display: Vintage year (no years displayed) see here
Filter Tags (need confirmation)
Populates filter tag on Apply Filter
if 1 parameter is selected, then show parameter in tag
Tag will remain fixed width, if parameters text exceeds fixed width, then full name is displayed in hover text/Tool tip
If more than 1, show number of parameters
Clicking x in tag clear filter
Click on the tag opens filter in panel
Will provide the option to clear all filters in tag group
Tags will have a hover functionality to display the list of selections
Testing
Functional Testing
Test Case: TC247
Tester: @JanellC
508 Testing
Tester: @aprematta
Definition of Done

[x] Have all Acceptance Criteria been agreed to and validated by the scrum team?
[x] Has the code been unit tested and peer reviewed?
[x] Have the functional tests been executed?
[x] Have all defects been reviewed and dispositioned, resolved, or deferred?
[ ] Has the user story been accepted by the Product Owner?
[ ] Is the user story ready to be deployed to the test/staging environments?

TC247 failed. I logged Bug: Update Year Component Validation #1783 to track the changes


Add to Boxes3D the same array/array_format convenience features as Boxes2D

Boxes2D can be constructed with convenience array and array_format argument but Boxes3D is missing that feature.

See #3719 for a test case


Clarification on Hosting Module backward compat

As per the discussion here: https://docs.microsoft.com/en-us/answers/questions/176303/net-core-31-apps-with-the-net-5-hosting-bundle-are.html
It would be good to clarify somewhere whether the ASP.NET Core Module that get's installed via the Hosting Bundle, will still run older asp.net core websites. There can be some concern around this when you already have an IIS server hosting asp.net core 3.1 websites and you wish to test out deploying an asp.net core 5.0 website along side - the runtimes install side by side, but the ASP.NET Core Module appears to be "shared"

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: fe69b3a9-896f-e189-0c2c-f87ff735df9c
Version Independent ID: 97c8549b-66ad-0a23-1971-784fb923042a
Content: ASP.NET Core Module
Content Source: aspnetcore/host-and-deploy/aspnet-core-module.md
Product: aspnet-core
Technology: aspnetcore-hostdeploy
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

@jkotalik  please review

Tagged as "Needs more info" - is there anything else I can provide or clarify?

Thank you for contacting us. Due to a lack of activity on this discussion issue we're closing it in an effort to keep our backlog clean. If you believe there is a concern related to the ASP.NET Core which hasn't been addressed, please file a new issue.


[MINOR]Optimize hudi-client module

Tips

Thank you very much for contributing to Apache Hudi.
Please review https://hudi.apache.org/contributing.html before opening a pull request.

What is the purpose of the pull request
Optimize hudi-client module code, including simplifing lamada expression, fixed spelling mistake, removing unused variable.
Brief change log

Simplify lamada expression.
Fixed spelling mistake.
Remove unused variable.

Verify this pull request
This pull request is a trivial rework / code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end.
Added HoodieClientWriteTest to verify the change.
Manually verified the change by running a job locally.

Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

maybe use git merge --squash first to set a single commit, pleas.

@SteNicholas Thanks for opening the PR, looks good overall, only left some minor comments.

@leesf Please review the modification accoring to your comments. @vinothchandar Do you have any comments about this?

@SteNicholas can you please rebase and resolve the conflicts?

Thanks. merging @leesf !


Hardcoded decode cache makes logging jpeg defacto more memory intensive than raw images

If a user sends all their images as jpeg, e.g. rr.log_image(f'{label}', img, jpeg_quality=75), they will observe higher memory usage until the tensor decode cache fills up than they would have with uncompressed images.
This cache has a hardcoded size right now of 4gb on desktop and 1gb on web
https://github.com/rerun-io/rerun/blob/be7a03134b86376689f7c4d4d19231035ac6f22d/crates/re_viewer_context/src/tensor/tensor_decode_cache.rs#L62
A bunch of action points:

[ ] document decoding caching behavior to not make any false promises (point out that viewer may use more memory, saved files won't, etc.)
[ ] make cache size configurable
[ ] default to a smaller cache - 4gb seems excessive. The reason we have a cache as big is that jpeg decoding is too slow for smooth playback. There's a bunch of ways to address this

[ ] look into faster decompression libraries
[ ] more clever caching than just a size budget

[ ] if an image wasn't used for very long it should be evicted
[ ] don't cache an image if its decode didn't take very long to begin with

[ ] ?

A whole different solution for this popped up now:
In https://github.com/rerun-io/rerun/pull/7457 we successfully read back texture values from a video. The code is written in a way that we could do this with any rgb texture (and formats in the future).
Also, it turns out that it seems that by now the only reason we have to keep around decompressed images is inspection via mouse.
-> We can make this a gpu-memory only cache, thus solving this issue
this ofc still means up to 4gb of vram used by the cache for which all previous notes are still relevant, but this solves the issue at hand nonetheless. On a system wide perspective (very relevant for shared ram/vram architectures) it halves the memory needed.
In my experience running out of vram also swaps fine into ram without issues.

turns out our decode cache is cpu-only right now, creating gpu textures only on the fly if needed.
That makes above idea a lot less attractive. I still think it would be the right move, but also means we don't get "half the memory consumption" either.


Should we start using Quay.io ?

Following @bgruening suggestions, I did some tests with Quay.io and it is really, much better than the native DockerHub service. So here are some points for us to consider:
Advantages

Better API
Faster (my test image was created and shipped within 10 minutes, compared to DockerHub that is taking from 2 to 6 hours)
Better namespace (here we can have the biodocker namespace, instead of biodckr)

Disadvantages

Looks that the build system is 'repository-centric', I'm not sure how to provide our containers with different versions. In DockerHub I create an Automated Build entry and inside I can specify every Dockerfile that is located on each folder.
Looks like you need an account to use images from Quay.io (see below).

The good thing about this, @ypriverol @bgruening , is that I figured out how to use the native docker client to search and download images from quay.io ( you need to create an account at quay.io )

docker login quay.io

docker search quay.io/comet   or docker search quay.io/biodocker

simple as that!

@prvst As we decided we will push to both places dockerhub and the quay.io. I fully support that.

@prvst As we decided we will push to both places dockerhub and the quay.io. I fully support that.

I remember, I just added this as a question to put everyone in context.

BTW @ypriverol , if we adopt both, how do we manage the namespace? keep using biodckr in Quay.io too?

Yes please is better for integration and understanding.

Yes please is better for integration and understanding.
In any case, I got both under my user control, so we can protect the name.


bug: panic on memory dump

Describe the bug
thread 'risingwave-main' panicked at src/compute/src/memory_management/policy.rs:132:58:
called `Result::unwrap()` on an `Err` value: Attempt to read or write `void` value, or attempt to write read-only value.
stack backtrace:
   0:     0x55bb8bb06bd7 - std::backtrace_rs::backtrace::libunwind::trace::h75b30c0cd68bc7af
                               at /rustc/62ebe3a2b177d50ec664798d731b8a8d1a9120d1/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
   1:     0x55bb8bb06bd7 - std::backtrace_rs::backtrace::trace_unsynchronized::h87a4581e60db2a55
                               at /rustc/62ebe3a2b177d50ec664798d731b8a8d1a9120d1/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x55bb8bb06bd7 - std::sys_common::backtrace::_print_fmt::hb61bbd4cad6c8d05
                               at /rustc/62ebe3a2b177d50ec664798d731b8a8d1a9120d1/library/std/src/sys_common/backtrace.rs:67:5
   3:     0x55bb8bb06bd7 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h78c87721a2fe0851
                               at /rustc/62ebe3a2b177d50ec664798d731b8a8d1a9120d1/library/std/src/sys_common/backtrace.rs:44:22
   4:     0x55bb8569ae9c - core::fmt::rt::Argument::fmt::h3a1f78153a55cf27
                               at /rustc/62ebe3a2b177d50ec664798d731b8a8d1a9120d1/library/core/src/fmt/rt.rs:138:9
   5:     0x55bb8569ae9c - core::fmt::write::h78df6aa1c7979fcd
                               at /rustc/62ebe3a2b177d50ec664798d731b8a8d1a9120d1/library/core/src/fmt/mod.rs:1094:21
   6:     0x55bb8bb0151e - std::io::Write::write_fmt::h51abcfccd2bfcce9
                               at /rustc/62ebe3a2b177d50ec664798d731b8a8d1a9120d1/library/std/src/io/mod.rs:1714:15
   7:     0x55bb8bb069b4 - std::sys_common::backtrace::_print::h900c04ec6229a3eb
                               at /rustc/62ebe3a2b177d50ec664798d731b8a8d1a9120d1/library/std/src/sys_common/backtrace.rs:47:5
   8:     0x55bb8bb069b4 - std::sys_common::backtrace::print::heb918b083479ba12
                               at /rustc/62ebe3a2b177d50ec664798d731b8a8d1a9120d1/library/std/src/sys_common/backtrace.rs:34:9
   9:     0x55bb8bb0847b - std::panicking::panic_hook_with_disk_dump::{{closure}}::h308105f13195c33c
                               at /rustc/62ebe3a2b177d50ec664798d731b8a8d1a9120d1/library/std/src/panicking.rs:280:22
  10:     0x55bb8bb08136 - std::panicking::panic_hook_with_disk_dump::h2f5bad6440ca252c
                               at /rustc/62ebe3a2b177d50ec664798d731b8a8d1a9120d1/library/std/src/panicking.rs:314:9

Error message/log
No response
To Reproduce
No response
Expected behavior
No response
How did you deploy RisingWave?
No response
The version of RisingWave
No response
Additional context
No response

Hmmm...
https://github.com/jemalloc/jemalloc/blob/54eaed1d8b56b1aa528be3bdd1877e59c56fa90c/src/ctl.c#L3310


schema_json does not include $schema version it is supposed to comply with

Bug
The generated schemas do not include any default $schema field which is quite dangerous as it gives no indication who can use them.
Which ones are they supposed to respect?

https://json-schema.org/draft/2020-12/schema
https://json-schema.org/draft/2019-09/schema
https://json-schema.org/draft/draft-07/schema

As someone that expect to produce schemas using pydantic, I do need to also indicate the consumers which version of the spec they should expect to use, especially as not all consumers can deal with all versions.

nearly a year old, would love to know this as well.  Does anyone know off the top of their head what draft is being targeted?

fwiw, the link in this sentence in the docs currently points to draft/2020-12

The generated schemas are compliant with the specifications: JSON Schema Core, JSON Schema Validation and OpenAPI.

If I may jump in the conversation. We had an issue with schema version as well.
The generated schema for data classes were used as sub schema inside a larger schema. Validation tools do not allow for mixed versions. Therefore, all schemas must respect one and only one version.
Using pydantic to generate schema automatically should provide a mechanism to target specific version.

In V2, we are explicitly targeting https://json-schema.org/draft/2020-12/schema. I know that currently it generates something with openapi extensions when discriminators are involved, and we do plan to add support for modifying that behavior.
We designed the JSON schema generation approach to be much easier to modify thanks to the ability to override GenerateJsonSchema, so while we don't currently include the schema version by default when generating the schema, it should be straightforward to add it by overriding the GenerateJsonSchema.generate method (or similar) as you see fit.


Remove code for the help and services around the world page

Whitehall no longer renders this page so the code can be removed
Trello - https://trello.com/c/9zmuoJrZ/519-remove-code-for-help-and-services-around-the-world-from-whitehall
⚠ This repo is Continuously Deployed: make sure you follow the guidance ⚠

There's a failing test. Looks like test/functional/world_locations_controller_test.rb also needs to be removed.

yeah, I think this must have been reintroduced in one of my many rebases, had everything green this morning.


Cannot Compile on Python3.9 Android ARM64 Termux

I have installed a near complete python stack on my Android including popular chemical libraries like CoolProp, Cantera which are fully functional in my jupyter notebooks.
My libraries versions

Cython               0.29.21
setuptools           58.1.0
scipy                1.6.3
numpy                1.20.3

While installing through pip (git version), following error occurs
rocessing /data/data/com.termux/files/home/pyequion2
  Installing build dependencies ... error
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [228 lines of output]
      Collecting setuptools>=42
        Using cached setuptools-65.3.0-py3-none-any.whl (1.2 MB)
      Collecting Cython>=0.15.1
        Using cached Cython-0.29.32-py2.py3-none-any.whl (986 kB)
      Collecting numpy>=1.10
        Using cached numpy-1.23.3.tar.gz (10.7 MB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Building wheels for collected packages: numpy
        Building wheel for numpy (pyproject.toml): started
        Building wheel for numpy (pyproject.toml): finished with status 'error'
        error: subprocess-exited-with-error

        × Building wheel for numpy (pyproject.toml) did not run successfully.
        │ exit code: 1
        ╰─> [202 lines of output]
            Running from numpy source directory.
            setup.py:86: DeprecationWarning:

              `numpy.distutils` is deprecated since NumPy 1.23.0, as a result
              of the deprecation of `distutils` itself. It will be removed for
              Python >= 3.12. For older Python versions it will remain present.
              It is recommended to use `setuptools < 60.0` for those Python versions.
              For more details, see:
                https://numpy.org/devdocs/reference/distutils_status_migration.html

              import numpy.distutils.command.sdist
            Processing numpy/random/_bounded_integers.pxd.in
            Processing numpy/random/_bounded_integers.pyx.in
            Processing numpy/random/_common.pyx
            Processing numpy/random/_generator.pyx
            Processing numpy/random/_mt19937.pyx
            Processing numpy/random/_pcg64.pyx
            Processing numpy/random/_philox.pyx
            Processing numpy/random/_sfc64.pyx
            Processing numpy/random/bit_generator.pyx
            Processing numpy/random/mtrand.pyx
            Cythonizing sources
            INFO: blas_opt_info:
            INFO: blas_armpl_info:
            INFO: customize UnixCCompiler
            INFO:   libraries armpl_lp64_mp not found in ['/data/data/com.termux/files/usr/lib']
            INFO:   NOT AVAILABLE
            INFO:
            INFO: blas_mkl_info:
            INFO:   libraries mkl_rt not found in ['/data/data/com.termux/files/usr/lib']
            INFO:   NOT AVAILABLE
            INFO:
            INFO: blis_info:
            INFO:   libraries blis not found in ['/data/data/com.termux/files/usr/lib']
            INFO:   NOT AVAILABLE
            INFO:
            INFO: openblas_info:
            INFO: C compiler: aarch64-linux-android-clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -fstack-protector-strong -O3 -fstack-protector-strong -O3 -fPIC

            creating /data/data/com.termux/files/usr/tmp/tmprg6neymz/data
            creating /data/data/com.termux/files/usr/tmp/tmprg6neymz/data/data
            creating /data/data/com.termux/files/usr/tmp/tmprg6neymz/data/data/com.termux
            creating /data/data/com.termux/files/usr/tmp/tmprg6neymz/data/data/com.termux/files
            creating /data/data/com.termux/files/usr/tmp/tmprg6neymz/data/data/com.termux/files/usr
            creating /data/data/com.termux/files/usr/tmp/tmprg6neymz/data/data/com.termux/files/usr/tmp
            creating /data/data/com.termux/files/usr/tmp/tmprg6neymz/data/data/com.termux/files/usr/tmp/tmprg6neymz
            INFO: compile options: '-c'
            INFO: aarch64-linux-android-clang: /data/data/com.termux/files/usr/tmp/tmprg6neymz/source.c
            INFO: aarch64-linux-android-clang /data/data/com.termux/files/usr/tmp/tmprg6neymz/data/data/com.termux/files/usr/tmp/tmprg6neymz/source.o -L/data/data/com.termux/files/usr/lib -lopenblas -o /data/data/com.termux/files/usr/tmp/tmprg6neymz/a.out
            INFO:   FOUND:
            INFO:     libraries = ['openblas', 'openblas']
            INFO:     library_dirs = ['/data/data/com.termux/files/usr/lib']
            INFO:     language = c
            INFO:     define_macros = [('HAVE_CBLAS', None)]
            INFO:
            INFO:   FOUND:
            INFO:     libraries = ['openblas', 'openblas']
            INFO:     library_dirs = ['/data/data/com.termux/files/usr/lib']
            INFO:     language = c
            INFO:     define_macros = [('HAVE_CBLAS', None)]
            INFO:
            non-existing path in 'numpy/distutils': 'site.cfg'
            INFO: lapack_opt_info:
            INFO: lapack_armpl_info:
            INFO:   libraries armpl_lp64_mp not found in ['/data/data/com.termux/files/usr/lib']
            INFO:   NOT AVAILABLE
            INFO:
            INFO: lapack_mkl_info:
            INFO:   libraries mkl_rt not found in ['/data/data/com.termux/files/usr/lib']
            INFO:   NOT AVAILABLE
            INFO:
            INFO: openblas_lapack_info:
            INFO: C compiler: aarch64-linux-android-clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -fstack-protector-strong -O3 -fstack-protector-strong -O3 -fPIC

            creating /data/data/com.termux/files/usr/tmp/tmpjojkuiiu/data
            creating /data/data/com.termux/files/usr/tmp/tmpjojkuiiu/data/data
            creating /data/data/com.termux/files/usr/tmp/tmpjojkuiiu/data/data/com.termux
            creating /data/data/com.termux/files/usr/tmp/tmpjojkuiiu/data/data/com.termux/files
            creating /data/data/com.termux/files/usr/tmp/tmpjojkuiiu/data/data/com.termux/files/usr
            creating /data/data/com.termux/files/usr/tmp/tmpjojkuiiu/data/data/com.termux/files/usr/tmp
            creating /data/data/com.termux/files/usr/tmp/tmpjojkuiiu/data/data/com.termux/files/usr/tmp/tmpjojkuiiu
            INFO: compile options: '-c'
            INFO: aarch64-linux-android-clang: /data/data/com.termux/files/usr/tmp/tmpjojkuiiu/source.c
            INFO: aarch64-linux-android-clang /data/data/com.termux/files/usr/tmp/tmpjojkuiiu/data/data/com.termux/files/usr/tmp/tmpjojkuiiu/source.o -L/data/data/com.termux/files/usr/lib -lopenblas -o /data/data/com.termux/files/usr/tmp/tmpjojkuiiu/a.out
            INFO:   FOUND:
            INFO:     libraries = ['openblas', 'openblas']
            INFO:     library_dirs = ['/data/data/com.termux/files/usr/lib']
            INFO:     language = c
            INFO:     define_macros = [('HAVE_CBLAS', None)]
            INFO:
            INFO:   FOUND:
            INFO:     libraries = ['openblas', 'openblas']
            INFO:     library_dirs = ['/data/data/com.termux/files/usr/lib']
            INFO:     language = c
            INFO:     define_macros = [('HAVE_CBLAS', None)]
            INFO:
            Warning: attempted relative import with no known parent package
            /data/data/com.termux/files/usr/lib/python3.9/distutils/dist.py:274: UserWarning: Unknown distribution option: 'define_macros'
              warnings.warn(msg)
            running bdist_wheel
            running build
            running config_cc
            INFO: unifing config_cc, config, build_clib, build_ext, build commands --compiler options
            running config_fc
            INFO: unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
            running build_src
            INFO: build_src
            INFO: building py_modules sources
            creating build
            creating build/src.linux-aarch64-3.9
            creating build/src.linux-aarch64-3.9/numpy
            creating build/src.linux-aarch64-3.9/numpy/distutils
            INFO: building library "npymath" sources
            WARN: Could not locate executable armflang
            WARN: Could not locate executable gfortran
            WARN: Could not locate executable f95
            WARN: Could not locate executable ifort
            WARN: Could not locate executable ifc
            WARN: Could not locate executable lf95
            WARN: Could not locate executable pgfortran
            WARN: Could not locate executable nvfortran
            WARN: Could not locate executable f90
            WARN: Could not locate executable f77
            WARN: Could not locate executable fort
            WARN: Could not locate executable efort
            WARN: Could not locate executable efc
            WARN: Could not locate executable g77
            WARN: Could not locate executable g95
            WARN: Could not locate executable pathf95
            WARN: Could not locate executable nagfor
            WARN: Could not locate executable frt
            WARN: don't know how to compile Fortran code on platform 'posix'
            creating build/src.linux-aarch64-3.9/numpy/core
            creating build/src.linux-aarch64-3.9/numpy/core/src
            creating build/src.linux-aarch64-3.9/numpy/core/src/npymath
            INFO: conv_template:> build/src.linux-aarch64-3.9/numpy/core/src/npymath/npy_math_internal.h
            INFO:   adding 'build/src.linux-aarch64-3.9/numpy/core/src/npymath' to include_dirs.
            INFO: conv_template:> build/src.linux-aarch64-3.9/numpy/core/src/npymath/ieee754.c
            INFO: conv_template:> build/src.linux-aarch64-3.9/numpy/core/src/npymath/npy_math_complex.c
            INFO: None - nothing done with h_files = ['build/src.linux-aarch64-3.9/numpy/core/src/npymath/npy_math_internal.h']
            INFO: building library "npyrandom" sources
            INFO: building extension "numpy.core._multiarray_tests" sources
            creating build/src.linux-aarch64-3.9/numpy/core/src/multiarray
            INFO: conv_template:> build/src.linux-aarch64-3.9/numpy/core/src/multiarray/_multiarray_tests.c
            INFO: building extension "numpy.core._multiarray_umath" sources
            Traceback (most recent call last):
              File "/data/data/com.termux/files/usr/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
                main()
              File "/data/data/com.termux/files/usr/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
                json_out['return_val'] = hook(**hook_input['kwargs'])
              File "/data/data/com.termux/files/usr/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
                return _build_backend().build_wheel(wheel_directory, config_settings,
              File "/data/data/com.termux/files/usr/tmp/pip-build-env-qd9wz6jq/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 230, in build_wheel
                return self._build_with_temp_dir(['bdist_wheel'], '.whl',
              File "/data/data/com.termux/files/usr/tmp/pip-build-env-qd9wz6jq/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 215, in _build_with_temp_dir
                self.run_setup()
              File "/data/data/com.termux/files/usr/tmp/pip-build-env-qd9wz6jq/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 267, in run_setup
                super(_BuildMetaLegacyBackend,
              File "/data/data/com.termux/files/usr/tmp/pip-build-env-qd9wz6jq/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 158, in run_setup
                exec(compile(code, __file__, 'exec'), locals())
              File "setup.py", line 494, in <module>
                setup_package()
              File "setup.py", line 486, in setup_package
                setup(**metadata)
              File "/data/data/com.termux/files/usr/tmp/pip-install-9m104c90/numpy_0de947bf681d4f19a814efad3b9cb55d/numpy/distutils/core.py", line 169, in setup
                return old_setup(**new_attr)
              File "/data/data/com.termux/files/usr/tmp/pip-build-env-qd9wz6jq/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
                return distutils.core.setup(**attrs)
              File "/data/data/com.termux/files/usr/lib/python3.9/distutils/core.py", line 148, in setup
                dist.run_commands()
              File "/data/data/com.termux/files/usr/lib/python3.9/distutils/dist.py", line 966, in run_commands
                self.run_command(cmd)
              File "/data/data/com.termux/files/usr/lib/python3.9/distutils/dist.py", line 985, in run_command
                cmd_obj.run()
              File "/data/data/com.termux/files/usr/tmp/pip-build-env-qd9wz6jq/overlay/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 299, in run
                self.run_command('build')
              File "/data/data/com.termux/files/usr/lib/python3.9/distutils/cmd.py", line 313, in run_command
                self.distribution.run_command(command)
              File "/data/data/com.termux/files/usr/lib/python3.9/distutils/dist.py", line 985, in run_command
                cmd_obj.run()
              File "/data/data/com.termux/files/usr/tmp/pip-install-9m104c90/numpy_0de947bf681d4f19a814efad3b9cb55d/numpy/distutils/command/build.py", line 62, in run
                old_build.run(self)
              File "/data/data/com.termux/files/usr/lib/python3.9/distutils/command/build.py", line 135, in run
                self.run_command(cmd_name)
              File "/data/data/com.termux/files/usr/lib/python3.9/distutils/cmd.py", line 313, in run_command
                self.distribution.run_command(command)
              File "/data/data/com.termux/files/usr/lib/python3.9/distutils/dist.py", line 985, in run_command
                cmd_obj.run()
              File "/data/data/com.termux/files/usr/tmp/pip-install-9m104c90/numpy_0de947bf681d4f19a814efad3b9cb55d/numpy/distutils/command/build_src.py", line 144, in run
                self.build_sources()
              File "/data/data/com.termux/files/usr/tmp/pip-install-9m104c90/numpy_0de947bf681d4f19a814efad3b9cb55d/numpy/distutils/command/build_src.py", line 161, in build_sources
                self.build_extension_sources(ext)
              File "/data/data/com.termux/files/usr/tmp/pip-install-9m104c90/numpy_0de947bf681d4f19a814efad3b9cb55d/numpy/distutils/command/build_src.py", line 318, in build_extension_sources
                sources = self.generate_sources(sources, ext)
              File "/data/data/com.termux/files/usr/tmp/pip-install-9m104c90/numpy_0de947bf681d4f19a814efad3b9cb55d/numpy/distutils/command/build_src.py", line 378, in generate_sources
                source = func(extension, build_dir)
              File "numpy/core/setup.py", line 513, in generate_config_h
                check_math_capabilities(config_cmd, ext, moredefs, mathlibs)
              File "numpy/core/setup.py", line 176, in check_math_capabilities
                raise SystemError("One of the required function to build numpy is not"
            SystemError: One of the required function to build numpy is not available (the list is ['sin', 'cos', 'tan', 'sinh', 'cosh', 'tanh', 'fabs', 'floor', 'ceil', 'sqrt', 'log10', 'log', 'exp', 'asin', 'acos', 'atan', 'fmod', 'modf', 'frexp', 'ldexp']).
            [end of output]

        note: This error originates from a subprocess, and is likely not a problem with pip.
        ERROR: Failed building wheel for numpy
      Failed to build numpy
      ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

It seems that you hit discrepancy between setup.py and requirements.txt dependencies. Latter looks more valid, but also depends on recent pip version to install pyqt correctly. Pip 20.0.2 fails, but 22.3.1 works well

@BubaVV I cannot run a GUI using python on termux. So I remove pyqt5 line from requirements.txt also:
nano /data/data/com.termux/files/usr/lib/python3.9/site-packages/pyequion2/__init__.py
#from .interface import InterfaceSystem
#from .gui import run as rungui

After that I did got along with the installation.

I cannot run a GUI using python on termux

Looks like good feature request to implement package options, like pyequion[gui]

Yes, the previous release was non-GUI so no problem. But I can run some examples that do not require GUI. Can I request some more examples non-GUI based

@defencedog Created PR making GUI optional requirement. It should work in Termux environment
https://github.com/pyequion/pyequion2/pull/4

Worked like charm
~/pyequion2 $ pip install .
Processing /data/data/com.termux/files/home/pyequion2
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy>=1.21.4 in /data/data/com.termux/files/usr/lib/python3.9/site-packages (from pyequion2==<IP_ADDRESS>) (1.23.4)
Collecting auto-mix-prep>=0.2.0 (from pyequion2==<IP_ADDRESS>)
  Downloading auto-mix-prep-0.2.0.tar.gz (2.9 kB)
  Preparing metadata (setup.py) ... done
Collecting cloudpickle>=2.0.0 (from pyequion2==<IP_ADDRESS>)
  Downloading cloudpickle-3.0.0-py3-none-any.whl.metadata (7.0 kB)
Requirement already satisfied: commentjson>=0.9.0 in /data/data/com.termux/files/usr/lib/python3.9/site-packages (from pyequion2==<IP_ADDRESS>) (0.9.0)
Requirement already satisfied: matplotlib>=3.5.0 in /data/data/com.termux/files/usr/lib/python3.9/site-packages (from pyequion2==<IP_ADDRESS>) (3.6.0)
Requirement already satisfied: ordered-set>=4.0.2 in /data/data/com.termux/files/usr/lib/python3.9/site-packages (from pyequion2==<IP_ADDRESS>) (4.1.0)
Collecting periodictable>=1.6.0 (from pyequion2==<IP_ADDRESS>)
  Downloading periodictable-1.6.1-py2.py3-none-any.whl (752 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 752.5/752.5 kB 2.1 MB/s eta 0:00:00
Requirement already satisfied: scipy>=1.7.3 in /data/data/com.termux/files/usr/lib/python3.9/site-packages (from pyequion2==<IP_ADDRESS>) (1.8.0)
Collecting pydub (from auto-mix-prep>=0.2.0->pyequion2==<IP_ADDRESS>)
  Downloading pydub-0.25.1-py2.py3-none-any.whl (32 kB)
Requirement already satisfied: lark-parser<0.8.0,>=0.7.1 in /data/data/com.termux/files/usr/lib/python3.9/site-packages (from commentjson>=0.9.0->pyequion2==<IP_ADDRESS>) (0.7.8)
Requirement already satisfied: contourpy>=1.0.1 in /data/data/com.termux/files/usr/lib/python3.9/site-packages (from matplotlib>=3.5.0->pyequion2==<IP_ADDRESS>) (1.0.5)
Requirement already satisfied: cycler>=0.10 in /data/data/com.termux/files/usr/lib/python3.9/site-packages (from matplotlib>=3.5.0->pyequion2==<IP_ADDRESS>) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in /data/data/com.termux/files/usr/lib/python3.9/site-packages (from matplotlib>=3.5.0->pyequion2==<IP_ADDRESS>) (4.37.3)
Requirement already satisfied: kiwisolver>=1.0.1 in /data/data/com.termux/files/usr/lib/python3.9/site-packages (from matplotlib>=3.5.0->pyequion2==<IP_ADDRESS>) (1.4.4)
Requirement already satisfied: packaging>=20.0 in /data/data/com.termux/files/usr/lib/python3.9/site-packages (from matplotlib>=3.5.0->pyequion2==<IP_ADDRESS>) (21.3)
Requirement already satisfied: pillow>=6.2.0 in /data/data/com.termux/files/usr/lib/python3.9/site-packages (from matplotlib>=3.5.0->pyequion2==<IP_ADDRESS>) (9.2.0)
Requirement already satisfied: pyparsing>=2.2.1 in /data/data/com.termux/files/usr/lib/python3.9/site-packages (from matplotlib>=3.5.0->pyequion2==<IP_ADDRESS>) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7 in /data/data/com.termux/files/usr/lib/python3.9/site-packages (from matplotlib>=3.5.0->pyequion2==<IP_ADDRESS>) (2.8.2)
Requirement already satisfied: six>=1.5 in /data/data/com.termux/files/usr/lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib>=3.5.0->pyequion2==<IP_ADDRESS>) (1.16.0)
Downloading cloudpickle-3.0.0-py3-none-any.whl (20 kB)
Building wheels for collected packages: pyequion2, auto-mix-prep
  Building wheel for pyequion2 (pyproject.toml) ... done
  Created wheel for pyequion2: filename=pyequion2-<IP_ADDRESS>-py3-none-any.whl size=94792 sha256=3f995ae54bcc836073d577010ffd571ebc4e113fe3c11b686be4dc517b83ca41
  Stored in directory: /data/data/com.termux/files/usr/tmp/pip-ephem-wheel-cache-i80g2szi/wheels/76/92/06/505db0a50c9642f71f05d6f2964389fc5071019bbb65b70d92
  Building wheel for auto-mix-prep (setup.py) ... done
  Created wheel for auto-mix-prep: filename=auto_mix_prep-0.2.0-py3-none-any.whl size=3474 sha256=272c66f34c610e5cd25cad1f86e50ed3344bb2bdc139c5a7f8472614d66c52e3
  Stored in directory: /data/data/com.termux/files/home/.cache/pip/wheels/7e/47/ec/0d009666e22721445062e1cab7f036f2f9f8b8b5d99714fcde
Successfully built pyequion2 auto-mix-prep
Installing collected packages: pydub, periodictable, cloudpickle, auto-mix-prep, pyequion2
Successfully installed auto-mix-prep-0.2.0 cloudpickle-3.0.0 periodictable-1.6.1 pydub-0.25.1 pyequion2-<IP_ADDRESS>


ci/cd: build example fix

:tada: This PR is included in version 0.2.3 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Empty screen if try to Edit schema of proto type

Describe the bug
(A clear and concise description of what the bug is.)
There is an empty screen when you try to edit the schema of proto type.
Set up
(How do you run the app?
Which version of the app are you running? Provide either docker image version or check commit hash at the top left corner. We won't be able to help you without this information.)
http://dependabot-npm-and-yarn-kafka-ui.internal.kafka-ui.provectus.io/
Steps to Reproduce
Steps to reproduce the behavior:

Navigate to Schema Registry
Click the schema with a name containing 'proto'
Press the button Edit Schema
Observe the empty screen with the right URL (see attachment)

Expected behavior
(A clear and concise description of what you expected to happen)
Edit screen for the schema.
Screenshots
(If applicable, add screenshots to help explain your problem)

Additional context
(Add any other context about the problem here)
Works correctly for other schema types.

No failed backend requests

A duplicate of #1188


allow complex searches using "and" and "or"

Trying to find articles which tackle a combination of words is a nightmare here. Trying to find articles which tell me how to dock a window in Visual Studio, I want a combination of "dock" "AND "window" AND "Visual studio". Put even the first two into the search string and you get every single article which has any of these words.
Or is there some refined search engine hiding somewhere?

⚠ Idea migrated from UserVoice
Created By: PeterT
Created On: 2017/12/18 12:34:43 +0000
Votes at Migration: 3
Supporters at Migration: 3

The articles having any of these words will be returned, but search engine does have the logic to rank them by similarity from the search query. So an article having all these words "dock", "window", "visual", and "studio" will definitely be at better position comparing to others just having one or two words matched. Searching in Docs entire documents can be the main reason in this case because there have some many articles having all these words. Suggest searching in "VS IDE" scope, which will return much better results. On the other hand, we have a feature work item (planning phase) of adding filters with facets in search results, which later will help on this scenario.

@log-suggestion @presleyy

Site search does consider both "and" and "or" when looking for relevant content. What can help the scenario here is to have a search scope like "VS IDE" - which could return more relevant results. Currently we don't allow users to change the scope or filter by product/service scope. We have the feature in planning phase about this.

Hello - reviewing and cleaning up some older feedback/issues. Thanks!


Check that all source files are using strict mode

Coverage remained the same at 88.03% when pulling 45bb3eb651792b7bacfccba19282f121d7e484c9 on zkat/strict-test into 021bd593501443fd671a2e7d4f01a0bf111bba5f on latest.

Coverage remained the same at 88.03% when pulling a331c0e18767a6f766bbb5ed0d95781d1fab8197 on zkat/strict-test into 021bd593501443fd671a2e7d4f01a0bf111bba5f on latest.

Coverage remained the same at 88.03% when pulling d14edb50be7547439a9f634e4dfa5db81507ccc7 on zkat/strict-test into 021bd593501443fd671a2e7d4f01a0bf111bba5f on latest.

Coverage remained the same at 88.03% when pulling 60a9d4965785d1ffd9aa1d10a3715c13ca389737 on zkat/strict-test into 021bd593501443fd671a2e7d4f01a0bf111bba5f on latest.


feat: support Lima VMs iptables rules

New Features:

Adds support for Lima-based VM by added a necessary iptables rule

Caveats:

For Colima, still need to manually symlink /var/run/docker.sock

TODO:

Update documentation

any updates on this?

I wanted to add my experience using colima. Since v0.6.0 colima uses ubuntu as the base vm os, which fixes a bunch of issues related to kind. docker-mac-net-connect works as is with the addition of running sudo iptables -A FORWARD -s <IP_ADDRESS> -j ACCEPT when ssh'd into the vm.

Guys, this would be great as it would allow to provide out of the box support to folks that don't want to use Docker Desktop.
Does this require any further testing?

@gregnr Is this something that you see landing in this project? Anything that still needs tested, etc. as part of this (or future) PRs?


Add Tech University of Korea

@moonsae I am sorry, the domain tukorea.ac.kr is removed from our trusted domains list because it was abused in the past. If you are the student of this school, please enroll to GitHub Student Developer Program. Then apply for JetBrains educational subscription with your GitHub account (GitHub tab at https://www.jetbrains.com/shop/eform/students).


Tuple of strings as sender

As of blinker 1.3 we can now reliably use strings as senders, but it seems that the next logical step, using a tuple of strings, does not work reliably.
if I do:
def blah(sender, **kwargs):
    print sender, kwargs

s = blinker.Signal()

s.connect(blah, ('ab', 'cde'))
s.send(('ab', 'cde'))

The call to s.send(('ab', 'cde')) will result in blah() sometimes being called, sometimes not.
Using a tuple of strings seems, to me at least, a nice way to have a finer grained signal subscription mechanism without having separate signals.

Unfortunately I don't see supporting this at the moment- chasing down interned types in structures seems like an endless rabbit hole.


Added drivers for the fxls8974 accelerometer

Added drivers for the fxls8974 accelerometer, tested over I2C on the mimxrt1040_evk board.

While fixing bugs I found TODO comment in samples/sensor/sensor_shell/sample.yml which should've been deleted since the #63414 is resolved.


Question: is this plugin possible in Firefox using their CDP?

Hello - firstly, I'm well aware that this project only works in Chromium and it says so in the readme. My understanding is that this is because the project uses the Chrome Devtools Protocol to fire the events in a trusted way.
Firefox has some support for CDP though it is incomplete, or "half-baked" as claimed over in SeleniumHQ with Firefox's focus being on WebDriver BiDi.
Two questions:

Is this plugin even possible with the current CDP implementation in Firefox
Would the plugin be able to use BiDi instead?

No and no
We are reusing the same connection that cypress maintains and it is doing this only for a chrome browser


Question about scaling DTU intrinsics

Hi, I was wondering what the following comment means:
https://github.com/FangjinhuaWang/PatchmatchNet/blob/65ff2b452da5dbecdee92eaa2ab1fb7fea5cb447/datasets/dtu_yao.py#L170
It seems that you assume intrinsics in training file to be already downscaled by a factor of 4, but in your preprocessed DTU datasets, intrinsics are as follows:
intrinsic
2892.33 0 823.205
0 2883.18 619.072
0 0 1
which match the ones in the MVSNet repo and they seem to match the original resolution of 1600 x 1200. I checked the other datasets and you correctly do:
intrinsics[:2,:] *= 0.125
at stage 3, rather than *= 0.5 as you do for DTU. Could you please clarify on this point? I'm asking since I'm adding support for training on BlendedMVS and I would like to make things right. Thank you in advance!

Hi,
The training camera parameters that is read in is here (https://github.com/FangjinhuaWang/PatchmatchNet/blob/65ff2b452da5dbecdee92eaa2ab1fb7fea5cb447/datasets/dtu_yao.py#L162), which is already scaled for the image size of 1/4.

Hi, thanks for the answer. So the following intrinsics (e.g from Cameras_1/train/00000000_cam.txt)::
intrinsic
2892.33 0 823.205
0 2883.18 619.072
0 0 1
are not expressed in unit pixels, right? If you consider full size to be 640 x 512 and 1/4 size to be 160 x 128, they seem to be expressed in "units" of 0.1 pixels. Otherwise the principal point would be outside the image plane.

The camera file that I provide "Cameras_1/train" is the same as MVSNet, it looks like:
intrinsic
361.54126 0.0 82.90063
0.0 360.3975 66.38387
0.0 0.0 1.0

Sorry, you are right. I got confused by the layout of the folder. Closing as solved.


LibCrypto+LibTLS+LibWeb: Get closer to SECP521r1 support

This moves the code closer to supporting P521 across the board. This many changes are required because P521 keys cannot be represented as full bytes or full words, but need to be serialized as 66 bytes.
The current implementation of SECPxxxr1 is not able to do calculations on P-521 for some reason that I was not able to figure out. The first failure point is saying that the generator is not on the curve, which clearly denotes a problem with the math. I think that the problem lays in the fact that AK::UFixedBigInt does not handle 521 bits correctly.
Gets a few tests here and there by not failing early and by serializing keys of the correct size.

Test failure seems unrelated to the PR.

Thank you!


Implement Unit testing

implement unit testing

Check with already generated file or maybe run via dot? whichever is faster and convenient.


Fix Dracula color scheme spelling

https://draculatheme.com/

This is not a mistake; see #216 where the colorscheme was added.

Note that the lightline theme for Dracula is distributed at https://github.com/dracula/vim. See #299 for reference.

This is not a mistake; see #216 where the colorscheme was added.

Sorry my bad, they look similar.

Note that the lightline theme for Dracula is distributed at https://github.com/dracula/vim. See #299 for reference.

Thank you!


C ++ nn :: conv2d Can you take a look at the problem?

Hi @fmassa
I think there is an error in the NN module. Can you confirm it?
C ++ nn :: conv2d Can you take a look at the problem?
https://github.com/pytorch/pytorch/issues/21332

Hi,
I replied in the original issue. This seems to be unrelated to torchvision, so I'm closing this issue


fix(backend): Remove cf schema validation blocking addition of default: 0 scores

Pull Request
Purpose
Remove the custom format schema validation which blocks default: 0 scores. This is because we need to add a few default: 0 scores in service of #2143
Approach
Remove the part of the schema which blocks these scores
Open Questions and Pre-Merge TODOs
None
If you're adding a new Custom Format, make sure you follow the Radarr/Sonarr Custom Format (JSON) Guidelines. -->
Requirements

[x] These changes meet the standards for contributing.
[x] I have read the code of conduct.

This is no longer required


Suggestion: Add a child() method to quickly retrieve a property as observable

Having to manually get a new FirebaseObservable from a child property is a bit cumbersome. While it's a best practice to denormalize your database, there will be some cases with nested data:
object: FirebaseObjectObservable<any>;
list: FirebaseListObservable<any>;

let object = this.af.database.object('/object');
let list = this.af.database.list('object/childlist');

What if we could reference this property and return it as it's own observable:
object: FirebaseObjectObservable<any>;
list: FirebaseListObservable<any>;

let object = this.af.database.object('/object');
let list = object.child('childlist');

We could have a distinction between returning the observable as FirebaseListObservable / FirebaseObjectObservable
object: FirebaseObjectObservable<any>;

let listObservable = object.childList('childlist');
let objectObservable = object.childObject('childlist');

With this we could pass a new Observables much easier. Suppose we get a FirebaseListObservablereference and want to get one of the childs and return it as a FirebaseObjectObservable. We'll do this by getting the $key value from the list item and using that to get the full URI for the resource:
let listObjectObservable = this.af.database.object('/list/' + listObject.$key);

Instead we could just get the list and use the new childObject() method:
list: FirebaseListObservable<any>;
object: FirebaseObjectObservable<any>;

let object = list.childObject(listObject.$key);

So basically the child() method would make it a bit quicker to get the property as a observable. I guess using the current object's firebaseref and amending to that would be a simple solution.
But are there any pitfalls performance wise or is there a fundamental problem with my thinking?
Perhaps there are better ways to do this.
In any case thanks for the great library: it makes it very easy to handle Firebase related tasks!

This would a) encourage nesting data and b) encourage synchronizing the same data twice (the child list is already present and synchronized in the parent object). We removed child() from AngularFire back in alpha, years ago, to discourage this and it saved developers a great deal of pain. I'm not in favor of adding it back in now.

I get that nesting large objects isn't the best method but we do exactly what @antti-pyykkonen mentioned:
library: FirebaseObjectObservable<any>;
itemsAf: FirebaseListObservable<any>;

constructor(private af:AngularFire) {
  this.library = this.af.database.object('path/to/lib/id');
  this.itemsAf = this.af.database.list('path/to/lib/id/items')

Library Name: {{(library |async)?.name}}

  {{item.name}}

When Angular could loop object properties it was no big deal but now it has to be an array and unpacking it requires a subscription. It was cheaper/faster to just sub twice.
Our data looks like:
libraries: {
    $key: {
        name: 'myLib',
        items: {
            $key:{name:'itemName'}
        }
    }
}

I know large scale nesting isn't the best plan but:
libraries: {
    $key: {
       name: 'myLib'
    }
}
items: {
    $libraryKey: {
        $itemKey: {name:'itemName', library:'libraryKey'}
    }
}

Would still require two bindings as well and seems like a lot of work, especially once we start adding in security rules and whatnot..
Id love to be able to do:
var library = this.af.database.list('library');
var items = library.getChild('items',true); //listObservable

function getItem(key:string) {
  return items.getChild(key); //objectObservable
}

I guess we'll have to find other ways to do this. So are there any good practices on this?
For example: would recommend to map additional Observable properties to a FirebaseListObservable like seen here.
Thanks for the responses!

So it is suggested to have rather 3 independent collections eg
Slideshow
Slideshow
Slides

than nesting :
Slideshows / Slideshow / Slides
?

hi all,
I am beginner firebase
how to add child in firebase using angular fir2
kindly advice me,
Thanks & regarda

I used $ref.child() method available within this.af.database.object().


Teams app remove command added

PR contains the graph teams app remove [options] command + documentation.

Coverage remained the same at 100.0% when pulling 7971723ec9d5373707ea5a5baf4c953bf68e4323 on estruyf:teams-app-remove into 6200f63f350ef22779f70e1ce3a886d97f796841 on pnp:dev.

Hero!

Merged manually. Thanks! 💪


feat: update to eye v3.12.0

feat: update to eye v3.12.0

:tada: This PR is included in version 4.11.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release
v4.11.0

Your semantic-release bot :package::rocket:


Post reassignment of jira details of the issue gives stale data.

A jira was created with id jiraId and reassigned from personX to personY.
details of the issue jiraId returns assignee as personX instead of personY

@DivakarUngatla I would like to pick it up.


Update to current tab name

@zoran-rilak-msft
Can you review the proposed changes?
When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team


PowerAccent: make All collect from Lang functions rather than return …

…from manually updated list
Summary of the Pull Request
I noticed the Currency language entries weren't included in the All Available list, then it dawned on me that maintaining multiple manually entered lists was not ideal. I don't know C# but I figured Union() would help make sure only one copy of each character showed up, but then All would be all.
PR Checklist
N/A (none of them seemed fitting, other than maybe tests but I'm not set up for that)
Validation Steps Performed
I read it real careful - sorry I can't do more than that

@microsoft-github-policy-service agree [company="default"]
Options:default
I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.
@microsoft-github-policy-service agree
(when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.
@microsoft-github-policy-service agree company="Microsoft"
---- On Sat, 26 Aug 2023 10:07:38 -0400 microsoft-github-policy-service[bot] @.***> wrote ---
https://github.com/dannysummerlin please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
@microsoft-github-policy-service agree [company="{your company}"]
Options:
(default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.
@microsoft-github-policy-service agree
(when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.
@microsoft-github-policy-service agree company="Microsoft"
Contributor License AgreementContribution License Agreement
This Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
and conveys certain license rights to Microsoft Corporation and its affiliates (“Microsoft”) for Your
contributions to Microsoft open source projects. This Agreement is effective as of the latest signature
date below.
Definitions.
“Code” means the computer software code, whether in human-readable or machine-executable form,
that is delivered by You to Microsoft under this Agreement.
“Project” means any of the projects owned or managed by Microsoft and offered under a license
approved by the Open Source Initiative (http://www.opensource.org).
“Submit” is the act of uploading, submitting, transmitting, or distributing code or other content to any
Project, including but not limited to communication on electronic mailing lists, source code control
systems, and issue tracking systems that are managed by, or on behalf of, the Project for the purpose of
discussing and improving that Project, but excluding communication that is conspicuously marked or
otherwise designated in writing by You as “Not a Submission.”
“Submission” means the Code and any other copyrightable material Submitted by You, including any
associated comments and documentation.
Your Submission. You must agree to the terms of this Agreement before making a Submission to any
Project. This Agreement covers any and all Submissions that You, now or in the future (except as
described in Section 4 below), Submit to any Project.
Originality of Work. You represent that each of Your Submissions is entirely Your original work.
Should You wish to Submit materials that are not Your original work, You may Submit them separately
to the Project if You (a) retain all copyright and license information that was in the materials as You
received them, (b) in the description accompanying Your Submission, include the phrase “Submission
containing materials of a third party:” followed by the names of the third party and any licenses or other
restrictions of which You are aware, and (c) follow any other instructions in the Project’s written
guidelines concerning Submissions.
Your Employer. References to “employer” in this Agreement include Your employer or anyone else
for whom You are acting in making Your Submission, e.g. as a contractor, vendor, or agent. If Your
Submission is made in the course of Your work for an employer or Your employer has intellectual
property rights in Your Submission by contract or applicable law, You must secure permission from Your
employer to make the Submission before signing this Agreement. In that case, the term “You” in this
Agreement will refer to You and the employer collectively. If You change employers in the future and
desire to Submit additional Submissions for the new employer, then You agree to sign a new Agreement
and secure permission from the new employer before Submitting those Submissions.
Licenses.
Copyright License. You grant Microsoft, and those who receive the Submission directly or
indirectly from Microsoft, a perpetual, worldwide, non-exclusive, royalty-free, irrevocable license in the
Submission to reproduce, prepare derivative works of, publicly display, publicly perform, and distribute
the Submission and such derivative works, and to sublicense any or all of the foregoing rights to third
parties.
Patent License. You grant Microsoft, and those who receive the Submission directly or
indirectly from Microsoft, a perpetual, worldwide, non-exclusive, royalty-free, irrevocable license under
Your patent claims that are necessarily infringed by the Submission or the combination of the
Submission with the Project to which it was Submitted to make, have made, use, offer to sell, sell and
import or otherwise dispose of the Submission alone or with the Project.
Other Rights Reserved. Each party reserves all rights not expressly granted in this Agreement.
No additional licenses or rights whatsoever (including, without limitation, any implied licenses) are
granted by implication, exhaustion, estoppel or otherwise.
Representations and Warranties. You represent that You are legally entitled to grant the above
licenses. You represent that each of Your Submissions is entirely Your original work (except as You may
have disclosed under Section 3). You represent that You have secured permission from Your employer to
make the Submission in cases where Your Submission is made in the course of Your work for Your
employer or Your employer has intellectual property rights in Your Submission by contract or applicable
law. If You are signing this Agreement on behalf of Your employer, You represent and warrant that You
have the necessary authority to bind the listed employer to the obligations contained in this Agreement.
You are not expected to provide support for Your Submission, unless You choose to do so. UNLESS
REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING, AND EXCEPT FOR THE WARRANTIES
EXPRESSLY STATED IN SECTIONS 3, 4, AND 6, THE SUBMISSION PROVIDED UNDER THIS AGREEMENT IS
PROVIDED WITHOUT WARRANTY OF ANY KIND, INCLUDING, BUT NOT LIMITED TO, ANY WARRANTY OF
NONINFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
Notice to Microsoft. You agree to notify Microsoft in writing of any facts or circumstances of which
You later become aware that would make Your representations in this Agreement inaccurate in any
respect.
Information about Submissions. You agree that contributions to Projects and information about
contributions may be maintained indefinitely and disclosed publicly, including Your name and other
information that You submit with Your Submission.
Governing Law/Jurisdiction. This Agreement is governed by the laws of the State of Washington, and
the parties consent to exclusive jurisdiction and venue in the federal courts sitting in King County,
Washington, unless no federal subject matter jurisdiction exists, in which case the parties consent to
exclusive jurisdiction and venue in the Superior Court of King County, Washington. The parties waive all
defenses of lack of personal jurisdiction and forum non-conveniens.
Entire Agreement/Assignment. This Agreement is the entire agreement between the parties, and
supersedes any and all prior agreements, understandings or communications, written or oral, between
the parties relating to the subject matter hereof. This Agreement may be assigned by Microsoft.
—
Reply to this email directly, https://github.com/microsoft/PowerToys/pull/28164#issuecomment-1694348993, or https://github.com/notifications/unsubscribe-auth/AFFRHZHEBSCZKJZLZR6526TXXH7KVANCNFSM6AAAAAA37T3PXY.
You are receiving this because you were mentioned.

@microsoft-github-policy-service agree
---- On Sat, 26 Aug 2023 10:10:26 -0400 microsoft-github-policy-service[bot] @.***> wrote ---
https://github.com/dannysummerlin the command you issued was incorrect. Please try again.
Examples are:
@microsoft-github-policy-service agree
and
@microsoft-github-policy-service agree company="your company"
—
Reply to this email directly, https://github.com/microsoft/PowerToys/pull/28164#issuecomment-1694349544, or https://github.com/notifications/unsubscribe-auth/AFFRHZBQG56ICV3777RELLTXXH7VFANCNFSM6AAAAAA37T3PXY.
You are receiving this because you were mentioned.

@dannysummerlin
If there isn't an issue for this please create a new one. We need it for organization and tracking of the work/changes.

Sure, no problem - resolves #23180


vCheck is not running as scheduled task

Hi Alan.
Need your Help.
I am trying to run vCheck as scheduled task. I have created the task in the scheduler to run daily. The task gets started and shows as running and never gives the output. I had to forcefully end the task after 1 hours. Below is how my action window looks in the task scheduler.

My environment:
vCheck v6.25
Powershell 5.1.14409
PowerCLI 6.3 Realease 2
vCenter 6.0 Update3
Aside from this I can run the batch file manually and its working fine and its not asking password during script running . also note schedule task created with same user and running with same user.
Batch file contains ( C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -noe -c "."C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI\Scripts\Initialize-PowerCLIEnvironment.ps1"";D:\Ankoji_Scripts\vSpherevCheck\vcheck.ps1")

Anyone Please help on this issue

Had the same issue when trying to run the task under SYSTEM. Set it to use a service account and that got it running at least. Still not functional as the report is empty but I am getting there.
I believe the issue is how it loads the PowerCLI module and that it requires a user environment to run.

To troubleshoot this you can amend the vCheck.ps1 as follows:
At this point:
$vCheckVersion = "6.25"
$Date = Get-Date

Add this line:
Start-Transcript -Path ('PATH-TO-LOG-FILE-vCheckTask.{1:yyyy-MM-dd}.log' -f $env:temp, $Date)
Then at the very bottom of the script add the following:
Stop-Transcript
Examining the log output will probably tell you why it's failing. For me it was failing to log into vCenter as I was running on another server under System

@gregharms Is it possible to run the check using system user? If not then why it is not possible to run the check as system user?

I am having an issue getting vCheck to run as a scheduled task.  I have a Service Account setup and have given that user Admin rights on the server.
My environment:
2 vCenter currently (Enhanced Linked Mode)
vCenter 6.7
vSphere 6.5
PowerCLI 11.5
PowerShell 5.1
Windows 2019
If I run it manually it works as expected.  When I try to run it as a Scheduled Task, it says ran successfully, but I get nothing from it.  I am sure that where I am having the issue is in the Action part of the task.
Program/Script: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
Add Argument: -c ". "C:\Program Files (x86)\VMware\Infrastructure\PowerCLI\Scripts\Initialize-PowerCLIEnvironment.ps1" $true" "C:\PowerShell-Scripts\vCheck-vSphere-master\vcsa-lab\vCheck.ps1"
I have tried changing the Add Argument with the -PSConsoleFile and vim.psc1, but that doesn't work at all.
I am not sure how to get it to run.  I don't want to run it manually everyday.

I was able to get it to work by putting it in a batch file and running it.

pardon the necro-bump.
@Scepticalist 's comment was very useful.
No matter what i did. my old copy of vCheck failed on Win11 even though it worked on Win10 verbatim for years.
Even via exporting/importing the Task Scheduler XML and using the same domain account, the transcript-log reported access denied no matter what i did.
To circumvent the problem, I added the following code near the beginning of vCheck.ps1 which worked.
Import-Module -Global VMware.VimAutomation.Core | Out-Null
$vCenter="vCenter.domain.local"    ## your vCenter host name
Disconnect-VIServer * -Confirm:$false -Force | Out-Null
Connect-VIServer -Server $vCenter -Protocol https -Force

I cannot determine why 00 Initialize/00 Connection Plugin for vCenter.ps1 is not working as expected as there were no changes whatsoever.  Of course, due to my workaround,  I removed my $Server variable from said plugin.
I hope this helps someone in lieu of no vCheck being stale (https://github.com/alanrenouf/vCheck-vSphere/issues/746) . or maybe someone might point out some other resolution not thought of.   Thank you.


How to run C++ Native Modules like React-Native do in Flutter

Use case
React-Native have its own way to  reuse some existing Objective-C, Swift or C++ code in app:
https://facebook.github.io/react-native/docs/native-modules-ios.html
Proposal
How to run C++ Native Modules like React-Native do  in Flutter ?

For now, we have to use platform channel, that is
C/C++ lib <---binding---> Swift/ObjC <---platform channel---> Dart
C/C++ lib <-----JNI-----> Java/Kotlin <---platform channel---> Dart
For direct access, follow https://github.com/flutter/flutter/issues/7053

Thank you @truongsinh ! Closing as a duplicate, please follow #7053.


float exponentiation for gates and integer exponentiation for instructions

Fixes #2805
Fixes #2810
~This PR allows integer exponentiation using instruction.gate. I did not fully implement rational exponentiation because a root method is needed (#2810). Adding it here it seems too intrusive (and I'm not fully sure how to do it :-/)~

Calling integer exponentiation Instruction.power sounds strange for me. Why don't we call it Instruction.repeat (It can have the same implementation as Gate, different from power)?

Calling integer exponentiation Instruction.power sounds strange for me. Why don't we call it Instruction.repeat (It can have the same implementation as Gate, different from power)?

What it means Instruction.repeat(-2)? If you need repeat, it can be added. But it's not exactly the same...

Calling integer exponentiation Instruction.power sounds strange for me. Why don't we call it Instruction.repeat (It can have the same implementation as Gate, different from power)?

What it means Instruction.repeat(-2)? If you need repeat, it can be added. But it's not exactly the same...

No. I means Instruction.repeat defined for positive integer would be enough and better. What Instruction.power(-2) means? I need Gate.power(-2), but I don't understand the meaning and necessity of Instruction.power(-2).

Right now, custom gates are instruction (to be changed, #2896). Instruction.power(-2) of -S-S- should be -Sdg-Sdg-Sdg-Sdg-. Or am I missing something?

I think it’s okay for now for CompositeInstructions to only support repeat. When #2896 makes CompositeGates (including implementing their to_matrix), they can implement a power method. (This will also probably require moving Barrier from an Instruction to a Directive.)
But for now, Instructions having repeat and Gates having power is consistent with both how things are laid out now, and the direction we’d like to head.

from @ajavadia

yeah i think repeat and power are both valuable. repeat just saves you from writing a loop. power is smart. We should allow gates to override their .power method. Like an Adder(5).power(2) should be Adder(10) not Adder(5).Adder(5)

Looks good, it just has to pass CI.

@ajavadia says:

Looks good, it just has to pass CI.

Done in https://github.com/Qiskit/qiskit-terra/pull/2811/commits/176ac56aab046dc4803498dce5b08861d20b4649


Shifting speed info to new line

Sometimes during downloading appears such effect with shifting speed info to new line and hiding ratio and total size. May be just put speed info on the left (in two lines) and other info on the right? I mean constantly.

The total size on the right shouldn't disapear, normally only the ratio should be elipsed. I will try to fix this in the next version, thanks for reporting this.

And the speed information on two lines is useful only on small screen, as there is enough space on large screen to display it on one line. This allow to have two lines for the torrent name.

Fixed in v1.1.6, now lines are truncated if they are too long.


Add a quiet mode

I'd love a flag that disables printing the following lines:
Starting session with SessionId: <user>-<session>.
Exiting session with sessionId: <user>-<session>.

This was originally requested here:
https://github.com/aws/session-manager-plugin/pull/20#discussion_r1043911507
So I'm just opening a new issue to track this specifically.

@Yangtao-Hua is an env-var based option acceptable?


Image too large. How suppress drawing when using pyWGCNA_obj.preprocess() ?

I am using pyWGCNA on a large data pyWGCNA_obj .
 AnnData object with n_obs × n_vars = 11963 × 28991

There is an Error when I use pyWGCNA_obj.preprocess() .
Image size of 85900x1000 pixels is too large. It must be less than 2^16 in each direction.
<Figure size 85900x1000 with 0 Axes>

How can I suppress drawing the hierarchical clustering plot when using pyWGCNA_obj.preprocess() to resolve the issue?

Hi,
right now, I don't have any parameters to suppress drawing. you can put save to False to avoid saving but it will draw it anyway so I assume it's not gonna solve your problem.
I'll add another parameter for this in the next few days but just make sure you don't have any outliers.

Hi @yizhang-zoey
I added an option (show) to suppress the drawing. Hopefully, that will fix your problem
more info in API doc.

Hi @yizhang-zoey
I added an option (show) to suppress the drawing in v2.0.1. Hopefully, that will fix your problem more info in API doc.

Thanks, it has been fixed!


Shortest Path changes

This PR has all the current changes of Shortest Path including frontend and physical operators.

All front end related changes for shortest path have been removed.


Support .NET 9

What are you trying to accomplish?
Support .NET 9 by installing both the STS and LTS releases of .NET.
Relates to #10496, but doesn't resolve it in the general case (i.e. it won't work for .NET 10).
Anything you want to highlight for special attention from reviewers?
As .NET 9 is in RC and not "official" yet, I figured the least risky option (at the expense of increasing the image size) was to install both .NET 8 and .NET 9. These could then be updated on alternate years (LTS changes to 10 next year, STS changes to 11 in 2026, etc.)
The .NET 9 SDK also makes NuGet Audit enabled by default and breaks package restore when vulnerable packages are in the dependency tree. I've resolved that for building NuGetUpdated, but it might occur when upgrading users' own projects. I couldn't see an obvious place in the code to change the tool to set NuGetAudit=false to avoid that when updating packages. I did do a search for NoWarn, but those all seemed to be in projects being built rather than when invoking the tool.
How will you know you've accomplished your goal?
Dependabot can update NuGet packages for projects targeting net9.0.
Checklist

[x] I have run the complete test suite to ensure all tests and linters pass.
[x] I have thoroughly tested my code changes to ensure they work as expected, including adding additional tests for new functionality.
[x] I have written clear and descriptive commit messages.
[x] I have provided a detailed description of the changes in the pull request, including the problem it addresses, how it fixes the problem, and any relevant details about the implementation.
[x] I have ensured that the code is well-documented and easy to understand.

From the tests Docker build:
#9 [ 4/10] RUN dotnet --list-runtimes
#9 0.185 Microsoft.AspNetCore.App 8.0.8 [/usr/local/dotnet/current/shared/Microsoft.AspNetCore.App]
#9 0.185 Microsoft.AspNetCore.App 9.0.0-rc.1.24452.1 [/usr/local/dotnet/current/shared/Microsoft.AspNetCore.App]
#9 0.185 Microsoft.NETCore.App 8.0.8 [/usr/local/dotnet/current/shared/Microsoft.NETCore.App]
#9 0.185 Microsoft.NETCore.App 9.0.0-rc.1.24431.7 [/usr/local/dotnet/current/shared/Microsoft.NETCore.App]
#9 DONE 0.2s

#10 [ 5/10] RUN dotnet --list-sdks
#10 0.212 8.0.401 [/usr/local/dotnet/current/sdk]
#10 0.212 9.0.100-rc.1.24452.12 [/usr/local/dotnet/current/sdk]
#10 DONE 0.2s

I'm going to move this out of draft so that someone who knows Ruby and the repo layout can point me at where I need to make the changes to make the build happy.

Changing the C# tests to run in the current working directory of NuGetUpdater to force it to use the .NET SDK from global.json instead of rolling forward to the .NET 9 SDK has fixed one of the tests, but 4 tests are failing for NuGetUpdater.Core.Test for some reason I need to dig into.

@brettfo Should I abandon this PR?
It looks like changes you've made in #10521 (specifically this bit) will have the stuff you're working on install the SDK in global.json if it's not installed, and would also resolve #10496 for arbitrary versions of .NET going forwards.

@martincostello I think this PR is still needed.  The changes in #10521 aren't active yet, that's just the skeleton for longer term future work.  I marked one place where a PR that's about to get merged will conflict with this, but it's a minor fix.

@brettfo Thanks - once that's merged I'll rebase this.

@martincostello The other PRs have all been merged.  Can you update this on the latest main?  There should be 2 conflicts: 1 right before ProcessEx.RunAsync is called; the function was changed to take IEnumerable<string> arguments instead of string arguments, and another conflict in nuget/Dockerfile; we're now installing PowerShell and rearranged slightly.

@brettfo Should be good now.


Clean up and setup jupyterbook docs

Overview
This PR setup the docs for echoshader via jupyter book. I've moved the GSoC Contributors guide stuff to docs/source. We can eventually setup RTD for hosting this like echopype.

This is awesome @lsetiawan ! I'll merge so we have the new structure in place for referencing in issues/other docs.


Question about high memory usage?

Hello, i run tdlight-bot-api on my 16gb ram Ubuntu 16 server .
I have a load on my bot and the tdlight-bot-api takes for my bot 5gb of ram .
In my case i use api.telegram.org for sending messages etc, and tdlight-bot-api server for getParticipants and getChat, UserMode ;
can u add a command line option that run the td engine just for calling methods not for storing,sending updates or cache something like file_ids or users data?

Hi, can you please attach logs, graphs or something?
Cc: @Cavallium

https://broadcast.aym9n.xyz/getMemStats.php
https://broadcast.aym9n.xyz/memstats.php (HTTP stats)

Hello, set a webhook to an empty location to allow the update queue to empty, this way you'll save some memory, if you can't set a webhook use getUpdates with offset set to -1 (you'll have to set a cron or do it every request or so depending on the structure of your bot)

Hello, set a webhook to an empty location to allow the update queue to empty, this way you'll save some memory, if you can't set a webhook use getUpdates with offset set to -1 (you'll have to set a cron or do it every request or so depending on the structure of your bot)

What about My suggestion?
Also when calling optimizeMemory after some minutes my tdlight-bot-api server will be down
I call optimizeMemory just now and he stops the tdlight-bot-api server check the uptime here https://broadcast.aym9n.xyz/memstats.php

That's bad. We should do something about it

Leave it here, no problem

OptimizeMemory will be removed with the next update, please restart the api when you want to free the memory.
Sorry but it's unfortunately how bot api works 🤷♂


Last version not published on crates.io

I've just noticed that v0.7.1 version has not been published oncrates.io, but README's badges say differently. Is that an internal choice for the project?

No, it is an internal choice of doing too many things and not having enough time.
Will try to publish by EOW and once done will close this. :)

Ahhh, I see now. Thanks a lot! :)

All published!