Make Cache Invalidation Work per Function Instead of per File

TLDR: A caching issue and potential solution.
Something that is quite annoying is that if you edit anything in a source file every function decorated with @njit(cache=True) will have its cache invalidated, causing all of those functions to recompile. In a large project this can significantly slow down development since you need to wait several seconds or even minutes after every edit. It would be nice if only the functions effected by an edit needed to recompile. I'm especially worried about this issue becoming even more difficult in the near future after this PR (#8396) is implemented which will makes it so that cache invalidations cascades through dependencies. This is an important feature, although I selfishly worry that when this is incorporated nearly my entire project is going to need to unnecessarily recompile every time I tweak a file with any functions used heavily elsewhere in the project, even if the implementation of those dependencies are left unchanged.
I'm hoping to open up a discussion about how this issue could be addressed and offer a prototype solution. A big part of why this issue is hard is that function definitions are not self-contained in their bodies, they can leverage constants and other functions which are in a function's __global__ scope and __closure__. The current solution of timestamping files seems more straightforward than checking the whole __global__ dict for each function. But why not instead just check the elements of __global__  which are referenced in the function, i.e. the elements which overlap with func.__code__.co_names.
Below is a monkey patched solution along these lines. The function is stamped by a hash of its func.__code__.co_code, and the subset of  it's __globals__ dict which is explicitly referenced in the function body.
import cloudpickle
from numba.core.caching import Cache, _SourceFileBackedLocatorMixin, _CacheLocator, _UserProvidedCacheLocator, CompileResultCacheImpl
from numba.core.dispatcher import Dispatcher
from numba import njit
from numba import config
import os
import types as py_types
import functools
from copy import copy
import hashlib

import time
class PrintElapse():
    def __init__(self, name):
        self.name = name
    def __enter__(self):
        self.t0 = time.time_ns()/float(1e6)
    def __exit__(self,*args):
        self.t1 = time.time_ns()/float(1e6)
        print(f'{self.name}: {self.t1-self.t0:.6f} ms')

class _PreciseCacheLocator(_UserProvidedCacheLocator):
    def __init__(self, py_func, py_file):
        self._py_func = py_func
        cache_subpath = self.get_suitable_cache_subpath(py_file)
        self._cache_path = os.path.join(config.CACHE_DIR, cache_subpath)

        with PrintElapse("hashing time"):
            code =  self._py_func.__code__
            glbs = self._py_func.__globals__

            # Get __globals__ referenced in the function body
            _vars = code.co_names
            used_globals = {k:glbs[k] for k in _vars if k in glbs}

            # Hash the function bytecode and subset of globals
            func_bytes = code.co_code + cloudpickle.dumps(used_globals)
            self._func_hash = hashlib.sha256(func_bytes).hexdigest()

    def get_source_stamp(self):
        return self._func_hash

    def get_disambiguator(self):
        return self._func_hash[:10]

    @classmethod
    def from_function(cls, py_func, py_file):
        # if not config.CACHE_DIR:
        #     return
        self = cls(py_func, py_file)
        return self

class PreciseCacheImpl(CompileResultCacheImpl):
    _locator_classes = [
        _PreciseCacheLocator,
        *CompileResultCacheImpl._locator_classes
    ]

class PreciseCache(Cache):
    """
    Implements Cache that saves and loads CompileResult objects.
    """
    _impl_class = PreciseCacheImpl

# Monkey Patch Dispatcher
def enable_caching(self):
    self._cache = PreciseCache(self.py_func)
Dispatcher.enable_caching = enable_caching

We can then execute something like this
GLOBAL_VAL1 = 1
GLOBAL_VAL2 = 2
GLOBAL_VAL3 = 3

@njit(cache=True)
def foo():
    return GLOBAL_VAL1

@njit(cache=True)
def bar():
    return GLOBAL_VAL2

def gen_baz():
    x = GLOBAL_VAL3+0
    @njit(cache=True)
    def baz():
        return x
    return baz

baz = gen_baz()
print(foo(), bar(), baz())
print("FOO", "HIT" if len(foo._cache_misses) == 0 else "MISS")
print("BAR", "HIT" if len(bar._cache_misses) == 0 else "MISS")
print("BAZ", "HIT" if len(baz._cache_misses) == 0 else "MISS")

on the first run the cache misses as expected, and a quick timing of the hashing parts show that this method seems fast enough to be a viable solution.
hashing time: 0.034180 ms
hashing time: 0.012939 ms
hashing time: 0.006592 ms
1 2 3
FOO MISS
BAR MISS
BAZ MISS

Subsequent runs without any edits hit
...
1 2 3
FOO HIT
BAR HIT
BAZ HIT

And if I change any of the  GLOBAL_VAL values then only the functions that depends on those globals will recompile.  This seems to work (at least in this example) even if the jitted function was  produced in a closure.
GLOBAL_VAL1 = 7
GLOBAL_VAL2 = 2
GLOBAL_VAL3 = 8

7 2 8
FOO MISS
BAR HIT
BAZ MISS

I'm curious if something like this has been explored already. Does this seem like it would work in general or am I missing something. Forgive me if I overlooked something in a prior discussion along these lines.
Other affiliated issues/people: #9304 @luk-f-a @stuartarchibald

Many thanks for the detailed report @DannyWeitekamp. This will be discussed at the triage meeting tomorrow (2023-12-05). My initial thought is that this is similar/related to the issue of caching functions that are compiled from the exec of strings, a solution that works for one should work for the other. Creating a cache key based on the hash of the code object should work, the question is potentially how far-reaching (into the program state) such a cache key needs to be and what is a "sensible" behaviour. Whether this is generally preferable behaviour compared to file-at-a-time should probably be discussed more broadly with the Numba user base. Thanks again for looking into this!

So in my efforts to use this in my own projects a few additional considerations came up.

Some special consideration needs to be given to Dispatcher and _Intrinsic dependencies. Just using cloudpickle
to hash these doesn't generate a consistent hash, so some recursive accounting of dependencies is necessary. This would be easy if every Dispatcher had a _cache object, but that isn't always the case. Although it often works well enough just hashing one level out.
Using this hash in get_disambiguator creates an nbi file for each unique recompile, which isn't ideal but honestly isn't much worse than using the line number which in the worst case creates several new indices in cases when you alter something high up in the file and all the line numbers change. It seems like the ideal disambiguator would be something like the n-th time a function with the same name was defined in a file... perhaps that 'n' could be determined by analyzing the file's byte code? Although perhaps that is not as easy as I think.

I'm curious what the PIXIE solution would be for all this? Ideally the new AOT stuff would also have some kind of caching so individual pieces can recompile since it sounds like the plan is for there to be per-function compilation and reuse. I wouldn't mind it if I had to assign a fixed name and module to AOT stuff, and have jitted stuff be a little more cache-finnicky (especially if they compiled faster by reusing code). In this case function disambiguation, and recursive dependency tracking would be much more robust.

looks like bazel also uses file-based cache? just FYI, link

Bazel caches all previously done work and tracks changes to both file content and build commands. This way, Bazel knows when something needs to be rebuilt, and rebuilds only that. To further speed up your builds, you can set up your project to build in a highly parallel and incremental fashion.


Feature/conversation mentions

Adding mentions to the Conversation component.
Using https://github.com/signavio/react-mentions to do most of the heavy lifting, with a custom search and and suggestion display.
Also we're now checking comment body for any @blah substrings. If any exist they will be wrapped in a span.mention

 @ing in comments

@kyleharper changes/comments replied to, let me know what you think


firefox 108.0.1

Checklist

[x] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

Notes:

adds 102 to abi_migration_branches

@conda-forge-admin please rerender


Don't kill the worker when it cannot connect to smtp server

Description
Catch socket.error exception when trying to connect smtp server in error email notification.
Motivation and Context
Worker dies when it tries to send out error email but cannot connect to smtp server. This is rather unfortunate in production environment. This PR is intended to fix it.
Have you tested this? If so, how?
I have included unit test.

Nice catch!

LGTM


STMP free service, to use with k9

Hi, i need an smtp to use with k9, but all that i used gmail, yahoo, outlook ecc. seem
does not work with k9, can you suggest one?
bye

Please use the mailing list for support requests.


Previous pages are not loading after page refresh

when load more than one page, the url will change like "page=2", and now if we refresh, can only display the last page data ,and can't go back to first page,

Hi @xiaomalover.
Thank you for finding this issue. It's a serious one.
The problem is caused because of IAS extension "History" enabled by default. The best you can do is just to disable it by defining your pager as follows:
'pager' => [
            'class' => ScrollPager::class,
            'enabledExtensions' => [
                ScrollPager::EXTENSION_TRIGGER,
                ScrollPager::EXTENSION_SPINNER,
                ScrollPager::EXTENSION_NONE_LEFT,
                ScrollPager::EXTENSION_PAGING,
            ],
],

I'm not really sure that it will be a good idea to fix it in the extension code, as it may break existing deployments. But I will make necessary changes in the next major release.

It's very kind of you.

Hi @kop.
Thank you very much.

Hi,
I tried defining my pager as the above code, it doesn't seem to fix the problem.
My problem is similar to xiaomalover's. I have a page which uses the scroll pager, when a user clicks on a button, it posts data via ajax so the page doesn't refresh, I use jQuery to load the updated content in a container. For example: $('#student-side .container-inner').load(window.location.href + ' .user-opportunities', function () { ... });
When it reloads the content, the scroll pager doesn't work anymore, it either only displays the last page or the first page, and as you scroll to the end of that specific page, it doesn't load anymore (doesn't load any other pages that there is), can't go back or forward.
Please assist urgently.
Thank you.
Kind regards.

@sandy0201, I don't really understand why you are using jQuery here? Scroll pager itself is loading everything via AJAX.

@kop, I'm using jQuery because by default, the button that users can click to apply or revoke an application refreshes the page, but we don't want it to refresh the page so we decided to use ajax to post the data and not refresh the page. Once a user apply for an application, an alert message shows but only when the page is refreshed, therefore we decided to reload the content of one part of the container only using jQuery, but we realised that this causes scroll pager not to work anymore. Is there a way to fix this?

@sandy0201 I got it now. Well, I don't really know what is causing plugin to stop working. Maybe it have different ID after new rendering. Or, maybe, it just loses JavaScript events attached. The best thing I can recommend is to use Pjax container for your needs and see does it work. Or, if you will find a reason why it gets broken after Ajax request, I will be happy to accept a PR.

@kop is there any way to re-attach the JavaScript events? Like is there any JavaScript function I can call to re-attach the Scroll Pager events to make it work?

@sandy0201 Well, you can try to reinit scroll plugin itself and remove old events. But it's better to ask such sort of questions in the original repo https://github.com/webcreate/infinite-ajax-scroll, as it's not related to this PHP package.

@kop Thank you for solution of history problem!
Is there a plan how to fix it in future?
ps I love your extension!

@indeveler, yes, it will be fixed soon. I'm having a vacation right now, but after return, I'm planning to release a new major version with some refactoring and fixes for the most annoying issues.

this one seems to be fixed i am using list view with the following setting for the scroller
'pager' => [
                'class' => ScrollPager::className () ,
                'container' => '#review-wrapper' ,
                'item' => '.loader-class-review' ,
                'paginationSelector' => '#review-wrapper>.pagination' ,
                'next' => '#review-wrapper .pagination li.next a' ,
                'historyPrev' => '#review-wrapper .pagination li.prev>a' ,
                'noneLeftText' => 'No more Listings to display.' ,
                'noneLeftTemplate' => '
{text}
' ,
                'triggerText' => 'Load More Reviews' ,
                'triggerTemplate' => '
 {text}
' ,

            ] ,

i am able to load the previous buttons after refreshing the page by going to the last record and even both next and previous if in the middle of the pagination

I need to changing address bar after scroll.
if i disabled HISTORY address bar does not changing.
if i enabled HISTORY address bar changing but previos data does not showing.
How can i fix this ?


feat: dynamic Agent configuration

Automatic Agent configuration is a great tool. The limitation is that it maps to exactly one asset.
I sometimes encounter the situation that there are several identical assets that need to be feed depending on which physical asset the data is coming from.
It would be great, if I could trigger the Agent configuration from within a flow by giving an assetId or perhaps even better an asset name (if unique) and then automatically consume the data in the payload.
Even with the limitation, that the mapping must be assumed to be known, this would enhance the node for several use cases.
Alternatives would be to create one agent for each asset, which might not always work because they may be appear dynamic by some other process and also because it is probably considered waste.
Writing the same logic with direct API calls is obviously possible, but since we already have this great implementation with the Agent configuration, maybe an enhancement would be the more clean way.

We are working on a concept which would relax the need to have a full blown data configuration and agent configuration just for data ingesting (and adding some more methods to mindconnect api) I will see how fast we can make the changes on the service and API side of the mindsphere and how to incorporate it into the node :)


No error when trying to add duplicate services with different request host/path/port to LB services.

No error when trying to add duplicate services with different request host/path/port to LB services.
Steps to reproduce the problem:
Create Balancer service with 2 targets say lb1 with request host-h1 and lb2 with  request host-h2.
Edit Load Balancer and change lb1 to lb2 and save.
After this , LB has only 1 target lb1 with host-h1.
Expected Behavior:
API should error out  and not accept duplicate services instead of dropping one of the target entries.

Tested with latest build from master:
When there are multiple service entries with different host/path/port info , they all get saved .


it doesnt work with Go Microsoft

the extension doesn't work when GO Microsoft its enable, just works when its disabled

Current fix is to revert to version 0.1.3


Why are router stack up pages?

When I call other page from the menu on sidebar (left panel), it will render a new page component, ignoring the cached page, and then I have this case:

...

...

I have this code in the menu:
<ListButton view="#main-view" href="/" animatePages={false}>
    <IconHome />
</ListButton>
<ListButton view="#main-view" href="/fast-delivery" animatePages={false}>
    <IconFastDelivery />
</ListButton>

I need to open some pages without "back" functionality. Open new page withou caching. Is it a bug or configuration?

From your code, the behavior I would expect is that you click one of the links, and that new page will be added to the existing pages in the DOM cache, but there will not be an animated transition to the new page.
Is your goal to have the new page not animate and also replace the last page in the DOM cache? If so, I think the force option on the link might be what you want:
http://framework7.io/react/link.html

Is your goal to have the new page not animate and also replace the last page in the DOM cache?
Exactly. Your solution works fine and don`t stack up all pages, but first page is always rendered (home). Why?

My mainview it's:
<View id="main-view" navbarThrough dynamicNavbar={true} main>
   <Pages>
      <Home />
   </Pages>
</View>

I think if I remove  and load it dynamically, it will solve my problem, but I don`t found how make this.
If I put url="/" the  doesn't work. No erros, but no load page.


Develop

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (c8ac17d) 96.00% compared to head (7943354) 96.00%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff           @@
##           master       #5   +/-   ##
=======================================
  Coverage   96.00%   96.00%
=======================================
  Files           2        2
  Lines          25       25
=======================================
  Hits           24       24
  Misses          1        1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


New Package Static Structure

Note for clarity
We (at least I) have been using Software  Structure and Architecture indistinctly.
Static Structure: Refers to how the files and directories are organized.
Dinamic Structure: Refers to how the pieces of software interact with each other.
Architecture: Refers to the rules of development.
This Blog seems to me a good source with proper references.
Having said that this issue will be dedicated to discuss the static structure (hereafter structure) of the goodman pipeline. Related issues are #109 regarding automatic testing since depending on the structure we choose we will arrange the test files.

Since we plan to make the pipeline an astropy affiliated package or at least astropy-compatible here is the Astropy Affiliated Package Template

This issue no longer applies


[BUG] Iceberg multiple file readers can not read files if the file paths contain encoded URL unsafe chars

The multi-threaded reader and coalescing reader for Iceberg fail to read the file if its path contains encoded URL unsafe chars.
Here is a repro case.
First, create an Iceberg table by leveraging our bigDataGen in the local database.
import org.apache.spark.sql.tests.datagen._

val dbgen = new DBGen()
dbgen.addTable("download_100b", """local_create_dt string, client_name string, other long""", 1000000L)
dbgen("download_100b")("local_create_dt").setSeedRange(0, 10)
dbgen("download_100b")("client_name").setSeedRange(0, 10)
val df = dbgen("download_100b").toDF(spark).repartition(col("local_create_dt"), col("client_name"))

spark.sql("""CREATE TABLE local.download_100b (local_create_dt string, client_name string, other long) using iceberg partitioned by (local_create_dt, client_name)""")
df.writeTo("local.download_100b").append()

Then launch a GPU enabled spark-shell and read the table by Iceberg scan.
sql("select * from local.download_100b").show

It will complain the below error.
23/11/14 08:08:37 WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0) (liangcail-ubuntu20 executor driver): java.util.concurrent.ExecutionException: java.io.FileNotFoundException: File /data/tmp/local/download_100b/data/local_create_dt=)>tkiudF4</client_name=>{%nzw`HO*/00000-12-09af04cb-4423-49ef-a925-b85c0de28c39-00001.parquet does not exist
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at com.nvidia.spark.rapids.MultiFileCoalescingPartitionReaderBase.$anonfun$readPartFiles$7(GpuMultiFileReader.scala:1135)
	at com.nvidia.spark.rapids.MultiFileCoalescingPartitionReaderBase.$anonfun$readPartFiles$7$adapted(GpuMultiFileReader.scala:1134)
	at scala.collection.Iterator.foreach(Iterator.scala:943)
	at scala.collection.Iterator.foreach$(Iterator.scala:943)

The actual file path in local disk is /data/tmp/local/download_100b/data/local_create_dt=%29%3EtkiudF4%3C/client_name=%3E%7B%25nzw%60HO*/00000-12-09af04cb-4423-49ef-a925-b85c0de28c39-00001.parquet

After some invetigation, I found the file path from Iceberg is the same as its actual path, but the file path from Spark is url encoded again, although the original one is already url encoded by Iceberg writer. One part of the file path conversion from disk to the multiple readers is
For Spark:   %29%3EtkiudF4%3C -> %2529%253EtkiudF4%253C -> %29%3EtkiudF4%3C
For Iceberg: %29%3EtkiudF4%3C -> %29%3EtkiudF4%3C -> )>tkiudF4<


Segmentation Fault on closing video/music control menu

OS/Platform
Debian, Ubuntu, and derivatives
Installed
Other (please use 'Additional context' field below to mention how exactly)
Version
Version 103.0.5060.53 (Official Build, ungoogled-chromium) (64-bit)
Have you tested that this is not an upstream issue or an issue with your configuration?

[ ] I have tried reproducing this issue in Chrome and it could not be reproduced there
[ ] I have tried reproducing this issue in vanilla Chromium and it could not be reproduced there
[ ] I have tried reproducing this issue in ungoogled-chromium with a new and empty profile and it could not be reproduced there

Description
On the sound control menu if you click it then try to exit it, the browser seg faults.
How to Reproduce?

Start a audio source or video
Click the "Control for video, music, and more" menu
Try to exit it

Actual behaviour
Segfault
Expected behaviour
It just exits the menu
Relevant log output
No response
Additional context
I use void linux with the dainra repo
https://github.com/DAINRA/ungoogled-chromium-void

I do mention it in the new readme, but it doesn't have anything to do with ungoogled-chromium, I can reproduce it even on void's regular chromium package.

I hope your question(-s) has(-ve) been answered, otherwise please let us know.
Closing this issue for now.


Hide links to apps for non-privileged users

Hides all the links on the left sidebar except for Dashboards, Discover and Visualize for users that have the bap_plugins_visibility role.

Added a changelog file.


Open SAP Web IDE for SAP HANA Development

Tutorials: https://developers.sap.com/tutorials/webide-multi-cloud.html
Write here how you think we can improve the tutorial ...
The content of this turorial is not ralvant any more. It will be great if you do some clean up and delete thie tutorial.

This tutorial remains for customers that have productive accounts and still use the Web IDE fullstack there.  It is marked as such.


Event handlers for evict?

Is it possible to have eviction handler like in the lib https://github.com/bluele/gcache? Because we need to do some cleanup whenever a cache key is evicted.
I checked out your source and it seems you do have some support for it, buts internal only here
https://github.com/allegro/bigcache/blob/master/bigcache.go#L138

Hi, indeed, this feature is possible and rather simple, I will prepare it today, please stay in touch 😀

@pyros2097 please take a look at #15 and let us know if it solves this issue.

@druminski Great!That does it! Man you guys are awesome :trophy:  so quick in making this change. Thanks! :smile:

I guess the readme has to be update with the new  OnRemove func(key string, entry []byte).

Good point, @pyros2097 would you like to do it?

I've found a minute and created PR #16 which updates readme.md. I decided to set default value (nil) in the example from readme, because when callback is provided then the oldest entries are unwrapped and this is unnecessary operation for clients who don't need OnRemove feature.


remove deployment.yaml

currently, we have pvc.yaml for creating PVC and pod.yaml to show to mount PVC to an app. removing deployment.yaml in favor of pvc.yaml and pod.yaml

@gman0 PTAL

@Madhu-1 I am bit skeptical here. That said, imo, having or keeping deployment yaml in this repo is good.

@Madhu-1 I am bit skeptical here. That said, imo, having or keeping deployment yaml in this repo is good.

@humblec what advantage we are going to get if we keep deployment.yaml?
IMO in the current deployment and pod files, the usage of PVC seems to be the same for me.  there is no difference in it

@Madhu-1 I am bit skeptical here. That said, imo, having or keeping deployment yaml in this repo is good.

@humblec what advantage we are going to get if we keep deployment.yaml?

The deployment is actually the way of bringing a workload to kubernetes. That said, in a cluster someone will deploy pod via deployment artifact compared to pod artifact.

I believe it makes sense to keep both Pod and Deployment here: Pod example can be used to build e.g. StatefulSet and DaemonSet, while Deployment itself is such an example.

closing as per the comments.


OnDeserialized Failed using SyncVar Int for weapon change

Describe the bug
I am trying to synchronize weapons across clients when they join. Weapon change happens using mouse scroll. The scroll change works, but clients have the default weapon in all cases upon joining and I see this error:
OnDeserialize failed for: object=LocalPlayer(Clone) component=WeaponsControl sceneId=0 length=1. Possible Reasons:
  * Do WeaponsControl's OnSerialize and OnDeserialize calls write the same amount of data(1 bytes)?
  * Was there an exception in WeaponsControl's OnSerialize/OnDeserialize code?
  * Are the server and client the exact same project?
  * Maybe this OnDeserialize call was meant for another GameObject? The sceneIds can easily get out of sync if the Hierarchy was modified only in the client OR the server. Try rebuilding both.

System.NullReferenceException: Object reference not set to an instance of an object
  at WeaponsControl.OnWeaponChange (System.Int32 previousWeapon, System.Int32 activeWeapon) [0x0001e] in D:\Unity\Projects\The World\Assets\DontDie\Scripts\Player\WeaponsControl.cs:111
  at WeaponsControl.DeserializeSyncVars (Mirror.NetworkReader reader, System.Boolean initialState) [0x00034] in <3b1fe3ba436a4647b3e3502c9331631e>:0
  at Mirror.NetworkBehaviour.OnDeserialize (Mirror.NetworkReader reader, System.Boolean initialState) [0x0001c] in D:\Unity\Projects\The World\Assets\Mirror\Runtime\NetworkBehaviour.cs:722
  at Mirror.NetworkIdentity.OnDeserializeSafely (Mirror.NetworkBehaviour comp, Mirror.NetworkReader reader, System.Boolean initialState) [0x0008b] in D:\Unity\Projects\The World\Assets\Mirror\Runtime\NetworkIdentity.cs:923
UnityEngine.Logger:Log(LogType, Object)
Mirror.ILoggerExtensions:LogError(ILogger, Object) (at Assets/Mirror/Runtime/LogFactory.cs:58)
Mirror.NetworkIdentity:OnDeserializeSafely(NetworkBehaviour, NetworkReader, Boolean) (at Assets/Mirror/Runtime/NetworkIdentity.cs:928)
Mirror.NetworkIdentity:OnDeserializeAllSafely(NetworkReader, Boolean) (at Assets/Mirror/Runtime/NetworkIdentity.cs:957)
Mirror.ClientScene:ApplySpawnPayload(NetworkIdentity, SpawnMessage) (at Assets/Mirror/Runtime/ClientScene.cs:718)
Mirror.ClientScene:OnSpawn(SpawnMessage) (at Assets/Mirror/Runtime/ClientScene.cs:756)
Mirror.<>c__DisplayClass31_0`1:<RegisterHandler>b__0(NetworkConnection, SpawnMessage) (at Assets/Mirror/Runtime/NetworkClient.cs:331)
Mirror.<>c__DisplayClass6_0`2:<MessageHandler>b__0(NetworkConnection, NetworkReader, Int32) (at Assets/Mirror/Runtime/MessagePacker.cs:146)
Mirror.NetworkConnection:InvokeHandler(Int32, NetworkReader, Int32) (at Assets/Mirror/Runtime/NetworkConnection.cs:220)
Mirror.NetworkConnection:TransportReceive(ArraySegment`1, Int32) (at Assets/Mirror/Runtime/NetworkConnection.cs:273)
Mirror.NetworkClient:OnDataReceived(ArraySegment`1, Int32) (at Assets/Mirror/Runtime/NetworkClient.cs:172)
UnityEngine.Events.UnityEvent`2:Invoke(ArraySegment`1, Int32)
Mirror.TelepathyTransport:ProcessClientMessage() (at Assets/Mirror/Runtime/Transport/TelepathyTransport.cs:102)
Mirror.TelepathyTransport:LateUpdate() (at Assets/Mirror/Runtime/Transport/TelepathyTransport.cs:136)

To Reproduce
Here is my weapons control code. Essentially, it should disable all gameobjects of the child weapons and then reenable the current active index.
using Mirror;
using System.Collections.Generic;
using UnityEngine;

public class WeaponsControl : NetworkBehaviour
{
    private InputMaster controls;

    [SerializeField]
    private GameObject weapons;

    [SerializeField]
    private static int defaultActiveWeapon = 0;
    [SyncVar (hook = nameof(OnWeaponChange))] public int currentActiveWeapon = defaultActiveWeapon;

    private List<GameObject> weaponsList;

    private void Start()
    {
        // Create local list of weapons
        weaponsList = new List<GameObject>();
        foreach (Transform child in weapons.transform)
        {
            weaponsList.Add(child.gameObject);
        }

        // Set 0 index weapon to default active
        //SetActiveWeapon(currentActiveWeapon);
    }

    private void OnEnable()
    {
        controls = new InputMaster();
        controls.Player.Fire.performed += _ => CmdFireWeapon();

        controls.Player.ScrollWeapons.performed += ctx => {
            Debug.Log(ctx.ReadValue<Vector2>().y);
            CmdSetWeaponActive(currentActiveWeapon + 1);
        };
        controls.Enable();
    }

    private void OnDisable()
    {
        if (!hasAuthority)
            return;

        controls.Player.ScrollWeapons.performed -= _ =>
        {
            CmdSetWeaponActive(currentActiveWeapon + 1);
        };
        controls.Player.Fire.performed -= _ => CmdFireWeapon();
        controls.Disable();
    }

    // ===============================
    // Fire weapons
    // ===============================
    [Command]
    private void CmdFireWeapon()
    {
        FireWeapon(); // Set flashlight on server
        RpcFireWeapon(); // Set flashlight on clients
    }

    [ClientRpc]
    private void RpcFireWeapon()
    {
        FireWeapon();
    }

    private void FireWeapon()
    {
        // Fire Weapon
    }

    // ===============================
    // Change weapon
    // ===============================
    [Command]
    private void CmdSetWeaponActive(int weaponIndex)
    {
        SetActiveWeapon(weaponIndex); // Set weapon on server
    }

    [ClientRpc]
    private void RpcSetWeaponActive(int weaponIndex)
    {
        SetActiveWeapon(weaponIndex);
    }

    private void SetActiveWeapon(int activeWeapon)
    {
        if (!isServer)
            return;

        if (activeWeapon >= weaponsList.Count)
            activeWeapon = 0;

        if (activeWeapon < 0)
            activeWeapon = weaponsList.Count - 1;

        currentActiveWeapon = activeWeapon;

        //weaponsList[currentActiveWeapon].SetActive(true);
    }

    private void OnWeaponChange(int previousWeapon, int activeWeapon)
    {
        for (int weaponIndex = 0; weaponIndex < weaponsList.Count; weaponIndex++)
        {
            weaponsList[weaponIndex].SetActive(false);
        }
        currentActiveWeapon = activeWeapon;
        weaponsList[activeWeapon].SetActive(true);
    }
}

Desktop (please complete the following information):

OS: Windoes
Build target: standalone
Unity version: 2019.3.11f1
Mirror branch: 13.0.1 Using package manager

This is a problem with your OnWeaponChange function, from looking at that I assume one of your weapons is null
System.NullReferenceException: Object reference not set to an instance of an object
  at WeaponsControl.OnWeaponChange (System.Int32 previousWeapon, System.Int32 activeWeapon) [0x0001e] in D:\Unity\Projects\The World\Assets\DontDie\Scripts\Player\WeaponsControl.cs:111

Thanks @James-Frowen. Any idea what to change? This is a self-contained script, and it's populating the transforms of a prefab that has 4 children on all of the Player prefabs. Not clear how it could be empty.

The Exception is at line 111 in WeaponsControl.cs so you'd have to look at what can be null on that line.
Maybe add some Debug.Log  to find out what is null

If you need more help with this, the best option is to ask for help in the help channel in the discord server

It's the first line here:
for (int weaponIndex = 0; weaponIndex < weaponsList.Count; weaponIndex++)
{
       weaponsList[weaponIndex].SetActive(false);
}
currentActiveWeapon = activeWeapon;
weaponsList[activeWeapon].SetActive(true);

Either weaponList is null or you have null objects in the list/array.

@SoftwareGuy I understand that now. Just now trying to figure out WHY it's null. All instances should populate with the 3 child transforms that are contained within on Start. I've narrowed it down that it has something to do with the Server vs client instances.


NIFI-2828: Fix Hive NAR dependencies

The approach is to remove the parent dependency and put all Hadoop/Hive dependencies in the Hive NAR instead of expecting them in a parent classloader.

Tried this out on against an HDP 2.4 sandbox and got an exception that boiled down to:
Caused by: java.io.IOException: No FileSystem for scheme: hdfs
	at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584) ~[hadoop-common-2.6.2.jar:na]
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591) ~[hadoop-common-2.6.2.jar:na]
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91) ~[hadoop-common-2.6.2.jar:na]
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630) ~[hadoop-common-2.6.2.jar:na]
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612) ~[hadoop-common-2.6.2.jar:na]
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) ~[hadoop-common-2.6.2.jar:na]
	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) ~[hadoop-common-2.6.2.jar:na]
	at org.apache.hadoop.hive.ql.io.orc.OrcRecordUpdater.<init>(OrcRecordUpdater.java:221) ~[hive-exec-1.2.1.jar:1.2.1]
	at org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat.getRecordUpdater(OrcOutputFormat.java:292) ~[hive-exec-1.2.1.jar:1.2.1]
	at org.apache.hive.hcatalog.streaming.AbstractRecordWriter.createRecordUpdater(AbstractRecordWriter.java:141) ~[hive-hcatalog-streaming-1.2.1.jar:1.2.1]
	at org.apache.hive.hcatalog.streaming.AbstractRecordWriter.newBatch(AbstractRecordWriter.java:121) ~[hive-hcatalog-streaming-1.2.1.jar:1.2.1]
	... 10 common frames omitted

Comparing the JARs that get included with Hadoop Libraries NAR vs the Hive NAR there appear to be some differences.
Hadoop Libraries NAR:
hadoop-annotations-2.6.2.jar
hadoop-auth-2.6.2.jar
hadoop-client-2.6.2.jar
hadoop-common-2.6.2.jar
hadoop-hdfs-2.6.2.jar
hadoop-mapreduce-client-app-2.6.2.jar
hadoop-mapreduce-client-common-2.6.2.jar
hadoop-mapreduce-client-core-2.6.2.jar
hadoop-mapreduce-client-jobclient-2.6.2.jar
hadoop-mapreduce-client-shuffle-2.6.2.jar
hadoop-yarn-api-2.6.2.jar
hadoop-yarn-client-2.6.2.jar
hadoop-yarn-common-2.6.2.jar
hadoop-yarn-server-common-2.6.2.jar
Hive NAR:
hadoop-annotations-2.6.2.jar
hadoop-auth-2.6.2.jar
hadoop-common-2.6.2.jar
hadoop-mapreduce-client-core-2.6.2.jar
hadoop-yarn-api-2.6.2.jar
hadoop-yarn-common-2.6.2.jar
hadoop-yarn-server-applicationhistoryservice-2.6.0.jar
hadoop-yarn-server-common-2.6.0.jar
hadoop-yarn-server-resourcemanager-2.6.0.jar
hadoop-yarn-server-web-proxy-2.6.0.jar
I think the Hive NAR at least needs the hadoop-hdfs jar, but not sure what else.

Latest commit worked, thanks for updating, merged to master


Array analysis for transpose with arbitrary arguments

Now that we support arbitrary transpose arguments, add array analysis for it.

Codecov Report

Merging #2816 into master will increase coverage by <.01%.
The diff coverage is 56.52%.

@@            Coverage Diff             @@
##           master    #2816      +/-   ##
==========================================
+ Coverage   86.17%   86.18%   +<.01%
==========================================
  Files         323      323
  Lines       67284    67300      +16
  Branches     7554     7558       +4
==========================================
+ Hits        57985    58001      +16
- Misses       8095     8097       +2
+ Partials     1204     1202       -2

This looks good, thanks for the patch. Merging.


Support for command aliases (short command)

Missing command aliases as a feature.
For example such a thing is now possible:
      .subCommand<CreateOptions, CreateArgs>('access-roles [name]')
      .alias('ar')
      .description('Create new access role')
      .action(this.execute.bind(this));

It's possible now to execute the commands callback with access-role or ar.

sorry for late response.
this PR contains #13 ? please independent of each other.
What is the motivation of this changes?
If you can write -a, --access-role, --ar, What do you think about this syntax?


[Feature Request] Revamped buy/sell log

For each item on the market that is a "commodity" there is a buy/sell log. Would be nice to have:

longer log (ie save history)
timestamps

Actually, that is quite a nice idea. Something like "Show more activity" and it just expands it. I'll see when I have the time to implement this since I've got some other stuff going on right now.


Initial implementation of full DAG pipeline with Dreem FS setup

This PR pulls in the working port of the Dreem (DRM) pipeline into the Airflow structure, using Airflow's structure of Hooks and Operators. This PR is merged in order to setup tasks and a collaborative workflow, but is still missing tests, checks and further work on re-usable code.

Acknowledge the failing tests, but merge for ease of collaboration. Will set tests and fixes as further milestone.


update packages and use shx for cross platform npm script

shx is the wrapper of shelljs which can be used in npm script thttps://www.npmjs.com/package/shx

Thanks!


allow autocompletion for only 1 file extension while disabling others.

e.g:
I want autocompletion only for .js/.ts file while disabling it for .HTML file bcoz it messes up with emmet.

Hi @Abhirup-99
This is possible. Just to get more information: how does the Kite plugin interfere with Emmet?

Thanks for the reply. In vs code, with emmet, and vs code autocompletion for the tags, kite is not generally required for both HTML and CSS files, but kite suggestions come at the top and for using emmet, I have to scroll down to get the emmet suggestion.
Also, can you suggest the procedure of how to achieve disabling for .HTML and .css files?
Lastly, kite autocompletion is great. Thanks for this.

Note for implementer:

provide in-editor options to enable/disable per filetype
for disabled filetypes, show "Kite: disabled" in the status bar.

If we can add a tooltip (or similar), perhaps "Enable this file type in VS Code settings"

Hi @Abhirup-99, if you update to v0.132.0 of the vscode plugin, you should be able to disable Kite completions for html and css with  "kite.completions.disabledFileExtensions": [ ".css", ".html" ] in your settings

We have to revert the change temporarily in 0.133.0 as it is causing another issue. It will be reintroduced shortly.

Reintroduced in 0.134.0, https://github.com/kiteco/vscode-plugin/commit/28b28c1cb6340c58b93391ee8053cf73c2847022


LMS buy issue - Special Attacks not in bank

🐛 Bug Report
I purchased 2 of the Golden special attacks and neither are in my bank but my points are spent.

These are CL only items, there isn't a physical item in osrs its a toggle feature
also partially reported in #3568

These are CL only items, there isn't a physical item in osrs its a toggle feature
also partially reported in #3568

well I'll be. maybe on bso tho they should become the physical item? do to they do exist on bso. so would only make sense.


Please explain how to setup Cloud Witness with a managed identity

This documentation does not explain how to use a Managed Identity to access the storage account. It only describes using a shared access key, which Microsoft recommends not using to access storage accounts.
We are trying to stop using shared access keys, as recommended here:
https://learn.microsoft.com/en-us/azure/storage/common/shared-key-authorization-prevent?tabs=portal
To that end, some guidance for this product on how to use a managed identity would be appreciated.
If MI is not supported, please provide an ETA on when support will be added.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 7e6da99e-a148-b5ea-224d-8a4c48604945
Version Independent ID: a6a90a1f-870c-3501-ce6e-fbf52f802dfb
Content: Deploy a cloud witness for a failover cluster in Windows Server
Content Source: WindowsServerDocs/failover-clustering/deploy-cloud-witness.md
Product: windows-server
Technology: failover-clustering
GitHub Login: @JasonGerend
Microsoft Alias: jgerend

We hugely appreciate your contribution, feedback, and support to our technical documentation.
We’ve phased out GitHub Issues on this repository as the feedback mechanism for content and have replaced it with a new feedback system. For more information see, Content User Feedback. As part of this rollout, we are migrating your issue where our team will use your comments to improve the documentation experience. Unfortunately, this does mean you won't be able to access this issue after the migration is complete.
All issues will be migrated by Thursday 29th August 2024.
Once again, thank you for your valuable feedback, we hope you’ll continue to contribute using our new feedback system.


Build failure for libssh2 on Windows 10, MSYS2

I wasn't able to build cargo on Windows 10 64-bit, using MSYS2.
I checked out rust from git, and built it according to the instructions, and installed it into /usr/local.
For cargo, I followed the instructions in the README, and used this for configure:
./configure --local-rust-root=/usr/local

And then ran make.   Here's the end of the build:
   Compiling cmake v0.1.12
   Compiling curl-sys v0.1.29
   Compiling url v0.2.38
   Compiling libgit2-sys v0.3.8
   Compiling libssh2-sys v0.1.34
Build failed, waiting for other jobs to finish...
failed to run custom build command for `libssh2-sys v0.1.34`
Process didn't exit successfully: `C:\msys64\usr\local\src\cargo\target\release\build\libssh2-sys-b1f216ef589edc64\build-script-build` (exit code: 101)
--- stdout
running: "cmake" "C:\\msys64\\home\\james.c.graves.jr\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\libssh2-sys-0.1.34\\libssh2" "-G" "MSYS Makefiles" "-DCRYPTO_BACKEND=WinCNG" "-DZLIB_INCLUDE_DIR=/" "-DBUILD_SHARED_LIBS=OFF" "-DENABLE_ZLIB_COMPRESSION=ON" "-DCMAKE_INSTALL_LIBDIR=lib" "-DBUILD_EXAMPLES=OFF" "-DBUILD_TESTING=OFF" "-DCMAKE_INSTALL_PREFIX=C:\\msys64\\usr\\local\\src\\cargo\\target\\x86_64-pc-windows-gnu\\release\\build\\libssh2-sys-b1f216ef589edc64\\out" "-DCMAKE_C_FLAGS= -O0 -ffunction-sections -fdata-sections -m64" "-DCMAKE_CXX_FLAGS= -O0 -ffunction-sections -fdata-sections -m64" "-DCMAKE_BUILD_TYPE=Release"

--- stderr
CMake Error: Could not create named generator MSYS Makefiles
thread '<main>' panicked at '
command did not execute successfully, got: exit code: 1

build script failed, must exit now', C:\msys64\home\james.c.graves.jr\.cargo\registry\src\github.com-1ecc6299db9ec823\cmake-0.1.12\src\lib.rs:424

Makefile:93: recipe for target 'cargo-x86_64-pc-windows-gnu' failed
make: *** [cargo-x86_64-pc-windows-gnu] Error 101

I'm not sure if this is a cargo issue, or if it is a problem with libssh2.  I'm new to rust and cargo.

Where did you install cmake from? I would expect that this generator would be pretty common among all cmakes...

I just ran pacman -S cmake, so it is whatever currently installed by default with MSYS2.

Oh right this is when I don't quite understand what's up with MSYS2 and whatnot. The foo packages typically aren't actually the ones you want, but rather mingw-w64-{x86_64,i686}-foo. So for example in this case the cmake package is somewhat broken but the mingw-w64-x86_64-cmake package is what you want.
I have no idea why this is the case :(

OK, I installed that version of cmake, and uninstalled the plain cmake package using pacman.
The build ran to completion, and I was able to install it and it seems to work.  Excellent!
Is this something that people more familiar with MSYS2 are likely to know about already, or should this be mentioned in the documentation?
I did see a bunch of warnings during the compile like this:
src\bin\bench.rs:64:25: 64:32 warning: private type in public interface (error E0446), #[warn(private_in_public)] on by default
src\bin\bench.rs:64 pub fn execute(options: Options, config: &Config) -> CliResult<Option<()>> {
                                            ^~~~~~~
src\bin\cargo.rs:155:22: 63:22 note: in this expansion of cmd! (defined in src\bin\cargo.rs)
src\bin\cargo.rs:155:5: 155:27 note: in this expansion of each_subcommand! (defined in src\bin\cargo.rs)
src\bin\bench.rs:64:25: 64:32 note: this lint will become a HARD ERROR in a future release!

Is this already known?  Should we create a separate issue for them?
Thanks for your help.

Ah it looks like some new warnings are showing up on nightly which haven't been handled yet, feel free to open an issue about that.
I'm also honestly not sure if it's standard practice on MSYS2, but feel free to open an issue about the docs! (or send a PR yourself is also cool)


How to change transition style?

Hello, any idea to customise the transition style when presenting the image?

An image displacement effect was the core idea to write this piece of code. The transition logic to full screen is not well encapsulated so at the moment I don't think you can easily do it.


Refactor

@Tehnix I've created PR as it's easier to go through diff :).

Related refactor can be found at https://github.com/Tehnix/spaceneovim-layers/pull/26

@lowski so, after trying a lot of different approaches I had to regress to adding the language layer commands directly as a dictionary. vim-leader would do weird redraws and other things in various cases.
I've also updated the README to reflect this, it should all be working now, but let me know if you run into any bugs! This assumes the refactor branch from Tehnix/spaceneovim-layers#26 is also in effect.

@Tehnix sorry for delay.
All keybindings in leader-guide show correctly now! 🎉
However when I open single file (i.e. like this: nvim init.vim) I get the following error:
Error detected while processing FileType Auto commands for "vim":
E722: Missing comma in Dictionary:
E15: Invalid expression: { "name": "+major-mode-cmd","-": ["echo 'No commands exist yet!'", "no-commands-exist!"]

One more thing: function to apply 'nord' colorscheme does not work for me (as in the comment).

@lowski  Ah yeah, the vim issue was a syntax error on my part in the layer. It should be fixed now in the refactor branch in the layers repo.


Load from DynamoDb fails for derived classes

Load of DynamoDb entity fails when the property is overriden with exception:
System.InvalidOperationException: Dictionary already contains item with key [name of overriden property]
Expected Behavior
Load should succeed no matter if property of base class is overriden or not.
Current Behavior
Load fails if property is overriden.
Exception is thrown:
System.InvalidOperationException: Dictionary already contains item with key RangeFoo
from:
https://github.com/aws/aws-sdk-net/blob/master/sdk/src/Core/Amazon.Util/Internal/InternalSDKUtils.cs#L171
called from here:
https://github.com/aws/aws-sdk-net/blob/master/sdk/src/Services/DynamoDBv2/Custom/DataModel/InternalModel.cs#L319
Possible Solution
Proper handling of property names when using reflection on base/derived class, to avoid duplicate key entries. Improve this method:
https://github.com/aws/aws-sdk-net/blob/master/sdk/src/Services/DynamoDBv2/Custom/DataModel/InternalModel.cs#L296
Note: This was working fine in full .net, so could be because of reflaction changes in .net core.
Steps to Reproduce (for bugs)
public class Base {
  public virtual string RangeFoo {get;set;}
}

public class Derived : Base {
  [DynamoDBHashKey]
  public string KeyBar {get;set;}
  [DynamoDBRangeKey]
  public override string RangeFoo {get;set;}
}

var entity = await iDynamoDbContext.LoadAsync<Derived>(key, range);

Context
Migrating the code from .net 4.6.2 to .net core 2
Your Environment
PackageReference Include="AWSSDK.Core" Version="<IP_ADDRESS>"
PackageReference Include="AWSSDK.DynamoDBv2" Version="3.3.7"
PackageReference Include="AWSSDK.S3" Version="<IP_ADDRESS>"
PackageReference Include="AWSSDK.SecurityToken" Version="<IP_ADDRESS>"
PackageReference Include="AWSSDK.SQS" Version="<IP_ADDRESS>"
.NET Core Info
netcoreapp2 and netstandard2

Having same problem.

Ditto... :-(

Same

Yep same here :(

Any progress?

I have the fix and I'm currently going through code review and testing. Hope to have the fix released soon.

Version <IP_ADDRESS> of AWSSDK.DynamoDBv2 is out with the fix.


denyUpdate conflicts with autoValue

If I have an autoValue property set, I am not able to use denyUpdate. I get the error that the value cannot be updated because autoValue is trying to set it when I update the document.

The best way to resolve this is to write your autoValues carefully. Either don't set the value when it's an update or don't set it when Meteor.isClient is true.


upstream compliance notification

Codecov Report

Merging #1036 into master will decrease coverage by 0.03%.
The diff coverage is 7.69%.

@@            Coverage Diff             @@
##           master    #1036      +/-   ##
==========================================
- Coverage   30.01%   29.98%   -0.04%
==========================================
  Files         176      176
  Lines        4661     4669       +8
  Branches     1233     1238       +5
==========================================
+ Hits         1399     1400       +1
- Misses       3262     3269       +7

Impacted Files
Coverage Δ

.../components/browser/containers/browser-viewport.ts
21.87% <ø> (ø)
:arrow_up:

wherehows-web/app/utils/array.ts
100% <ø> (ø)
:arrow_up:

...p/components/datasets/containers/dataset-schema.ts
50% <ø> (ø)
:arrow_up:

...omponents/datasets/containers/dataset-ownership.ts
67.64% <ø> (ø)
:arrow_up:

...mponents/datasets/containers/dataset-compliance.ts
74.54% <0%> (+0.47%)
:arrow_up:

wherehows-web/app/controllers/datasets/dataset.js
10% <0%> (-3.05%)
:arrow_down:

...mponents/datasets/containers/dataset-properties.ts
61.53% <0%> (ø)
:arrow_up:

wherehows-web/app/components/dataset-aclaccess.ts
78.57% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 70f5f78...f3f4806. Read the comment docs.

Coverage remained the same at 26.911% when pulling f3f4806de9dd3648fe0b08eb4054ec8e500bbaa4 on theseyi:upstream-compliance-notification into 70f5f78441af2a9f8fab3b1af568363ea2975508 on linkedin:master.


Core renderer rewrite for stability and functionality

Rewriting the renderer
Yoru Engine is in a complex state currently, where parts of the core logic are inconsistent, non-cross-platform, and the likes. The following classes are going to be targeted in a new branch for rewriting certain mechanics, the primary focus being more renderer-agnostic:

Element - Needs complete rewrite
Application - Missing event listeners
Transform - Utilising System.Numerics Matrix4x4 or similar

New drawing system
Yoru Engine from heart was written with the purpose of targeting a SkiaSharp canvas across multiple platforms, allowing for consistent and straightforward drawing. This simplicity will remain, although Elements will no longer draw directly to a SkiaCanvas, but will instead send draw commands to a CommandPool (or something similar) to allow for dynamic draw command manipulation and switching. This renderer-agnostic method will bring more scalability into Yoru Engine, allowing it to spread to other renderers like SDL, Vulkan, etc.
More information about the CommandPool and future of elements, applications and renderers will be provided once there is a clear direction to take Yoru Engine in.

After careful consideration, I've decided to keep Yoru Engine in line with SkiaSharp, meaning that all core functionality will pass through the library, but some logic (potentially matrixes) will be custom.
I will determine whether the idea of a CommandPool will still be used, as it is a difficult concept to intertwine with the current state of Yoru Engine.

The first changes will consist of Appliation modifications and enhancements, mostly surrounding the event logic and configuration.
There are mostly two ways of doing this:

Have a dedicated 'ApplicationConfig' class or similar which:

On load will apply the appropriate values
Stay as a property of the Application allowing for the values to change

Have all properties be within the Application class

The style of configuration that will be opted for will determine how configuration is handled for other classes like Element, eg.
Additionally, an important change will be modifying how Transforms work, so that they are extensible and not restrained to a parent/pivot based anchoring system.
This will allow for more flexibility in creating GridElements, or just general elements that respond to their parent or children in different ways.


chore(android): prioritize mavenCentral over jcenter

Summary
Recent react-native dependencies are published to Maven Central only.
https://github.com/react-native-camera/react-native-camera/issues/2490

Will be done in
Upgrade to RN 0.65 PR

JCenter seems to go down multiple times a day now. Can we please prioritize this PR?


Remove unused code

Detailed description
There is no reason to keep it in the repository - if anything it could be extracted from the history.
Test plan
All green

Sdb uses to be a datastructure collection library and tool, all those algorithms were moved to Attic after deciding other implementations were more interesting for the actual sdb use case. despite Attic comes from the CVS era, i would like to have them somewhere else so its easy for anyone to find and use them if they are needed for their project, i spent quite a lot of time cleaning up and testing all of those to be easy to use and clean APIs.
I'll just create a repo with the contents of this Attic in my repository and make the dist tarballs 25KB smaller!

Done


Added a notice about windows errors

I've spent a long time figuring out why i couldn't rebuild my modules. On windows, even if your configuration is correct , there is environment difficulties and running it throught an npm script do the job.
I found some informations by reading the issues but maybe it could be nice to have this in the main documentation

Thanks, @maxdow. I found a little typo, but otherwise this looks good.

I went ahead and merged and fixed the typo. 👍

You are welcome. Thanks for the fix


Jekyll fixes

Description of changes
Fixes for two Jekyll issues:

webrick gem added to rubyPackages to fix jekyll serve on Ruby 3
github/pages-gem#752
Fix for sass-embedded gem added to gem-config so that the latest Jekyll 4 can be built with all gems from source.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 240000 run on x86_64-linux 1

3 packages marked as broken and skipped:

discourse
discourseAllPlugins
rubyPackages_2_7.webrick

3 packages built:

rubyPackages.webrick (rubyPackages_3_1.webrick)
rubyPackages_3_0.webrick
rubyPackages_3_2.webrick


fix(deps): update dependencies

PR summary
Update dependencies to avoid vulnerabilities
PR Checklist
Please make sure that your PR fulfills the following requirements:

[ ] The commit message follows the Angular Commit Message Guidelines.
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

Current vs new behavior

Does this PR introduce a breaking change?

[ ] Yes
[ ] No

Other information

:tada: This PR is included in version 0.18.3 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


swift 1.2

Hello,
i have tried today to run my project on the new xcode 6.3 beta with swift 1.2, but i have seen that your project isn´t compatible with this swift version. My question is when you want to adapt your project to swift 1.2.

Hi, same problem here. Updates would be great.

just pushed a new branch with swift 1.2 support

Awesome, thanks! :-)


[Feature]: Review 2

Guidelines

[X] I agree to follow this project's Contributing Guidelines.

Description
Reviewer @luxaritas requests
Problem
Video documentation is difficult to follow for some users.
Proposed Solution

[x] For functionality documentation, add a basic textual guide (with screenshots as appropriate) either to the readme or a separate markdown file in the docs directory with a link from the readme? Doesn't need to be exhaustive.

[x] Similarly for automation, add a checklist in a file in the docs directory that describes the testing procedure? The demo video is great, but doesn't provide a clear list of steps of things that should be checked to know the software is behaving appropriately.

Alternatives Considered
Video documentation.

See comment: https://github.com/openjournals/joss-reviews/issues/5136#issuecomment-1694759905

We do have some manual testing documentation already which is also cross-referenced in the readme: https://github.com/code4sac/trash-ai/tree/production/frontend/tests
I will add the checklist suggested.

Closing with these commits:
b2c4c70a490a0b6372b25b9080ec7cb69bd35881
3a6e27257daa49e9d2ae834d2c2cd76dfaa9c653
51aed34d5ff2958b074c5e148436eb4c916c0517
dcb2265065e469013879f02de53f016a1d2aae3f


Navbar dropdown items should support type "doc"

🚀 Feature
Relates to #4972
items: [
  {
    label: 'Flows',
    position: 'left',
    items: [
      {
        label: 'Library Flow',
        to: 'docs/library-flow/overview'
      },
    ],
  },

When navigating in previous version docs click on Library Flow switches to the latest docs. As a result, we lose the "version context".
Have you read the Contributing Guidelines on issues?
Yes
Motivation
Having versioned dropdown subitems could provide better users experience and improve navigation (not loosing version context)
API Design
items: [
  {
    label: 'Flows',
    position: 'left',
    items: [
      {
        type: 'doc',
        label: 'Library Flow',
        docId: 'app-flow',
      },
    ],
  },

Possible solution will be to use getNavbarItemComponent here:
https://github.com/facebook/docusaurus/blob/073fdeae8eea3cf13e27fcc7f0a9fe5265fad84c/packages/docusaurus-theme-classic/src/theme/NavbarItem/DefaultNavbarItem.tsx#L122-L129
Have you tried building it?
No

Thanks, seems reasonable to support this (as well as all future navbar items that are simple links)

In my case, I would like to use a drop-down menu for different docs instances, like:
items: [
        {
          label: "Docs",
          position: "left",
          items: [
            {
              type: "doc",
              docId: "index",
              label: "Repo A",
            },
            {
              type: "doc",
              docId: "index",
              docsPluginId: "repo-b",
              label: "repo-b",
            },
          ],
        },
        { to: "/blog", label: "Blog", position: "left" }
]

I'm not sure, is it included in this feature request?

@felipecrs docsPluginId  is supported by the doc type, so it will be supported as part of the current feature request


Extremely slow download of large files from GitHub Releases in winget (compared to downloading same file in browser)

Brief description of your issue
I find the download speed of a large 1.29GB package in winget is orders of magnitude slower than downloading the same package in the browser.
Steps to reproduce
Run winget install kiwix.wikimed.electron in Terminal. Download for me is around 500Kb per second. It would take around 45 minutes to completely download the file, if not longer. In any case I didn't wait around, it was too painful.
When I click on the download link provided in winget, and open it in my browser, on the same machine I get a download speed of 6.5Mb per second.
I have experienced this on multiple occasions on two different machines running on the same network. Winget is unusable for downloading this >1GB package.
Expected behavior
I expect similar download speeds to browser.
Actual behavior
See above -- painfully slow.
Environment
Windows Package Manager (Preview) v1.3.692-preview
Copyright (c) Microsoft Corporation. All rights reserved.

Windows: Windows.Desktop v10.0.22000.613
Package: Microsoft.DesktopAppInstaller v1.18.692.0

Have you tried changing which downloader is used in the settings?
https://github.com/microsoft/winget-cli/blob/2937bdfcb3681c45a53eabc6cc45fafc327fa956/schemas/JSON/settings/settings.schema.0.2.json#L139-L152

Thank you for the suggestion. I was not aware of this setting. I'll try it.

I've been seeing this in the last few days too, although I haven't troubleshooted (figured my connection had a blip). Is there something going on with Delivery Optimization and GitHub right now?

I'm not aware of any issues. Is it possible Windows Updates or something else is running in the background affecting the throughput for Delivery Optimization?

Well I've just done another test with the same file, and through winget the file downloaded 32MB (of a 1.29GB file) in around 40 seconds. So, without stopping winget, I ctrl-clicked the link that winget provides to the original paciage, I accepted the download in my browser, and the same file downloads over 200MB in around 6 seconds, while winget is still going at a snail's pace. My PC is fully up to date, no updates being downloaded.
Is it the PowerShell download process? Is it single-threaded? There's something very odd going on. It's clearly not my local computer or connection. It used to be much faster than this in winget.

I've just hit the same thing, and the wininet downloader seems as fast as ever, while DO (aka default) is 🐌 (and not Turbo)
Going to follow up with the DO team to see if they can help interpret the logs that I've collected.

Apparently the issue is DO getting timeouts from github's CDNs, resulting in DO thinking the maximum throughput is very low.  They are considering how future code can better handle this, but there isn't a lot we can do to fix this in winget right now.
Hopefully the service issues will resolve and things will go back to normal.  If not, we can consider putting in something like "if github, don't use DO".  Or maybe "if DO takes over X seconds to download Y MB, give up and fall back to wininet".
The immediate options are:

Use wininet as the downloader; it is slower than DO because it is a very basic implementation, but since it isn't trying to be smart it isn't as affected by externalities.
Just try again; CTRL+C, Up, Enter.  If you get lucky, the CDN doesn't hiccup.
According to the DO team, you can set Absolute bandwidth limits to prevent the throughput detection issue.
a. Open Settings (🪟 + i )
b. Find Delivery Optimization advanced options
c. Select Absolute bandwidth
d. Check Limit how much bandwidth is used for downloading updates in the foreground
e. Set to whatever limit you want (100 Mbps?)

That's very helpful. Thanks for investigating and following up. I'll try one of these solutions tomorrow (it's late in my time zone), and will report back.

I confirm that changing the network setting to "wininet" produces a much more acceptable download speed. Perhaps this should be an automatic fallback (or one of the other solutions if easier to implement as a fallback). I'm not sure how one would detect that DO is being artificially rate-limited, however.

Delivery Optimization, it's the system that powers Windows Update and some other things

what can i do to increase cmd => winget upgrade libreoffice

Honestly, at least specifically with libreoffice, this might just be a issue of slow servers and no mirror. Downloading it specifically on my system is slow with either backend. We can't do much about servers being slow

Honestly, at least specifically with libreoffice, this might just be a issue of slow servers and no mirror. Downloading it specifically on my system is slow with either backend. We can't do much about servers being slow

you are right. i copied the link displayed in cmd into ms edge and downloaded it from there. speed 4-9 KB/s. ehem. that's ... slow ...

This issue persists, implying it's basic to the DO and GitHub combination, and really needs a fallback put in place on the winget end. I'm only posting this now because I got a new PC, and was wondering why winget download performance had become so terrible on this spanking new laptop. Then I remembered this thread. On my old PC I had set winget to use wininet and had never looked back. To notice the (huge) difference, you need to attempt to install a large (>200MB) installer, like Kiwix.WikiMed.Electron.

i think it work it works this way: when a new version of libreoffice comes out, many upgrade automatically because they wrote a.script or soemthing lile that. i know 2 tools (gui) where you can force winget upgrade automatically every startup. because of that (must be lots of windows 10/11 gadgets) download gets very slow. then, when most of these soulless bots have libreoffice installed, its time for the sane, civilized people to upgrade their winget list. now the download speed of libreodfoce in its newest version (7.3.4 which exists since 9th of june 2022) is higher and acceptable. so statisticslly after a new version, you have to wait 1 month, which is ok for libreoffice because of all the program winget upgrades, it only happens with libreoffice that the download speed is low and updates for libreoffice arent that important to install immediately.

This issue persists, implying it's basic to the DO and GitHub combination, and really needs a fallback put in place on the winget end. I'm only posting this now because I got a new PC, and was wondering why winget download performance had become so terrible on this spanking new laptop. Then I remembered this thread. On my old PC I had set winget to use wininet and had never looked back. To notice the (huge) difference, you need to attempt to install a large (>200MB) installer, like Kiwix.WikiMed.Electron.

@hueldoeu Thanks for that explanation, but in my case I am installing-testing my app Kiwix.WikiMed.Electron, not libreoffice, and my app has a minute user base in comparison. There is an ongoing issue regarding the interaction between winget's use of Delivery Optimization and GitHub, at least for large installers (it's not likely to be very noticeable for installers < 70MB).

This issue persists, implying it's basic to the DO and GitHub combination, and really needs a fallback put in place on the winget end.

My previous comment contains the mitigations that one can use, and the only suggestion that I can think of that we could do.  But even that (which is effectively, "is DO achieving some arbitrary minimum bandwidth?") isn't foolproof, if you are truly bandwidth limited then we would be moving to the potentially slower option.
I'm not sure that there is really a good, automatic way for us to do a better job than a team that is dedicated to doing this work.  The right solution to my mind is to drive fixes into either DO or github or both.

Use wininet as the downloader; it is slower than DO because it is a very basic implementation, but since it isn't trying to be smart it isn't as affected by externalities.
Just try again; CTRL+C, Up, Enter.  If you get lucky, the CDN doesn't hiccup.
According to the DO team, you can set Absolute bandwidth limits to prevent the throughput detection issue.
a. Open Settings (🪟 + i )
b. Find Delivery Optimization advanced options
c. Select Absolute bandwidth
d. Check Limit how much bandwidth is used for downloading updates in the foreground
e. Set to whatever limit you want (100 Mbps?)

@JohnMcPMS Unfortunately, mitigation 1 and 3 are no longer working. At least on Windows 11 22H2 mitigation 3 no longer seems to make any difference to the ridiculously slow download of large installers from GitHub when DO is enabled in winget. By large, I'm referring to installers > 500MB. The one that affects me most, because I have to fight for hours to get it validated, is Kiwix.WikiMed, which is 1.2GB. I can literally wait an hour while winget plods its way through the download using DO, yet if I direct-download the installer file, it takes about 3 minutes.
And with the latest winget, wininet download seems to fail at the hash checking stage for large installers too.  It fails with:
An unexpected error occurred while executing the command:
0x80072efd : unknown error

Screenshot below shows tests I did with wininet enabled in the winget settings. The first shows it failing with a local manifest of the latest 1.2GB WikiMed release that I was trying to validate for release on winget. The second is a small installer which succeeds. The last is with the current WikiMed package on winget. All installers are hosted on GitHub.
I guess no-one tests winget with large downloads, and slow download speeds won't make much difference for a 50MB installer. The experience is pretty much unusable with large installers, and none of the workarounds are working any more...

Apparently the issue is DO getting timeouts from github's CDNs, resulting in DO thinking the maximum throughput is very low. They are considering how future code can better handle this, but there isn't a lot we can do to fix this in winget right now.
Hopefully the service issues will resolve and things will go back to normal. If not, we can consider putting in something like "if github, don't use DO". Or maybe "if DO takes over X seconds to download Y MB, give up and fall back to wininet".
The immediate options are:
1. Use `wininet` as the downloader; it is slower than DO because it is a very basic implementation, but since it isn't trying to be smart it isn't as affected by externalities.

2. Just try again; CTRL+C, Up, Enter.  If you get lucky, the CDN doesn't hiccup.

3. According to the DO team, you can set `Absolute bandwidth` limits to prevent the throughput detection issue.
   a. Open Settings (🪟 + i )
   b. Find `Delivery Optimization advanced options`
   c. Select `Absolute bandwidth`
   d. Check `Limit how much bandwidth is used for downloading updates in the foreground`
   e. Set to whatever limit you want (100 Mbps?)

Anyone know the powershell one-liner to set this without having to use mouse or keyboard (I'm applying this via automation)

As libreoffice is often stated this is a related PR #2310
As 2310 is closed I am fine to pick up the help-wanted of this PR to get the mysterious problem solved.

@Karl-WE As developer of large apps (1.29GB), this issue is still frustrating me. I get very very slow download with Kiwix.WikiMed for example. Here is a log of my installation of this app just now. It took 37 minutes to download 1.29GB, a download that normally takes no more than 5 or 6 minutes over my broadband.
2023-01-07 11:13:01.285 [CLI ] Failed to get msix signature hash, fall back to direct download.
2023-01-07 11:13:01.301 [CLI ] Generated temp download path: C:\Users\gkant\AppData\Local\Temp\WinGet\Kiwix.WikiMed.<IP_ADDRESS>\035080d826d851505358ea653aa0caafe4ab655f8d8ce9aaebe0eaec2319932a
2023-01-07 11:13:01.302 [CORE] Downloading to path: C:\Users\gkant\AppData\Local\Temp\WinGet\Kiwix.WikiMed.<IP_ADDRESS>\035080d826d851505358ea653aa0caafe4ab655f8d8ce9aaebe0eaec2319932a
2023-01-07 11:13:01.302 [CORE] DeliveryOptimization downloading from url: https://github.com/kiwix/kiwix-js-windows/releases/download/v2.2.62-WikiMed/KiwixWebAppWikiMed_<IP_ADDRESS>_AnyCPU.appxbundle
2023-01-07 11:48:27.275 [CORE] DeliveryOptimization error: 0x80d02002, extended error: 0x80190191
2023-01-07 11:48:27.276 [FAIL] D:\a\_work\1\s\external\pkg\src\AppInstallerCommonCore\DODownloader.cpp(289)\WindowsPackageManager.dll!00007FFB0EEB28F9: (caller: 00007FFB0EEB31C0) Exception(2) tid(29e4) 80D02002
2023-01-07 11:48:27.410 [CORE] Started applying motw to C:\Users\gkant\AppData\Local\Temp\WinGet\Kiwix.WikiMed.<IP_ADDRESS>\035080d826d851505358ea653aa0caafe4ab655f8d8ce9aaebe0eaec2319932a with zone: 3
2023-01-07 11:48:27.419 [CORE] Finished applying motw
2023-01-07 11:48:27.420 [CORE] WinINet downloading from url: https://github.com/kiwix/kiwix-js-windows/releases/download/v2.2.62-WikiMed/KiwixWebAppWikiMed_<IP_ADDRESS>_AnyCPU.appxbundle
2023-01-07 11:50:44.857 [CORE] Download hash: 035080d826d851505358ea653aa0caafe4ab655f8d8ce9aaebe0eaec2319932a
2023-01-07 11:50:44.857 [CORE] Download completed.
2023-01-07 11:50:44.858 [CLI ] Installer hash verified
2023-01-07 11:50:44.859 [CORE] Started applying motw to C:\Users\gkant\AppData\Local\Temp\WinGet\Kiwix.WikiMed.<IP_ADDRESS>\035080d826d851505358ea653aa0caafe4ab655f8d8ce9aaebe0eaec2319932a with zone: 2
2023-01-07 11:50:44.872 [CORE] Finished applying motw
2023-01-07 11:50:44.874 [CLI ] Successfully renamed downloaded installer. Path: C:\Users\gkant\AppData\Local\Temp\WinGet\Kiwix.WikiMed.<IP_ADDRESS>\KiwixWebAppWikiMed_<IP_ADDRESS>_AnyCPU.msix
2023-01-07 11:50:44.985 [CORE] Starting StagePackageAsync operation #0: C:\Users\gkant\AppData\Local\Temp\WinGet\Kiwix.WikiMed.<IP_ADDRESS>\KiwixWebAppWikiMed_<IP_ADDRESS>_AnyCPU.msix
2023-01-07 11:50:45.044 [CORE] Begin waiting for operation #0
2023-01-07 11:50:45.044 [CORE] Begin blocking for operation #0
2023-01-07 11:50:55.808 [CORE] Successfully completed #0
2023-01-07 11:50:55.808 [CORE] Starting RegisterPackageByFullNameAsync operation #1: Kiwix.WikiMed_<IP_ADDRESS>_neutral_~_qwjmsdmcnwx12
2023-01-07 11:50:55.823 [CORE] Begin waiting for operation #1
2023-01-07 11:50:55.823 [CORE] Begin blocking for operation #1
2023-01-07 11:50:56.290 [CORE] Successfully completed #1

@Karl-WE As developer of large apps (1.29GB), this issue is still frustrating me. I get very very slow download with Kiwix.WikiMed for example (hosted on GitHub). Here is a log of my installation of this app just now. It took 37 minutes to download 1.29GB, a download that normally takes no more than 5 or 6 minutes over my broadband.
2023-01-07 11:13:01.285 [CLI ] Failed to get msix signature hash, fall back to direct download.
2023-01-07 11:13:01.301 [CLI ] Generated temp download path: C:\Users\gkant\AppData\Local\Temp\WinGet\Kiwix.WikiMed.<IP_ADDRESS>\035080d826d851505358ea653aa0caafe4ab655f8d8ce9aaebe0eaec2319932a
2023-01-07 11:13:01.302 [CORE] Downloading to path: C:\Users\gkant\AppData\Local\Temp\WinGet\Kiwix.WikiMed.<IP_ADDRESS>\035080d826d851505358ea653aa0caafe4ab655f8d8ce9aaebe0eaec2319932a
2023-01-07 11:13:01.302 [CORE] DeliveryOptimization downloading from url: https://github.com/kiwix/kiwix-js-windows/releases/download/v2.2.62-WikiMed/KiwixWebAppWikiMed_<IP_ADDRESS>_AnyCPU.appxbundle
2023-01-07 11:48:27.275 [CORE] DeliveryOptimization error: 0x80d02002, extended error: 0x80190191
2023-01-07 11:48:27.276 [FAIL] D:\a\_work\1\s\external\pkg\src\AppInstallerCommonCore\DODownloader.cpp(289)\WindowsPackageManager.dll!00007FFB0EEB28F9: (caller: 00007FFB0EEB31C0) Exception(2) tid(29e4) 80D02002
2023-01-07 11:48:27.410 [CORE] Started applying motw to C:\Users\gkant\AppData\Local\Temp\WinGet\Kiwix.WikiMed.<IP_ADDRESS>\035080d826d851505358ea653aa0caafe4ab655f8d8ce9aaebe0eaec2319932a with zone: 3
2023-01-07 11:48:27.419 [CORE] Finished applying motw
2023-01-07 11:48:27.420 [CORE] WinINet downloading from url: https://github.com/kiwix/kiwix-js-windows/releases/download/v2.2.62-WikiMed/KiwixWebAppWikiMed_<IP_ADDRESS>_AnyCPU.appxbundle
2023-01-07 11:50:44.857 [CORE] Download hash: 035080d826d851505358ea653aa0caafe4ab655f8d8ce9aaebe0eaec2319932a
2023-01-07 11:50:44.857 [CORE] Download completed.
2023-01-07 11:50:44.858 [CLI ] Installer hash verified
2023-01-07 11:50:44.859 [CORE] Started applying motw to C:\Users\gkant\AppData\Local\Temp\WinGet\Kiwix.WikiMed.<IP_ADDRESS>\035080d826d851505358ea653aa0caafe4ab655f8d8ce9aaebe0eaec2319932a with zone: 2
2023-01-07 11:50:44.872 [CORE] Finished applying motw
2023-01-07 11:50:44.874 [CLI ] Successfully renamed downloaded installer. Path: C:\Users\gkant\AppData\Local\Temp\WinGet\Kiwix.WikiMed.<IP_ADDRESS>\KiwixWebAppWikiMed_<IP_ADDRESS>_AnyCPU.msix
2023-01-07 11:50:44.985 [CORE] Starting StagePackageAsync operation #0: C:\Users\gkant\AppData\Local\Temp\WinGet\Kiwix.WikiMed.<IP_ADDRESS>\KiwixWebAppWikiMed_<IP_ADDRESS>_AnyCPU.msix
2023-01-07 11:50:45.044 [CORE] Begin waiting for operation #0
2023-01-07 11:50:45.044 [CORE] Begin blocking for operation #0
2023-01-07 11:50:55.808 [CORE] Successfully completed #0
2023-01-07 11:50:55.808 [CORE] Starting RegisterPackageByFullNameAsync operation #1: Kiwix.WikiMed_<IP_ADDRESS>_neutral_~_qwjmsdmcnwx12
2023-01-07 11:50:55.823 [CORE] Begin waiting for operation #1
2023-01-07 11:50:55.823 [CORE] Begin blocking for operation #1
2023-01-07 11:50:56.290 [CORE] Successfully completed #1

i'm ok with the speed, could be faster, but i don't mind. at lest winget works so good that it shows me WHICH of my apps need an update. sometimes teamviewer doesn't work because it's updating in the background and winget doesn't tell me there is an update. i don't have to update immediately but it's good to have an overview if there is an update for edge, telegram (store version requires microsoft account), libreoffice, vlc, anydesk, ....

I am not declining your point it still exists and is only partly reproducible. Are you from EU? @ItzLevvie did a very in depht troubleshooting waiting for her instructions what could be helpful as a repro and what logs are needed from wireshark or fiddler.

yes germany (EU)

Try and see if https://github.com/microsoft/winget-pkgs/pull/93256 & https://github.com/microsoft/winget-pkgs/pull/93255 makes a difference. This removes the streaming install from the latest version of both packages.
Unfortunately, it's hard for some people to reproduce due to different configurations & locations but I was able to download Kiwix.WikiMed successfully in less than 5 minutes with 130 Mbps regardless if the manifest has SignatureSha256 or not.
WinINet with SignatureSha256 = less than 5 minutes
WinINet without SignatureSha256 = less than 5 minutes
There were a few times where WinGet done a fallback to direct download but still managed to install in less than 5 minutes.
I took a look at your logs and
2023-01-07 11:13:01.285 [CLI ] Failed to get msix signature hash, fall back to direct download.
2023-01-07 11:13:01.301 [CLI ] Generated temp download path: C:\Users\gkant\AppData\Local\Temp\WinGet\Kiwix.WikiMed.<IP_ADDRESS>\035080d826d851505358ea653aa0caafe4ab655f8d8ce9aaebe0eaec2319932a
2023-01-07 11:13:01.302 [CORE] Downloading to path: C:\Users\gkant\AppData\Local\Temp\WinGet\Kiwix.WikiMed.<IP_ADDRESS>\035080d826d851505358ea653aa0caafe4ab655f8d8ce9aaebe0eaec2319932a
2023-01-07 11:13:01.302 [CORE] DeliveryOptimization downloading from url: https://github.com/kiwix/kiwix-js-windows/releases/download/v2.2.62-WikiMed/KiwixWebAppWikiMed_<IP_ADDRESS>_AnyCPU.appxbundle
2023-01-07 11:48:27.275 [CORE] DeliveryOptimization error: 0x80d02002, extended error: 0x80190191
2023-01-07 11:48:27.276 [FAIL] D:\a\_work\1\s\external\pkg\src\AppInstallerCommonCore\DODownloader.cpp(289)\WindowsPackageManager.dll!00007FFB0EEB28F9: (caller: 00007FFB0EEB31C0) Exception(2) tid(29e4) 80D02002

WinGet spent 35 minutes doing a streaming install and it failed so WinGet reverted back to direct download and installed WikiMed in just 2 minutes.

I'm from the UK, and I mostly get these very slow speeds. I'm the developer of the WikiMed and Wikivoyage apps, which are very large because of the compressed copies of Wikipedia data contained in them, so I test installation of them a lot. Until very recently installation was completely broken with winget for large appx UWP apps due to a signature verification problem (signature was valid, but winget couldn't get if from GitHub and errored out). Now, you can see from the logs that the signature verification still can't be got, but instead winget defaults to downloading the app directly and verifying the signature after download, which is much better behaviour.
However, the speed is the issue under discussion here. Comparing winget install and Microsoft install of the same app (but different packages, because the winget one is on GitHub and the MS Store one is on the MS Store), the MS Store is way way faster, 5 or 6 mins versus 37 mins with the test logged above.
This isn't a criticism of winget. The problem is in the interaction between winget and GitHub servers. However, if I direct-download the exact same package from GitHub in my browser, the download is normal speed (5 or 6 mins, rarely exceeding 10 mins for 1.29GB). To me, this implies that winget could speed things up considerably by defaulting to a simple PowerShell Invoke-WebRequest command! The problem seems to come from the attempt to use Delivery Optimization. I think DO is highly acceptable for @hueldoeu's use-case, when wanting to do background updates of a number of apps, but if you want to download an app and use it straight away, it's frustratingly slow and is going to put people off from using winget (which I think is a great service, by the way -- I just want to improve the UX).

@ItzLevvie thanks for that analysis! I'll take a look at those issues.

@Jaifroid correct

Just got noticed about a new release of Winget @Jaifroid does it still repro with libreoffice or your packages?

@Karl-WE, I'll check when I get the new version and report back.

Just got noticed about a new release of Winget @Jaifroid does it still repro with libreoffice or your packages?

i'm ok with the speed. i am sure there are faster servers and i have a 1 gigabit speed connection at home (docsis 3.1) but at least i don't have to use firefox everytime i want to install a new version. cmd winget install/upgrade libreoffice is enough

Still suffering from this issue today with v1.5.2201 trying to upgrade Rancher Desktop or Podman Desktop.
Setting the downloader to wininet fixed it.

I have a slow internet connection. I can't download relatively big applications via winget. It always gets stuck halfway. Maybe adding resume download feature could be helpful.

issue still present in latest windows 11 when trying to install PowerToys : winget install Microsoft.PowerToys
setting a v high bandwidth limit for Delivery Optimisation in settings (999 Mbps) fixes it


Some plugins hold live pointers beyond liveness scope

The current implementation of the SDC plugin holds RTLIL objects between TCL commands.  Because of this, changes to those pointers (e.g. techmap passes, etc) cause those pointers to become invalid.
Test case for the SDC plugin attached: test.zip

There may be more examples of this.  In general, passes can only ensure liveness of RTLIL objects within the scope of the pass, and no longer.  Holding these pointers between invocations is not safe, and will result in memory corruption.

@tmichalak This is the highest priority issue, and must be resolved.  This is not optional.

@litghost so for SDC the issue is that we are storing pointers to RTLIL::Wire* in the Clock objects, right?
In case Yosys destroys the wires what should be the desired behavior apart from not crashing (in this a simple !=NULL will do)?
Removing the Wire and clock if that is the last or only one clock wire?

@litghost so for SDC the issue is that we are storing pointers to RTLIL::Wire* in the Clock objects, right?
In case Yosys destroys the wires what should be the desired behavior apart from not crashing (in this a simple !=NULL will do)?
Removing the Wire and clock if that is the last or only one clock wire?

You cannot hold the pointers at all.  It is not a safe behavior.  My recommendation is to defer module inspection until when you are ready to do so (e.g. write_sdc).  Having intermediate RTLIL::Wire* objects between TCL calls/yosys passes will always be unsafe.
Pointer checks are useless here, because yosys deleting an address won't update the pointer copy held in the Clocks object.

@litghost I created a draft PR targeting the issue.
Basically, I created a write_sdc script pass command which wraps the propagate_clocks and write_sdc commands effectively postponing the clock propagation to when write_sdc is called.
As for the pointers, I am just cleaning the clock structure after each run, so this is not optimal since every write_sdc triggers a new clock propagation step.
I am aware that we can't keep pointers between runs, so maybe instead of keeping RTLIL::Wire* objects, I could just keep the wire name.
However, in the test you added to the issue I think that cleaning the clock structure is actually the desired behavior.
One thing I noticed is that the second pass in your test just reads the design from json that was written in the first pass.
Currently that json doesn't keep any clock information, it probably should but that would require enhancing the write_json command. For know, you have to add the clocks (call all SDC commands again).

For know, you have to add the clocks (call all SDC commands again).

This is correct, but the SDC plugin doesn't get a signal from Yosys that the relevant objects no longer exists, hence the memory corruption angle.


Add support for outputting "metadata" in the render-json command

Something that would make render-json even more powerful for debugging is if the output included not only the fully-resolved values, but also other metadata, such as "breadcrumbs" that tell you where that value was found. This likely means changing the JSON output format, so we'd probably only want this with some flag like --with-metadata.
For example, let's say this is the original render-json output (truncated to keep the example simple):
{
  "generate": {
    "provider": {
      "comment_prefix": "# ",
      "contents": "# This is just a test",
      "disable_signature": false,
      "if_exists": "overwrite",
      "path": "provider.tf"
    }
  },
  "inputs": {
    "foo": "bar",
    "baz": "blah"
  },
  "terraform": {
    "source": "./foo"
  }
}

Here's what render-json --with-metadata might look like:
{
  "generate": {
    "provider": {
      "value": {
        "comment_prefix": "# ",
        "contents": "# This is just a test",
        "disable_signature": false,
        "if_exists": "overwrite",
        "path": "provider.tf",
      },
      "metadata": {
        "found_in_file": "/live/dev/vpc/terragrunt.hcl"
      }
    }
  },
  "inputs": {
    "foo": {
      "value": "bar",
      "metadata": {
        "found_in_file": "/live/dev/vpc/terragrunt.hcl"
      }
    "baz":  {
      "value": "blah",
      "metadata": {
        "found_in_file": "/live/terragrunt.hcl"
      }
  },
  "terraform": {
    "source": {
      "value": "./foo",
      "metadata": {
        "found_in_file": "/live/_envcommon/vpc.hcl"
      }
    }
  }
}

In other words, every value because an object with the keys:

value: The original value goes here.
metadata: Extra metadata goes in this object. Initially, the most useful metadata would be found_in_file, which gives you the file path where this value came from: you can see in the examples above, that may be a "child" config (/live/dev/vpc/terragrunt.hcl) or a parent config pulled in via include (/live/terragrunt.hcl, /live/_envcommon/vpc.hcl). In the future, we may also want to add metadata such as the name of the include that was used to pull in that file, the type of merging done, etc.

@denis256 Do you have any sense of the complexity of implementing this render-json --with-metadata functionality?

Also, it would be useful for locals metadata to indicate what locals are available in the parent config (includes). Even though they may not be available for use, it may be useful for deciding if one should use expose = true.

Hi,
to add additional fields on render-json I thought about:

adding of "metadata" map field to TerragruntConfig  to store metadata for all fields
add code to set metadata during execution
during render-json --with-metadata fetch metadata for respective field from map and include in output

This approach looks more feasible than changing each TerragruntConfig field to have "value" and "metadata"
Implementation shouldn't take more than a week

Roger. Do you have time to start on this now by any chance? Along with the related render-json changes in https://github.com/gruntwork-io/terragrunt/issues/2198?

Yes, started to work on it
Drafted a PR which include metadata for locals
Before changes render-json :
  "locals": {
    "aws_profile": "prod",
    "aws_region": "us-east-1",
    "aws_tags": {
      "test": "test"
    }
  },

with changes from PR:
  "locals": {
    "aws_profile": {
      "metadata": {
        "found_in_file": "/projects/gruntwork/terragrunt-tests/locals-tf/terragrunt.hcl"
      },
      "value": "prod"
    },
    "aws_region": {
      "metadata": {
        "found_in_file": "/projects/gruntwork/terragrunt-tests/locals-tf/terragrunt.hcl"
      },
      "value": "us-east-1"
    },
    "aws_tags": {
      "metadata": {
        "found_in_file": "/projects/gruntwork/terragrunt-tests/locals-tf/terragrunt.hcl"
      },
      "value": {
        "test": "test"
      }
    }
  },

Haha, you are fast. Thank you, that's great 👍 😁

Released in https://github.com/gruntwork-io/terragrunt/releases/tag/v0.38.6


MSID: 97916 Version: 2 DOI: 10.1101/2<IP_ADDRESS>2990

MSID: 97916
Version: 2
Preprint DOI: https://doi.org/10.1101/2<IP_ADDRESS>2990
Step 1. Awaiting reviews
Editorial to post reviews via hypothesis
Useful links:

DocMap: https://data-hub-api.elifesciences.org/enhanced-preprints/docmaps/v2/by-publisher/elife/get-by-manuscript-id?manuscript_id=97916
New model tracking: https://docs.google.com/spreadsheets/d/1_fHaoOy7hjyocptKtVJRijeNpUY4hBS7Ck_aVmx6ZJk/
Reviews on sciety: https://sciety.org/articles/activity/10.1101/2<IP_ADDRESS>2990

For trouble shooting (e.g. no Docmaps available):

DocMap issue addressing: https://miro.com/app/board/uXjVNCwK6EI=/
Explore DataHub DocMaps API: https://lookerstudio.google.com/reporting/4c2f0368-babb-4beb-b5b3-497e7e7b0f08/page/ejphD
Unmatched submissions and preprints: https://lookerstudio.google.com/u/0/reporting/9f86204f-3bf7-477c-9b18-5c5ef141bf69/page/p_gxi57ha93c
Unmatched manuscripts spreadsheet: https://docs.google.com/spreadsheets/d/15QcK8w-ssB7109RQEDtFpJPZ0J5HTGxoHa_2TtpMBbg/edit#gid=1336081641

Step 2. Preview reviewed preprint
Production QC content ahead of publication
Instructions:

QC preview: https://prod--epp.elifesciences.org/previews/97916v2
Update ticket with any problems (add blocked label)
When QC OK, add QC OK label to ticket and add publication date and time to https://docs.google.com/spreadsheets/d/1amAlKvdLcaDp5W8Z8g77NmkwbMF5n_u89ArSqPMO8jg
Move card to next column
(At end of the day post link in #enhanced-preprint and ask for PDF to be generated)

Useful links:

Preprint DOI :  https://doi.org/10.1101/2<IP_ADDRESS>2990
Confirm reviews returned by EPP: https://prod--epp.elifesciences.org/api/reviewed-preprints/97916/v2/reviews
To update the MECA path in the docmap: https://docs.google.com/spreadsheets/d/1mctCQuNFBjSn97Lihy7_vBO6z7-N-oqyLv4clyi6zHg

Step 3: Awaiting search reindex
This step adds the reviewed preprint to the homepage: https://elifesciences.org
The search reindex is triggered once an hour. We need the reviewed preprint to be indexed as the search application serves the journal homepage.
Useful links:

Jenkins pipeline to reindex search can be triggered sooner or monitored here: https://alfred.elifesciences.org/job/process/job/process-reindex-reviewed-preprints/

Step 4: Published! PDF requested
Waiting for PDF to be generated
Useful links:

PDF tracking: https://docs.google.com/spreadsheets/d/106_XeDjmuBae7gexOTNzg60lapeqjl2aRn9DzupGyS8/

Step 5: Introduce PDF to data folder and git repo
Upload PDF to relevent folder in git repo https://github.com/elifesciences/enhanced-preprints-data/
Step 6: Done!

[ ] Kettle is on!

warn: [journal-ref-text-content] This journal reference (id c26) has untagged textual content - , Kaasa, S. & . Is it tagged correctly?
error: [journal-ref-article-title] This journal reference (id c45) has no article-title element.
warn: [underline-warning] underline element contains more than 20 characters. Is this tracked change formatting that's been erroneously retained?


Particles2D

I added descriptions for all Particles2D methods (Not classes as the commit message says!)

Unfortunately, most of these method descriptions are going to go away. You are supposed to be documenting the member variables in the <member></member tags and ignoring the assorted get*/set* methods, as described in the contribution guide: http://docs.godotengine.org/en/latest/community/contributing/updating_the_class_reference.html
Also, please see the writing guidelines: http://docs.godotengine.org/en/latest/community/contributing/docs_writing_guidelines.html
This will help you stick to the vocabulary/wording that the other documentation is using.

Fair enough. I don't know how I missed the writing and documenting guidelines. I will follow them on my future contributions.

Thank you for contributing! If you have done fixing the files according to the guideline, please squash commits together. (PR workflow)

I moved the descriptions to the  tags and rewrote them following the guidelines.

Feeling a bit lost here. Maybe I should get some help on IRC.

Rebasing can be tricky. At this point since you're only changing one file, you might find it easier to just kill this PR, make a new branch from master, and copy your new Particles2D.xml in there.


X / Y axis center

Hi, I was wondering if there is a quick way to position the X/Y Axis in the center of the graph (or in different positions based on dynamic median)?
This is essentially what i am trying to achieve...

Hey @si-harps, thanks for checking out @data-ui. This isn't currently supported, but it's a great suggestion!
I think it wouldn't be too hard to implement, I think we'd just need to update the Axis components to take some type of offset prop, specifying the value of the other axis to place it at (e.g., in your example, <YAxis xIntersection={7500} />. I'd welcome a PR for this, else will add it to the list and will try to tackle it with a offset bug fix sometime this week.

Hi @williaster, thanks for the quick reply. I'll take a look at this today. Adding an xIntersection/yIntersection prop sounds like a great idea. Will get back to you asap


fix(provider): register Factory in LucidProvider

Closes: https://github.com/adonisjs/adonis-ignitor/issues/8

Hahaha I just fixed it in ignitor. https://github.com/adonisjs/adonis-ignitor/commit/fb3577f196ed4868b4c42483165de190acf48657
Sorry for the confusion, but I thought only 1% of projects will need the Factory during the HTTP server, so I don't want others to slow down their boot process

Coverage remained the same at 96.497% when pulling c36a81ab51e12eff9573d7626bfcd78a9ae8db4f on webdevian:patch-1 into 332f800f79cffaf58fd56586e9ad5aef8a7b54ac on adonisjs:develop.

Yes I think you are right, factories being used in the server is a rare use case and preLoading it is a good option.
Can you do a release of ignitor?

Yes released ignitor