Repro pageobject problem

Pull request context for a bug repro.

Yes I am going to make a ticket for this when I bring this branch back up to snuff. I thought I already had a ticket but I can't find it.

And now the branch has rotted, so I need to update it.


Unexpected error: FetchError: request to https://code4me.me/api/v1/prediction/autocomplete failed, reason: connect ECONNREFUSED <IP_ADDRESS>:8080

How can I fix this issue in VSC?

I realize it may the problem of VSC, TuT.


Fix renaming aggregated properties in groupBy with custom querySchema

Fixes #1035
Problem
Check issue description for more details
RenameProperties is not able to rename property since map on grouped query (created by groupBy) creates different ident to the one used in group by.
Solution
Add normalization step NormalizeAggregationIdent which substitute the correct ident so later it could be renamed.
The fix is not related to group by directly however this is only one known situation so far.
Checklist

[x] Unit test all changes
[x] Update README.md if applicable
[x] Add [WIP] to the pull request title if it's work in progress
[x] Squash commits that aren't meaningful changes
[x] Run sbt scalariformFormat test:scalariformFormat to make sure that the source files are formatted

@getquill/maintainers

sorry for the delay to review this


Fix service discovery auth issue on ASA-E

Description
Use the default constructor when creating an x509Certificate from a path on disk
Fix for 3.2 #1199
Quality checklist

[x] Your code complies with our Coding Style.
[x] You've updated unit and/or integration tests for your change, where applicable.
[ ] You've updated documentation for your change, where applicable.
If your change affects other repositories, such as Documentation, Samples and/or MainSite, add linked PRs here.
[x] There's an open issue for the PR that you are making. If you'd like to propose a new feature or change, please open an issue to discuss the change or find an existing issue.
[x] You've added required license files and/or file headers (explaining where the code came from with proper attribution), where code is copied from StackOverflow, a blog, or OSS.

/azp run Steeltoe.All


opencomposite: unstable-2023-09-11 -> unstable-2023-12-28

Description of changes
https://gitlab.com/znixian/OpenOVR/-/compare/cca18158a4b6921df54e84a3b23ff459f76a2bde...07321480e3db1d398d6ab1af5b06e1eb12881e9c
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

I can't seem to get either before or after to work with Steam to test this

After some testing, I can confirm this problem. I tried to get a backtrace, and it seems to occur in Monado, specifically in some Vulkan code:
#0  0x0000000000000000 in ?? ()
#1  0x00007fffe82d4dce in vk_name_object (vk=vk@entry=0x9d81c8, type=type@entry=VK_OBJECT_TYPE_COMMAND_POOL, object=<optimized out>,
    name=name@entry=0x7fffe8323fa0 "client_vk_compositor command pool") at /build/source/src/xrt/auxiliary/vk/vk_debug.c:38
#2  0x00007fffe82cfe9c in client_vk_compositor_create (xcn=<optimized out>, instance=0x69bca0,
    getProc=0x7fffe842d7b0 <vkGetInstanceProcAddr>, physicalDevice=0x678a90, device=0x706800,
    external_fence_fd_enabled=external_fence_fd_enabled@entry=true, external_semaphore_fd_enabled=true, timeline_semaphore_enabled=false,
    debug_utils_enabled=true, renderdoc_enabled=false, queueFamilyIndex=0, queueIndex=0)
    at /build/source/src/xrt/compositor/client/comp_vk_client.c:862
#3  0x00007fffe82cd9bf in xrt_gfx_vk_provider_create (xcn=<optimized out>, instance=<optimized out>,
    get_instance_proc_addr=<optimized out>, physical_device=<optimized out>, device=<optimized out>,
    external_fence_fd_enabled=external_fence_fd_enabled@entry=true, external_semaphore_fd_enabled=true, timeline_semaphore_enabled=false,
    debug_utils_enabled=true, renderdoc_enabled=false, queue_family_index=0, queue_index=0)
    at /build/source/src/xrt/compositor/client/comp_vk_glue.c:80
#4  0x00007fffe82b3b10 in oxr_session_populate_vk (log=log@entry=0x7fffffff0040, sys=sys@entry=0x69b220, next=next@entry=0x612458,
    sess=sess@entry=0x7cb010) at /build/source/src/xrt/state_trackers/oxr/oxr_session_gfx_vk.c:129
#5  0x00007fffe82aea2e in oxr_session_create_impl (out_session=0x7ffffffeffc8, xsi=0x7ffffffeffd0, createInfo=0x7fffffff0150, sys=0x69b220,
    log=0x7fffffff0040) at /build/source/src/xrt/state_trackers/oxr/oxr_session.c:848

So perhaps some Mesa update broke this for us?

Slighty OT.
I don't think it's a good idea to constantly keep packages like Monado and OpenComposite up-to-date. I had this idea for a while but finally decided to work on it, so now I have an automatically updating overlay for XR/AR/VR apps: https://github.com/Scrumplex/nixpkgs-xr
Right now it includes Monado and OpenComposite.
I hope to bring it into nix-community once I feel like it's in a good state.

After some testing, I can confirm this problem. I tried to get a backtrace, and it seems to occur in Monado, specifically in some Vulkan code ...

Welp. Good thing that I set up nixpkgs-xr :D
Looks like one of the many commits on January 2nd fixed OpenComposite on NixOS.
I have updated my Monado bump accordingly: https://github.com/NixOS/nixpkgs/pull/274526

I'll hopefully be able to try this out later today. Were you able to start a Steam game using this?

I don't think it's a good idea to constantly keep fast-moving packages like Monado and OpenComposite up-to-date. I had this idea for a while but finally decided to work on it, so now I have an automatically updating overlay for XR/AR/VR apps: https://github.com/Scrumplex/nixpkgs-xr

I personally don't think we need a separate repo for this. While development does happen quickly, it doesn't appear to be particularly unstable from the packaging side as evidenced by the fact that you were able to do an update after >3 months by simply bumping the src. Other projects switch frameworks and/or language a couple times in that time frame.
I think we should be able to keep up with it with maybe the help of a little automation and that's certainly less of a hassle to set up and maintain than a nix-community repo for like two packages?
This appears to do the trick for opencomposite:
  passthru.updateScript = nix-update-script {
    extraArgs = [ "--version=branch" ];
  };

nix-shell maintainers/scripts/update.nix --argstr package opencomposite

I'll hopefully be able to try this out later today. Were you able to start a Steam game using this?

Yep! H3VR worked perfectly fine

I personally don't think we need a separate repo for this. While development does happen quickly, it doesn't appear to be particularly unstable from the packaging side as evidenced by the fact that you were able to do an update after >3 months by simply bumping the src.

While that is true, my main issue is that both OpenComposite and Monado don't provide regular releases. It's mostly a shot in the dark when bumping as we have to hope that nothing is broken. Having a separate overlay, similar to nix-community/nixpkgs-wayland, allows people to always track the latest revisions for their XR stack without having to override packages themselves or wait for PRs to merge in Nixpkgs.
Considering that the number of open Nixpkgs PRs is slowly growing, I am not so sure about regularly updating unstable packages.
My main driver for doing so was so that I could use those newer versions conveniently on my own system.

Superseded by https://github.com/NixOS/nixpkgs/pull/283976


How to run player after build on ubuntu

I'm trying to run my player after its built, but I'm getting an error that says ./build/StandaloneLinux64: Is a directory. I'm not too familiar with ubuntu, so I'm not sure how I can run it.
- name: Build Project
  uses: game-ci/unity-builder@v2
  env:
    UNITY_LICENSE: ${{ secrets.UNITY_LICENSE_UBUNTU }}
  with:
    projectPath: IL2CPP_Tests
    targetPlatform: StandaloneLinux64

- name: Run Tests
  run: |
    ./build/StandaloneLinux64 -batchmode --result:TestResults/test-results-il2cpp.xml --trace=Info --labels=All

Is this possible?
(In case you're wondering why I want to do this, it's because I'm trying to run unit tests on the IL2CPP runtime directly using NUnitLite, since unity test runner doesn't work.)

The error seems accurate.

Is this possible?

If you application takes those flags I don't see why it wouldn't be possible, but you'll have to set up your runner to have all the requirements to run your app. Recommend starting out by getting it working on your machine, so you know the exact arguments to pass in. If you don't have access to a windows terminal you can always run it inside an ubuntu docker image.

I already got it working on my local windows machine. But I don't know how to run the built player on Ubuntu.

Since you're trying to get it to work for Ubuntu, you shouldn't be  testing it on a Windows machine (see previous comment).
For starters, read the error and make sure you're not targeting a directory, but the binary.
Then work with the output that you're getting.
I don't really know the specifics of NUnitLite.

Looks like I just needed ./build/StandaloneLinux64/StandaloneLinux64.

Weirdly, it doesn't actually work. The player never exits so the action eventually times out. I just re-jiggered it to run on a windows runner and it worked. I'm not sure if it's an issue with NUnitLite on the OS (though I wouldn't think so, it's just a netstandard2.0 dll), or something wrong with my passing arguments, or what. Which kind of sucks, because the windows runner is so much slower than ubuntu.


IPad Air 2 IOs 9.1 search text not inside search concept

Hi
First of all i love your search concept, its one of the best i ever saw. I validated your control for my project and for this it is necessary that it's running on tablets. In Ios 9.1 in safari, the search text is outside of the "search concept", see screenshot. If we can fix it, I definitely integration your control in my project.
Cheers
Tobias

That is a correct behavior.
The Search Box has 2 modes:

Search by parameter
Search as free text (the query property) -> the one you're seeing in your screen shot

I'll close this now. Let mw know if it is still unclear and reopen that issue if you like.


Automate Mobile Profiles

Just fooling around with the mobile profiles, I was wondering if there is any way to connect and control my mobile device with selenium automation.
When I try connecting to the profile id selenium throws the error: "selenium.common.exceptions.SessionNotCreatedException: Message: This profile is using the External Spoofing Engine, so Selenium connection is not allowed.'
Is there any work around you are aware of or other language which makes automation of the mobile device through Kameleo achievable?

Automation of mobile profiles was introduced since our latest reply. Please follow this article.
Just a quick example:

search for mobile base profiles:

base_profile_list = client.search_base_profiles(
  device_type='mobile',
  os_family='ios',
  browser_product='safari',
  language='en-us'
)

start profile with Chroma:

create_profile_request = BuilderForCreateProfile \
  .for_base_profile(base_profile_list[0].id) \
  .set_recommended_defaults() \
  .set_launcher('chromium') \
  .build()
profile = client.create_profile(body=create_profile_request)

set up touch events properly

client.start_profile_with_options(profile.id, body={
  'additionalOptions': [
    {
      'key': 'disableTouchEmulation',
      'value': True,
    },
  ],
})

Then you are ready to automate with each of the frameworks (Selenium, Puppeteer, Playwright)


ArangoDB endorsements tests

Currently the ArangoDB endorsements unit tests depend on a real ArangoDB instance running alongside the testing harness.
We should decouple the business logics from the store and let Arango sit behind an interface instead.

#21 is linked to this issue


more streamlined handling of point and cell data

For mesh formats that can only write one point/cell data array of int data, unconditionally pick the first one found and warn if there are more candidates.

Actually I still prefer maintaining a list of these official tag data for each format, since explicit is better than implicit

But that is implicit, too. The user never sees the "official tag list", it's a member of some function or class. When she writes a file, it's completely unclear why one set has been selected instead of another.
The way we have it right now, at least we have a warning about it. If the user thinks it's the wrong cell_data written out, she needs to provide only one cell_data item. That's as explicit as it gets.

Okay. So I suppose currently for practical use we must save "gmsh:physical" first in the cell_data?
Do you plan to add a test file for testing interoperability?
Also the MED format needs to be included. It's by far the most comprehensive format: point / cell data as well as point / cell tags are supported.
What about exposing the mesh.*_tags attributes?

So I suppose currently for practical use we must save "gmsh:physical" first in the cell_data?

Do you plan to add a test file for testing interoperability?

We can do that.

What about exposing the mesh.*_tags attributes?

Do you mean *_sets? (Otherwise I don't know what you mean.)

So I suppose currently for practical use we must save "gmsh:physical" first in the cell_data?

Actually for practical use downstream when reading Gmsh MSH 4.1,  I recommend users ignore "gmsh:physical" and instead read °.cell_sets, or rsther .cell_sets_dict`.
I don't know how to deprecate the old behaviour; it's part of the value of an attribute.
The complication is Gmsh version.. This is only for reading MSH 4.1, which is genersted by default by Gmsh 4. Older Gmsh can't generate MSH 4.1, only MSH 2.2. I believe Gmsh 3 is still distributed by a number of Linux package managers.
It would be easy enough to populate the .cell_sets` on reading any format with a simple cell-colouring (Gmsh's MSH 2.2, MEDIT, …), just with a bit of redundancy.
But what about attempting this in the property .cell_sets_dict? Should this dig into .cell_data for int-valued colourings?


GPT number of parameters

Hi, there seems to be a discrepancy between the GPT models described in the paper and the ones reported in the results.
According to the paper:

We use OpenAI GPT models corresponding to the GPT-3 model series in \citet{Brown2020}.  These are eight dense decoder-only transformers of varying sizes, from 125 million to 175 billion non-embedding parameters.

Whereas the results in the repo have a different set of model parameters. For example, the following code (for a single task but similar results are returned for other tasks):
import bigbench
bigbench_path = bigbench.__path__[0]
TASK_DATA = results.load_results(path=os.path.join(bigbench_path,'benchmark_tasks'))
TASK_DATA['protein_interacting_sites'].scores('Protein interacting sites prediction for file DSet_72.txt.', 'multiple_choice_grade', 0, 'GPT')

Returns
[ScoreSample(score=17.5, params=125000000),
 ScoreSample(score=17.22222222222222, params=350000000),
 ScoreSample(score=17.22222222222222, params=760000000),
 ScoreSample(score=17.22222222222222, params=1300000000),
 ScoreSample(score=17.22222222222222, params=3000000000),
 ScoreSample(score=42.22222222222222, params=6000000000),
 ScoreSample(score=68.61111111111111, params=13000000000),
 ScoreSample(score=72.5, params=200000000000)]

In other words, number of params varying from 125M to 200B.
Are these very rough approximations, data errors, something else?

Thanks for your question! Just a quick update that we're investigating now.


Fix selection of "Your account" first on loading "Notification" settings page.

GIF explains this glitch better:

For more info see the second point of https://github.com/zulip/zulip/pull/9676#issuecomment-394811677 which exists in master as well.
Steps to reproduce:

Focus notification setting section.
Reload the page and observe focused section on left panel carefully.
You will observe at first it's "Your account" focused and changed to notification setting after a lag.

@zulipbot claim


[BUG] VisualStudioCredential taking upwards of 10 seconds to retrieve access tokens

Library name and version
Azure.Identity 1.8
Describe the bug
When running our .NET Framework 4.8 Web Application locally using Microsoft Data SqlClient 5.0.1 and Active Directory Default authentication, we are seeing frequent "a task was canceled" exceptions with SQL Server database connections. These task exceptions appear to be caused by very slow retrieval of access tokens via VisualStudioCredential.
I stepped through VisualStudioCredential, and it appears access tokens are retrieved via Microsoft.Asal.TokenService.exe using the following command line:
.\Microsoft.Asal.TokenService.exe --resource “https://database.windows.net/” --serviceHubConfig “C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\servicehub.config.json” --verbose --tenant {redacted}

When I run this command manually, it takes upwards of 10 seconds to return an access token, which makes local development with Active Directory Default impossible for us.
Here is an example stack trace of the issue we are experiencing:
Microsoft.Data.SqlClient.SqlException
HResult=0x80131904
Message=A task was canceled.
Source=Framework Microsoft SqlClient Data Provider
StackTrace:
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.GetFedAuthToken(SqlFedAuthInfo fedAuthInfo)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.OnFedAuthInfo(SqlFedAuthInfo fedAuthInfo)
at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
at Microsoft.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.CompleteLogin(Boolean enlistOK)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean ignoreSniOpenTimeout, TimeoutTimer timeout, Boolean withFailover, Boolean isFirstTransparentAttempt, Boolean disableTnir)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential credential, TimeoutTimer timeout)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimer timeout, SqlConnectionString connectionOptions, SqlCredential credential, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance)
at Microsoft.Data.SqlClient.SqlInternalConnectionTds…ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions, SessionData reconnectSessionData, ServerCertificateValidationCallback serverCallback, ClientCertificateRetrievalCallback clientCallback, DbConnectionPool pool, String accessToken, SqlClientOriginalNetworkAddressInfo originalNetworkAddressInfo, Boolean applyTransientFaultHandling)
at Microsoft.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)
at Microsoft.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnection owningObject, DbConnectionOptions options, DbConnectionPoolKey poolKey, DbConnectionOptions userOptions)
at Microsoft.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)
at Microsoft.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)
at Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection) at Microsoft.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)
at Microsoft.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource1 retry, DbConnectionOptions userOptions) at Microsoft.Data.SqlClient.SqlConnection.TryOpenInner(TaskCompletionSource1 retry)
at Microsoft.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource1 retry, SqlConnectionOverrides overrides) at Microsoft.Data.SqlClient.SqlConnection.Open(SqlConnectionOverrides overrides) at TreelineUtilityLibrary.Database.DatabaseManager.GetDataInternal(String sql, String connectionString, IDictionary2 params, IEnumerable`1 outputParams, Int32 timeout, Boolean doReadOnly) in C:\Users\jbelaire\source\repos\AboveTheTreeline.release\treelineutilitylibrary\Database\DatabaseManager.vb:line 745

This exception was originally thrown at this call stack:
Azure.Identity.VisualStudioCredential.RunProcessesAsync(System.Collections.Generic.List<System.Diagnostics.ProcessStartInfo>, bool, System.Threading.CancellationToken)
Azure.Identity.VisualStudioCredential.GetTokenImplAsync(Azure.Core.TokenRequestContext, bool, System.Threading.CancellationToken)
Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(System.Exception, string)
Azure.Identity.VisualStudioCredential.GetTokenImplAsync(Azure.Core.TokenRequestContext, bool, System.Threading.CancellationToken)
Azure.Identity.VisualStudioCredential.GetTokenAsync(Azure.Core.TokenRequestContext, System.Threading.CancellationToken)
Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(Azure.Core.TokenCredential[], Azure.Core.TokenRequestContext, bool, System.Threading.CancellationToken)
Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(bool, Azure.Core.TokenRequestContext, System.Threading.CancellationToken)
Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(System.Exception, string)
Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(bool, Azure.Core.TokenRequestContext, System.Threading.CancellationToken)
Azure.Identity.DefaultAzureCredential.GetTokenAsync(Azure.Core.TokenRequestContext, System.Threading.CancellationToken)
…
[Call Stack Truncated]

Inner Exception 1:
TaskCanceledException: A task was canceled.

I raised this issue on the Visual Studio Developer Community forums, but I'm not getting any traction there:
https://developercommunity.visualstudio.com/t/Version-1770-Preview-20---WebApp-unab/10409641?port=1025&fsid=3b212c91-74c7-451e-966c-40aed6e28164&scope=follow&sort=recent&ref=native&refTime=1691104580788&refUserId=ca8acc17-6663-46d8-8c13-bc2c608a8318
Expected behavior
Ideally retrieving access tokens via VisualStudioCredential should only take a few milliseconds.
Actual behavior
Retreival of access tokens via VisualStudioCredential is taking upwards of 10 seconds.
Reproduction Steps
I'm not sure if this is a code issue per se, or an environmental/Azure/Visual Studio issue.
Environment
.NET  Framework 4.8.0
Microsoft Data SqlClient 5.0.1
Azure.Identity 1.8
Version 17.7.0 Preview 6.0

Hi @jbelaire-treeline.   Thank you for reaching out and we regret that you're experiencing difficulties.  Based on your insights around direct calls to Microsoft.Asal.TokenService.exe taking 10 seconds, we agree that appears to be the root cause of the latency in VisualStudioCredential that you're seeing.  We're unable to reproduce this locally or in our test infrastructure; it appears likely that this is related to your local environment or Visual Studio version.
Unfortunately, Microsoft.Asal.TokenService.exe is a part of Visual Studio and not something that Azure.Identity has any influence over nor insight into.  There's not much help that the Azure SDK for .NET can provide - we'll need to bring this to the attention of the Visual Studio team.  Since Visual Studio is not an open-source product, we're unable to transfer this issue directly on your behalf.
Your best path forward would be to Report a problem with Visual Studio.  That will ensure the folks best able to assist are engaged to do so.
I'm going to close this out; if I've misunderstood what you're describing, please let us know in a comment and we'd be happy to assist as we're able.


The power rename crashes few seconds after opening

Microsoft PowerToys version
v0.49.1
Running as admin

[ ] Yes

Area(s) with issue?
PowerRename
Steps to reproduce
Select multiple files, open context menu, click on power rename.
Window is open, but it is freezed and after few seconds it crash
✔ Expected Behavior
No response
❌ Actual Behavior
The power rename windows closes
Other Software
No response

tracked by https://github.com/microsoft/PowerToys/issues/14191  will be fixed with 0.51.  PR already in.


Windows Containers Support

Hi! It would be great to have a support of native windows containers. Do you have any plans for it?

Do you see what should we change in the plugin for that?

docker client detection for all OS version
docker info instrospection (to let ones select agents with correct docker deamon, but DOCKER_HOST)

anything else?
Let's have issues for each of such tasks to simplify the process

Right now docker exist on
Linux: Linux containers (full native)
Windows: Linux containers (as hyper-v vm, but shared folders work fine), Windows containers (full native)
macOS: Linus containers (as tiny vm, but shared folders work fine)
We can find out container engine type by parsing docker version.
Windows containers:
Server:
OS/Arch:      windows/amd64

All the rest containers:
Server:
OS/Arch:      linux/amd64

For windows containers maybe a good idea will be give user a choice between cmd and powershell shells.

#51

yes something other than /bin/bash or /bin/sh

@beaudryj you can join https://github.com/jonnyzzz/TeamCity.Virtual/pull/51 Right now we agree to implement only cmd shell. The idea is that you can call others shell via the basic ones (sh, bash, cmd).


Fix exception causes all over the codebase

Thanks for the PR!

I removed some of the change that applied to abstract_arrays.py since that code moved to core.py in another PR.

I just made a blog post about this change here: https://blog.ram.rachum.com/post/621791438475296768/improving-python-exception-chaining-with


Units should not be able to walk up through the bottom of bridges

Occurs when only spanning a shallow chasm.

Related to #905, #906.


Add support for custom Dart entrypoints

Usage:
In main.dart,
void main() {
  // I'm not a real main!
}

@pragma('vm:entry-point')
void anotherMain() {
  runApp(MyApp());
}

In runner.cc,
bool OnCreate() {
  dart_entrypoint_ = "anotherMain";
  ...

Caveats:

The app's main Dart file must declare main() even if it's unused. It's a requirement by the Dart compiler but not the flutter-tizen tool.
The use of custom entrypoints may break app builds if the app depends on one or more Dart-based plugins. This is supposed to be a very rare scenario.

Depends on https://github.com/flutter-tizen/engine/pull/152.
Closes https://github.com/flutter-tizen/engine/issues/111.

Public methods such as SetEntrypoint() may be added later. The current support is experimental.


Population Pyramid Updates

[x] Inconsistent rectangle rounding (one side is rounded, one side is square
[ ] No legend... which color is male, which is female
[ ] Show hard numbers instead of percentages

@chriswhong @kerrygathers they are expressed as percentages in the data profile - we'll just need to compute the counts from total population or change the dataset.

Preview of using estimates instead of percentages (with estimates computed using pop_11_15_acs column)

See #371


Modifier -> SwiftUI -> View fundamentals -> View -> Input and events modifiers: siriTipViewStyle(_:)

Doc: https://developer.apple.com/documentation/swiftui/view/siritipviewstyle(_:)

[ ] Swift implementation
[ ] Elixir implementation

LiveView Native does not have the SiriTipView, so this does not apply.


How is the branch node implemented?

How do the branch node choose different path based on different outputs?
I have found the traversal algorithm(iterateOverNodeDataDependentOrder(std::function<void(NodeDataModel)> visitor)*) in the source code but there seems to be no branch node related code...
I really want to understand the implementation, hope to get your help!

This is a part of Chigraph, you can find the implementation there


config: support online change lock manager config

What have you changed?
This pr is part of online change config.
This pr add support for online change lock manager config by adding new tasks to lock manger (
Task::ChangeConfig and Task::ChangeTTL).
What is the type of the changes?

New feature (a change which adds functionality)

How is the PR tested?

[ ] Unit test
[x] Integration test
[ ] Manual test (add detailed scripts or steps below)
[ ] No code

Does this PR affect documentation (docs) or should it be mentioned in the release notes?
No
Does this PR affect tidb-ansible?
No

/merge


[Feature Request]: Support line breaks with stamps

Feature Description
The stamp feature should support line breaks (e.g. \n) instead of printing them as "\n"
Why is this feature valuable?
With line breaks implemented, overflow of the text can be manually avoided
Suggested Implementation
The behaviour should be the same as watermarks, which already implements line breaks.
Additional Information
https://github.com/Stirling-Tools/Stirling-PDF/blob/d832a90de0ea3373de21f71c298230d27195de70/src/main/java/stirling/software/SPDF/controller/api/security/WatermarkController.java#L178-L223
No Duplicate of the Feature

[X] I have verified that there are no existing features requests similar to my request.

Can i work on this enhancement.

Hi @DogeAmazed
This feature has been successfully implemented, and the corresponding PR has been merged.
Feel free to test it out and let us know if you encounter any issues or have further suggestions.


Add support for FSL/TRACULA

TRACULA (Yendik, et al., 2011) is an automated pipeline to conduct a tract-based analysis of diffusion properties using FSL's diffusion tools and Freesurfer's anatomical pipeline. To make AFQ-Browser compatible with TRACULA's output we need:

[ ] Python code to read TRACULA stats table and write out in tidy data format for AFQ-Browser. See TRACULA example data available here:
ftp://surfer.nmr.mgh.harvard.edu/pub/data/diffusion_tutorial.tar.gz

[ ] 3d anatomy view of TRACULA ROIs. This could be done  average MNI locations that are saved in the TRACULA stats folder. See here for example of rendering of TRACULA nodes. dat GUI to select fibers or ultratube and AFQ or TRACULA render in 3js window

[ ] afqbrowser-assemble will write out fibers and ultrafiber from inputs. If left blank, we will use the hardcoded data

[ ] add a default afq ultratube

[ ] can keys be written as strings and that is used to populate other fields? This will handle the difference in number of fiber groups between afq and tracula. And it will be forward looking for new fiber groups

For the 3D Panel, I'm thinking that we'll have a "Fiber Representation" selector in the 3D gui control panel. When "all fibers" is selected, the behavior is as usual and "Fiber Opacity" controls the opacity of the individual streamlines. The core tubes are invisible. When "core fiber" is selected, the streamlines are invisible and "Fiber Opacity" controls that opacity of the core fibers.

I think this is the simplest interface. One disadvantage is that the user won't be able to show core fibers and streamlines at the same time. But after playing around a bit with the core fibers, I think that would be of limited utility anyway.
@arokem, @jyeatman, @anotherjoshsmith, @akeshavan: Your thoughts?

Would the alternative be to have two independent sliders for controlling
core fiber opacity and all fiber opacity? It might be nice to be able to
see where the core fiber representation is relative to the whole fiber
group. For example if we were to examine different ways to compute the core
fiber. But I think either works fine
Sent from a mobile device. Please excuse typos

Resolved by #181


Code improvement EncryptionResolverProvider and configuration settings validation

Description
Cleanup of Encryption resolver. Reduces code complexity in this class. Add setting validation
Quality checklist

[x] Your code complies with our Coding Style.
[x] You've updated unit and/or integration tests for your change, where applicable.
[x] You've updated documentation for your change, where applicable.
If your change affects other repositories, such as Documentation, Samples and/or MainSite, add linked PRs here.
[ ] There's an open issue for the PR that you are making. If you'd like to propose a new feature or change, please open an issue to discuss the change or find an existing issue.
[x] You've added required license files and/or file headers (explaining where the code came from with proper attribution), where code is copied from StackOverflow, a blog, or OSS.

@fennekit When I deploy an application, it shows a warning at startup. Is that intentional?
2023-08-09T15:53:12.978927182+02:00 Setting ASPNETCORE_URLS=http://<IP_ADDRESS>:8080
2023-08-09T15:53:13.578498289+02:00 info: Steeltoe.Management.Diagnostics.DiagnosticObserver[0]
2023-08-09T15:53:13.578521789+02:00       DiagnosticObserver TraceDiagnosticObserver Subscribed to Microsoft.AspNetCore
2023-08-09T15:53:13.578532089+02:00 info: Steeltoe.Management.Diagnostics.DiagnosticObserver[0]
2023-08-09T15:53:13.578535489+02:00       DiagnosticObserver AspNetCoreHostingObserver Subscribed to Microsoft.AspNetCore
2023-08-09T15:53:13.593481794+02:00 warn: Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[35]
2023-08-09T15:53:13.593501794+02:00       No XML encryptor configured. Key {83a14665-adaa-4bd6-b238-b93853255f32} may be persisted to storage in unencrypted form.
2023-08-09T15:53:13.669726629+02:00 info: Microsoft.Hosting.Lifetime[14]
2023-08-09T15:53:13.669753829+02:00       Now listening on: http://<IP_ADDRESS>:8080
2023-08-09T15:53:13.670411334+02:00 info: Microsoft.Hosting.Lifetime[0]
2023-08-09T15:53:13.670425434+02:00       Application started. Press Ctrl+C to shut down.
2023-08-09T15:53:13.670430034+02:00 info: Microsoft.Hosting.Lifetime[0]
2023-08-09T15:53:13.670434534+02:00       Hosting environment: Production
2023-08-09T15:53:13.670438134+02:00 info: Microsoft.Hosting.Lifetime[0]
2023-08-09T15:53:13.670442034+02:00       Content root path: /workspace/


Added prefer-readonly rule with fixes

PR checklist

[x] Addresses an existing issue: #0000
[x] New feature, bugfix, or enhancement

[x] Includes tests

[x] Documentation update

Overview of change:
Adds a prefer-readonly rule for private member variables that are never re-assigned.
Is there anything you'd like reviewers to focus on?
Am I missing any relevant test cases?
CHANGELOG.md entry:
[new-rule] prefer-readonly

@ajafff mind taking another look here?

In the AppVeyor build:

src/linter.ts(199,18): error TS2540: Cannot assign to 'program' because it is a constant or a read-only property.

Ping @adidahiya ?

lgtm -- CI status is not being reported, maybe because of the typescript@next test runner

@adidahiya CI was successful, but did not report. That's weird. Maybe reopening the PR and/or manually starting CI helps.
Besides this issue, there is still a TODO for you: https://github.com/palantir/tslint/pull/3490#issuecomment-344841563


Update README.md

just a correction of a typo

I'm closing it as it seems duplicated to PR #70.
Although we haven't agreed on how to update the documentation, I prefer not to update "main" yet,

@jrambla Just a note: the "Readme.md" is not part of the website documentation (which consists of anything under /docs, /mkdocs.yaml and /.github/workflows/mk-beacon-docs.yaml. IMO the Readme.md is one of the things which should be updated when necessary in main...


[Proposal] Introduce default_card utility method in Billable

That's something very often done at the application level because almost every app needs to display the last4 field of the used card to the user.
In my humble opinion, that's something worth having in the lib.
Thanks for considering it.

Got merged in #452.


Updated gem dependencies

Resolves conflicts, no errors noted so far.

Bump

Bump

@jbender @kevinelliott Sorry, did not have any time to work on personal stuff lately. Anyway, thanks for the PR. I am going to merge it, but if there is any problems let me know. The thing is that I am not on OS X anymore (switched to ArchLinux), so I can't test this.

@jbender @kevinelliott By the way, if you want I can add you as commiter. But it is probably better if someone forks the project though..

@RobertAudi I have tested this locally on OSX and it works great! I had some problems getting the convenience methods to work so I ended up calling the FontAwesomeKit form directly. I will try again later and see about contributing moving forward!


Jenkins tests with the address sanitizer are failing

Even though Jenkins says all test have passed, we have a total of 59 tests failing due to memory leaks. This is something @pervazea noticed on my PR. But I just checked that all the other PR and the master repo is also leaking the same memory. The master also leaks memory. Strangely, I wasn't able to reproduce it on my vagrant machine. Pervaze was unable to reproduce this with the same configuration.
This happens on Build / Ubuntu Xenial/gcc-5.4.0/llvm-3.7.1 (Debug/Test) for the cd build && make check -j4 || true test.

I'll talk to Chad tomorrow. We should be paying attention to the return value from the tests. Jenkins seems to have no mechanism to run a set of things, then report whether one of N items failed. You either stop as soon as one invocation fails, or ignore the error code and continue running. We should be able to write our own wrapper to address this, in due course.
At the moment, we've been trying to resolve the issue that Jenkins stalls waiting for "executors", even though resources are idle. Until this is fixed, we can't scale things up properly.

From: poojanilangekar<EMAIL_ADDRESS>Sent: Thursday, February 15, 2018 8:00 PM
To: cmu-db/peloton
Cc: Pervaze Akhtar; Mention
Subject: [cmu-db/peloton] Jenkins tests with the address sanitizer are failing (#1154)
Even though Jenkins says all test have passed, we have a total of 59 tests failing due to memory leaks. This is something @pervazeahttps://github.com/pervazea noticed on my PR. But I just checked that all the other PR and the master repo is also leaking the same memory. The masterhttp://jenkins.db.cs.cmu.edu:8080/blue/organizations/jenkins/peloton/detail/master/5/pipeline/18 also leaks memory. Strangely, I wasn't able to reproduce it on my vagrant machine. Pervaze was unable to reproduce this with the same configuration.
This happens on Build / Ubuntu Xenial/gcc-5.4.0/llvm-3.7.1 (Debug/Test) for the cd build && make check -j4 || true test.

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/cmu-db/peloton/issues/1154, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AhDfwBMOYbLZxf-wKEDKBjqx3nFY_Lz3ks5tVNMegaJpZM4SHvoX.

Also, I don't think the build command should be make check -j4 || true since that hides the errors.

Completely agree.

From: Prashanth Menon<EMAIL_ADDRESS>Sent: Thursday, February 15, 2018 10:22 PM
To: cmu-db/peloton
Cc: Pervaze Akhtar; Mention
Subject: Re: [cmu-db/peloton] Jenkins tests with the address sanitizer are failing (#1154)
With the exception of the ssl_test which is failing due to OpenSSL (see #926https://github.com/cmu-db/peloton/pull/926), all the others are reporting leaks in libpg_query - specifically creating the MemoryContexts. I recall fixing this a while back when we flipped on AddressSanitizer.
I believe the leaks are false positives. I added clean up code herehttps://github.com/cmu-db/peloton/blob/master/third_party/libpg_query/src/postgres/src_backend_utils_mmgr_mcxt.c#L111 and herehttps://github.com/cmu-db/peloton/blob/master/third_party/libpg_query/src/postgres/src_backend_utils_mmgr_mcxt.c#L111 which should get called when threads exit. What looks to be happening is that AddressSanitizer is kicking in before the pthread_exit() call is made.
In this case, it's memory that will get reclaimed by the OS on process exit.
Three possible solutions:

Add the lines from libpg_query this to an AddressSanitizer whitelist.
Create an explicit main() function for our tests that explicitly call pthread_exit() before exiting.
Make the memory context statically initialized without malloc().

Also, I don't think the build command should be make check -j4 || true since that hides the errors.

You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/cmu-db/peloton/issues/1154#issuecomment-366134567, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AhDfwO5Ufz5vD8IkjuMp4L-IQ9OSGEMCks5tVPRhgaJpZM4SHvoX.

Disregard what I said previously about make -k; it was nonsensical.
The purpose of the ||true is so that later steps, e.g., psql_test.sh and jdbc_validator.py can still run even if one of the unit tests fails.  I talked with @pervazea a bit about it this morning and I'm no longer even convinced that the current test invocation and collection is doing the right thing because the test results bar graph reports no failed tests even though the console logs clearly do.  It's on my short list to look into.

On the status of PR1134
We see in the Jenkins builds:
http://jenkins.db.cs.cmu.edu:8080/blue/organizations/jenkins/peloton/activity?branch=PR-1134
Failures:
1 - Xenial, Debug - Peloton failed with assertion failure
4 - same as 1.
6 - Trusty, Release - failed psql_test.sh, with data being inconsistent with reference output
12 - failed in communicating with Github. (Ignore this, nothing to do with Peloton)
I've tried to reproduce the failure(s) on my laptop, with no success, although I've not run the test a large number of times yet.
Using the docker container that Jenkins uses for these builds, it is possible to reproduce the assertion error. The problem is not 100% reproducible. Running the psql test in a loop, it usually fails after 4-6 iterations, but may not fail.
There are some additional complications to tracking this down:

No corefile is generated. I've checked the obvious settings with respect to corefiles and they are correct.

Running directly under GDB is no better. The message is approx:
During startup program terminated with signal SIGABRT, Killed.

Upon inspection, no process is running, so no backtrace or state examination is possible.

@pervazea I think we can add a breakpoint right before the assertion and check the value of GetLastReaderCommitId and then do a backtrace when it doesn't match the commit_id. Can we test this tomorrow?

We can try this. I believe I did exactly that, but it did not work.
Now, I was not sure that when I first did this, which type of build I had, since gdb complained about unknown filename. So, when we do this again, we should:

delete and recreate a debug build
then try inserting the breakpoint and running

Chad also suggested looking at lldb if gdb won't cooperate.

More on the psql failure.

Pooja thinks she has a fix, related to how the commit id is saved.

The failure we've been seeing, is intermittent. We determined today, that it only manifests when the Jenkins machine has a higher load. Even then, not 100% of the time.

Running via gdb in the Docker container, still doesn't work as expected. Somehow, the process terminates without gdb being able to stop it, so the assertion triggers, and there is no stack trace or core dump to examine. The last line, below, is the symptom of this problem.

peloton: /peloton/src/concurrency/timestamp_ordering_transaction_manager.cpp:641: virtual void peloton::concurrency::TimestampOrderingTransactionManager::PerformDelete(peloton::concurrency::TransactionContext*, const peloton::ItemPointer&, const peloton::ItemPointer&): Assertion `(GetLastReaderCommitId(tile_group_header, old_location.offset) == current_txn->GetCommitId())' failed.
During startup program terminated with signal SIGABRT, Aborted.

I tried adding a memory fence before the SetLastReaderCommitId function. However, this didn't fix the problem either. So now the assertion must necessarily pass. Not sure if there is anything else I am missing.

I tried on Friday to recreate this problem without success. Have solicited Chad's help to recreate.
I am not familiar with the interacting bodies of code, but have some speculation. Suggestions on what to instrument or how to debug would be welcome.
I suspect that after the assertion fails, the transaction level garbage collector executes and is destroying / freeing the structures GDB is trying to access, in order to honour the breakpoint. Is there a way to turn off the garbage collector, so we can examine the system state?

A quick note on how to induce enough load to reproduce:

ssh to jenkins
in one window:
docker run --rm -it pr1134:trusty
then inside that container, do:
cd /peloton/build; while : ; do make -j4 check; done
(repeat step 2 in second window)
in a third window
docker run -it pr1134:trusty
then inside that container, do:

cd /peloton/build
bash ../script/testing/psql/psql_test.sh

We recreated this with a couple of additional log statements right before the assert that trips:

last reader commit id was 0
transaction commit id was a non-zero, reasonable value.
The assertion did not trip. (This is odd, implying that the values changed between the time the LOG_INFO statements read them and the assertion was executed).

The psql test failed, with the update statement being executed matched 2 records, rather than 1.

2018-02-26 19:44:48 [src/concurrency/timestamp_ordering_transaction_manager.cpp:642:PerformDelete] INFO  - last rdr commit id 0
2018-02-26 19:44:48 [src/concurrency/timestamp_ordering_transaction_manager.cpp:645:PerformDelete] INFO  - cur commit id<PHONE_NUMBER>276
peloton: /peloton/src/concurrency/timestamp_ordering_transaction_manager.cpp:648: virtual void peloton::concurrency::TimestampOrderingTransactionManager::PerformDelete(peloton::concurrency::TransactionContext*, const peloton::ItemPointer&, const peloton::ItemPointer&): Assertion `(GetLastReaderCommitId(tile_group_header, old_location.offset) == current_txn->GetCommitId())' failed.

The problem appears to be independent of codegen.
No success when running under GDB (yet)

Success with GDB!
When GDB started, it had an error about being unable to turn off address space randomization. This was significant. Docker containers prevent various operations by default. To make GDB work, invoke the docker container like so (where pr1134:trusty is the docker container I'm using):
docker run --security-opt seccomp=unconfined --rm -it pr1134:trusty
Then GDB can correctly set breakpoints, and we break on them successfully.

The following approach looks to be working (but haven't examined carefully yet). Diff attached. Make Peloton think it has lots of CPUs, even though it doesn't. psql test failing on my laptop, whereas without this change, worked fine.
Thinking that a cleaned up version where we can manually pass in the value (when desired) via a command line parameter, might be useful for testing and flushing out problems of this type.
diff --git a/src/common/init.cpp b/src/common/init.cpp
index 17e7606..bb0d2af 100644
--- a/src/common/init.cpp
+++ b/src/common/init.cpp
@@ -30,14 +30,15 @@ namespace peloton {
ThreadPool thread_pool;
void PelotonInit::Initialize() {

CONNECTION_THREAD_COUNT = std::thread::hardware_concurrency();

// CONNECTION_THREAD_COUNT = std::thread::hardware_concurrency();

CONNECTION_THREAD_COUNT = 24;
LOGGING_THREAD_COUNT = 1;
GC_THREAD_COUNT = 1;
EPOCH_THREAD_COUNT = 1;
MAX_CONCURRENCY = 10;
// set max thread number.

thread_pool.Initialize(0, std::thread::hardware_concurrency() + 3);

thread_pool.Initialize(0, CONNECTION_THREAD_COUNT + 3);
// start worker pool
threadpool::MonoQueuePool::GetInstance().Startup();
@@ -47,7 +48,7 @@ void PelotonInit::Initialize() {
threadpool::MonoQueuePool::GetBrainInstance().Startup();
}

int parallelism = (std::thread::hardware_concurrency() + 3) / 4;

int parallelism = (CONNECTION_THREAD_COUNT + 3) / 4;
storage::DataTable::SetActiveTileGroupCount(parallelism);
storage::DataTable::SetActiveIndirectionArrayCount(parallelism);

I think we can do that by adding a parameter to the SettingsManager

@yingjunwu I think the bug lies in the AbortTransaction function of the TimeStampOrderingTransactionManager. When the psql_test.sh failed. I tried to find the version of the tuple it returned and the transaction id, begin cid and end cid of the old location. (Some background, the tuple is from the previous update statement which failed, so it contains the value <1,1000> which failed because the primary key constraint was violated).
Here are the contents of old_location:
Invalid Tuple <id = 1, year = 1000>
Transaction ID<PHONE_NUMBER>78
Begin Commit ID<PHONE_NUMBER>72
End Commit ID<PHONE_NUMBER>3709551615
Current transaction ID<PHONE_NUMBER>78
I think we are having this issue because in the code block from 986 - 1018, we never modified the end CID of the UPDATE object. Ideally we should've set it to INVALID_CID to ensure that it is not visible to any transaction. What do you think?

Do you mean that we did not set the end commit id of the newly created version (for the updated tuple) back to INVALID_CID?

Yes.

@yingjunwu I tried the fix I suggested. But that didn't work. I have a question regarding the way the PerformUpdate function is designed. Why is the index being updated even before the transaction commits? I specifically mean this line:
    UNUSED_ATTRIBUTE auto res =         AtomicUpdateItemPointer(index_entry_ptr, new_location);
Could this be causing the concurrency bug? Can this update be made when the transaction commits?

see #106

Fixed in #1272 & #1263


Upstream test failures

I think these are all just FutureWarnings coming from pandas. I'll try to take a look this week or next.
From: https://github.com/dask/dask/runs/2795401230
Here's the breakdown AFAICT
6: FutureWarning: The 'freq' argument in Timestamp is deprecated and will be removed in a future version.
24: FutureWarning: In a future version of pandas all arguments of concat except for the argument 'objs' will be keyword-only
12: FutureWarning: In a future version of pandas all arguments of DataFrame.drop except for the argument 'labels' will be keyword-only

We should maybe ensure that unpickling a Timestamp does not generate a warning?

Agreed.

I think the best thing from our end would be to not include the freq when selecting from a DatetimeIndex is that possible?

I'd be happy to see that change, but its an API change so i expect there would be pushback about doing it without the deprecation.

I opened https://github.com/pandas-dev/pandas/issues/41949 to keep track of fixing the unpickle on the pandas side

I think the best thing from our end would be to not include the freq when selecting from a DatetimeIndex is that possible?

I'd be happy to see that change, but its an API change so i expect there would be pushback about doing it without the deprecation.

@jsignell what do you actually mean exactly with "include the freq when selecting from a DatetimeIndex"?
You mean that if you slice a Series/DataFrame that has a DatetimeIndex with Timestamp objects, like s[Timestamp(..):Timestamp(..)] ?

I just mean if you select one value from an index. It should not have a freq.
This is what I am proposing:
import pandas as pd

pd.date_range("2012", periods=3, freq="D")[0]
# Timestamp('2012-01-01 00:00:00')

Ok I am going to close this since tests are passing on dask. But we are still affected by pandas-dev/pandas#41949 as described here: https://github.com/dask/dask/pull/7795#issue-668361488

Thanks for taking care of this @jsignell!

I just mean if you select one value from an index. It should not have a freq.

I don't think you should care too much about this case where the accessed Timestamp has a freq. In most cases that doesn't matter (it's only for a few specific properties, like is_year_start, where it gives different behaviour, and I don't think dask is testing those)

Yeah, I got rid of all the tests where we specifically are checking for freq. So the remaining issue is just with the pickle where the freq value gets serialized but then when we try to deserialize we get the freq deprecation warning.

Yes, we will ensure to fix that on the pandas side before the final 1.3 release.

Yes. In a future version Timestamp will have no 'freq' attribute, so we've deprecated passing it as an argument or accessing the property.

Sorry if this reopens the issue - that's not my intention - but I cannot find the thread where this has been discussed and decided. I'd like to see if there is a way for me to code around this deprecation. Could someone point me to the correct thread? Many thanks.

That would be somewhere in the pandas issue tracker or codebase @rwijtvliet, Dask is just following :)

Ah right, didn't see I was in the wrong project 😅


Added link to Jitsi documentation on setting up authentication

The default Jitsi configuration shown in the tutorial allows anyone accessing the website to create chatrooms. This might not be desirable because of bandwidth concerns, and also not wanting to allow unethical use of the service. I added a link to the original documentation that shows how to set up authentication so that only chosen users can create chatrooms while still allowing random guests to join them.

This is a good idea for an addition, but it should be actually included in the tutorial, not linked to on another site.

I have added a section called "Security" to the article presenting the easiest way to enable authentication based on the Jitsi documentation.
I have tried this solution myself and verified that it works on a VPS with Ubuntu 20.


Keras: custom data validation callback on training data always returns validation data results

Just for information: I already asked for help regarding this problem on stackoverflow. Since I did not get an answer yet, I assume that this is a non-trivial problem and therefore open an issue here.
System information
python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)" returned
v2.3.0-54-gfcc4b966f1 2.3.1
Describe the current behavior
When calling model.evaluate on training data (or any other data set) within a callback, always the results of the validation data set are returned.
Describe the expected behavior
I expected to get results for the data set passed to model.evaluate - no matter if called from within a callback or not.
Standalone code to reproduce the issue
I created a notebook that shows the problem:
https://colab.research.google.com/drive/1H-3ULqyRZCpaasXpU1foLkEg12fYNkYK?usp=sharing

I am able to replicate the issue reported,please find the gist here.

@oXwvdrbbj8S4wo9k8lSN May be I am missing something. I checked your colab. Data looks different and the results (losses) are also different as expected. Can you please check the gist here where I printed different datasets and results. Thanks!

@jvishnuvardhan I do not totally get the concept of gists. Therefore, please excuse if I state the obvious.
Between the colab notebook and the gist, something in the behavior of Keras/Tensorflow changed. In my notebook (with tensorflow version 2.3.0), the output of the history is
{'custom_loss': [0.7624925374984741, 0.5331208109855652],
 'loss': [0.9665887951850891, 0.6637843251228333],
 'val_loss': [0.7624925374984741, 0.5331208109855652]}

Here, the results for custom_loss and val_loss are the same although they should differ since the inputs differ. In the gist (currently tensorflow version 2.5.0-dev20201104), however, the output is
{'custom_loss': [0.7694963216781616, 0.541864812374115],
 'loss': [0.9665887951850891, 0.6637843251228333],
 'val_loss': [0.7624925374984741, 0.5331208109855652]}

Here, the outputs differ, which is the behavior I expected. I tested different versions of tensorflow and it seems that the change happened between version 2.3.1 and version 2.4.0rc0. Therefore, I guess that it is actually a bug but you already fixed it at least in the development versions.

@jvishnuvardhan I tested version 2.2.1 and also with this version, the outputs differ (as expected). It seems that the problem exists only in versions 2.3.0 and 2.3.1.

@oXwvdrbbj8S4wo9k8lSN May be there was a bug in TF2.3 but I just checked with most recent tf-nightly and !pip install tensorflow==2.4.0rc0 and both showed different results for custom_loss and val_loss.
Please use recent TF versions. In near future there will be stable TF2.4 which works better than TF2.3.
I am closing this issue as this was resolved in recent versions. If you see the same issue with recent version, feel free to reopen the issue. Thanks!


Move the CrashReporter to the PC facade and make it possible to embed it into a Terasology workspace

Okay I'm officially on vacation/sabbatical woohoo! First stage is only till mid-July, but hoping by then to have another couple months lined up :-)
Dug into the idea of having the CrashReporter embedded in a Terasology workspace as a library entry (same approach could be applied to other stuff like Gestalt) and got it working after a bit, sure has been a while but I missed it!
This was in part for @mtjvankuik to make it easier to make changes to the CR then immediately test them via Terasology. Reference: #1120 + https://github.com/MovingBlocks/CrashReporter/issues/29
Pinging @msteiger and @rzats for some review as well, especially for how this might handle in Eclipse rather than IntelliJ. I believe you can already link to the CR in a big Eclipse workspace, but it takes some setup, while this hopefully should make it as simple as running a couple Gradle commands.
Contains
Most the changes are specific to .gradle files, but I renamed the three sub-components in the CrashReporter to be less ambiguous (particularly "core" and "terasology") which means almost everything will show up as changed. So I thought I'd highlight the actual changes here to make it easier. Paths are relative to how they'll appear in a Terasology workspace with the CR embedded, but in reality naturally anything libs/CrashReporter/... is in the actual CR repo.

engine/build.gradle - The dependency on CR has finally been moved from here to the PC facade. Apparently a transitive dependency on some Google stuff in the CR led to misdirected imports in a few places (like Preconditions being present with two different import paths) - fixed
facades/PC/build.gradle - The CR dependency was moved here and made conditional to check for source vs binary. Can be done with a cleaner Gradle dependency substitution trick later, but that's for another PR
libs/subprojects.gradle - Enhanced to consider the presence of a settings.gradle in a library, such as the one that for the CR hooks up its three sub-components (which this way works both embedded and standalone)
libs/CrashReporter/settings.gradle - simplified sub-component naming and made the include paths conditional on whether the CR is embedded or standalone
libs/CrashReporter/build.gradle - added idea plugin and tweaked the setting of group to work the same both embedded and standalone
libs/CrashReporter/cr-core/build.gradle - no actual changes other than the different path (was "core" is now "cr-core")
libs/CrashReporter/cr-destsol/build.gradle + cr-terasology version - renamed and updated to work at different levels in the Gradle project tree

The imports in various Terasology engine classes have been changed but nothing else. This shouldn't happen again as now engine no longer has access to the transitive dependencies of the CR with its alternative Google stuff (something about guava vs. an HTTP gadget). Sure confused the heck out of me for a little while!
How to test

Grab the embeddedCR branch for Terasology
gradlew fetchLibCrashReporter
gradlew clean cleanIdea idea
Restart IntelliJ (should prompt you)
Have a way to crash the game, I usually just use the Sample module's crash block (console giveblock reallyCrashGameBlock then place and e click it)
Change something in the source version of the CR like the "editBeforeUpload" value in MessagesBundle.properties so you can verify you actually are using the embedded CR rather than the binary dependency
Be sure to run with the CR enabled (non-default utility run config provided for IntelliJ - "Terasology PC (CR enabled)")
Crash! Confirm that the CR pops up and that your change is present
Close IntelliJ
Delete libs/CrashReporter
Run gradlew clean cleanIdea idea again
Retest - you should now again be using the binary dependency

Since this messes with project structure some it may need some extra cleaning. Plain gradlew idea isn't necessarily enough.
Outstanding before merging

[ ] Check on any misdirected imports across modules (I'm working in a mini workspace for Gradle without my usual full module setup)
[ ] Increment the CR version to 4.0.0, release it, and make sure its new Artifactory name matches the dependency in Terasology
[ ] Remove the hard link to the CR PR branch in the Terasology utility.gradle (just included for easy testing)
[ ] Also check on Destination Sol - bump the version of the CR it uses (past v1.4) and make sure it has the updated name if changed.
[ ] Double check on import order (I edited some imports in-place and re-added others, so the order differs a bit) and other small stuff, maybe update the license header year etc

Screenie!

Hooray Jenkins reported success with all tests good!

Will have a look shortly - sorry for the delay

Can confirm that the entire testing sequence works as intended :smiley:

Some more stuff for the outstanding checklist:

The input order is definitely off (not alphabetically sorted) in BlockManagerImpl.java and TerasologyEngineBuilder.java. I think it's fine everywhere else, though.
cr-destsol needs to be removed since it doesn't serve any purpose - ditto for cr-terasology in Destination Sol.

Thanks for the review @rzats :-)

The input order is definitely off (not alphabetically sorted) in BlockManagerImpl.java and TerasologyEngineBuilder.java. I think it's fine everywhere else, though.
cr-destsol needs to be removed since it doesn't serve any purpose - ditto for cr-terasology in Destination Sol.

Could you expand on these? The first one ... import order? Second one removed from where? The Gradle dependency declaration on the CR? In existing code preceding this PR in Terasology that's CrashReporter-terasology which I'll update during merging to whatever name it ends up with in Artifactory. Maybe I'm missing something? :-)

@Cervator: sorry, that should have been import order. D'oh!
As for the second issue: I've expected CrashReporter-terasology to only include the cr-core and cr-terasology submodules, but it includes cr-destsol as well. This needs to be fixed within CrashReporter's Gradle configs - ideally it should be split into two Artifactory repos (one with core&terasology, the other with core&destsol submodules).

@rzats I'm still not entirely sure where exactly you're referring too, can you link to any involved lines? Sorry for me being so dense :-)
If you mean how https://github.com/MovingBlocks/CrashReporter/blob/b3245089719b00fe2dc8c59f386b9b39686a46d8/settings.gradle is referencing all three sub-projects then that's purely for the Gradle project tree, not ultimately something that gets used in a published Terasology or DS flavored jar in Artifactory.
For instance in the https://github.com/MovingBlocks/CrashReporter/blob/b3245089719b00fe2dc8c59f386b9b39686a46d8/cr-terasology/build.gradle the only actual library style dependency is the cr-core - no sign of DS anywhere. Same for the Terasology PC facade Gradle.
I'm probably still missing something!
As for a multi-repo setup well you could split out CrashReporter "plugins" or "adapters" or whatever we might call the app specific stubs. But they should be published to Artifactory as individual jars as-is even from a single repo. If we get to a point where the CR suddenly picks up in 3rd party activity then yeah at that point it could make sense to not have any app stubs in a central CR repo :-)

@Cervator: sorry, that was some confusion on my part - gradlew fetchLibCrashReporter downloads the entire repo including the DS adapter so I originally thought it's included as a Gradle dependency, but looks like the actual dependency is on CrashReporter-terasology - no problems here :sweat_smile:
Come to think of it I wouldn't split the game-specific facades (CrashReporter-terasology/CrashReporter-destsol) into separate repos just yet. Since the configurable variables are prone to change refactoring them will require going through three separate repos to modify the core code, then the respective files.

Yep any split would happen way later :-)
Thanks again for the review!

Looks good to me. I think that we can merge this one.

After just about exactly a month I've finally merged my own PR woo! That should highlight the impartiality of merge durations, hehe :D
@mtjvankuik: You should be able to dev on the CR in an embedded Terasology workspace now in pure source, woo!
@tdgunes @emanuele3d @rzats @Josharias @xrtariq2594: Be advised of minor import changes / sorting in some stuff affecting you (EventualSkills was the only affected module in the stable lineup). May cause conflicts in said imports only, should be pretty minimal.
I only hit files covered by this PR but added minor commits on top not visible here since I sent them directly to develop as part of the merge. https://github.com/MovingBlocks/Terasology/commits/develop should show all related commits with one or two including import sorting and such.
After pulling latest from develop a fresh gradlew idea (or comparable) should be done to pick up the updated lib dependency / adjust project structure (engine no longer depends on the CR)


NO-TICKET: Participation log message: separate inactive from faulty

Log inactive and faulty validators in separate lists.

bors r+

bors r+


compactOnLaunch does not compact when it should

How frequently does the bug occur?
Sometimes
Description
I have a database with a very limited amount of used bytes, although the overall file size is well over 50 MB. I expected this database to compacted on launch by using RealmConfiguration.Builder(....).compactOnLaunch(), but that didn't happen.
Checking the documentation of DEFAULT_COMPACT_ON_LAUNCH_CALLBACK, I see this comment:

The default implementation for determining if a file should be compacted or not. This implementation will only trigger if the file is above 50 MB and 50% or more of the space can be reclaimed.

But it seems that the current implementation is checking if less than 50% can be reclaimed, so the other way around than described in the comment.
Adding the last assertion to the unit test confirms this:
@Test
fun `doCompact test`() = runTest {
    assertEquals(false, LocalStorageInitializer.doCompact(50 * 1024 * 1024, 40 * 1024 *1024)) // not > 50MB and < 50% reclaimable -> should be false
    assertEquals(false, LocalStorageInitializer.doCompact(50 * 1024 * 1024 + 8, 25 * 1024 * 1024)) // > 50MB and > 50% reclaimable -> should be true
    assertEquals(false, LocalStorageInitializer.doCompact(50 * 1024 * 1024 + 8, 25 * 1024 * 1024 + 3)) // > 50MB and > 50% reclaimable -> should be true
    assertEquals(true, LocalStorageInitializer.doCompact(50 * 1024 * 1024 + 8, 25 * 1024 * 1024 + 4)) // > 50MB and >= 50% reclaimable -> should be true
    assertEquals(true, LocalStorageInitializer.doCompact(50 * 1024 * 1024 + 8, 25 * 1024 * 1024 + 5)) // > 50MB and < 50% reclaimable -> should be false

    // adding the following test, more than 50MB filesize and 25 bytes used, should be compacted
    assertEquals(true, LocalStorageInitializer.doCompact(50 * 1024 * 1024 + 8, 25)) // > 50MB and > 50% reclaimable -> should be true
}

Stacktrace & log output
N/A

Can you reproduce the bug?
Always
Reproduction Steps
Use the unit test doCompact test, that currently exists in the codebase.
Version
all versions since https://github.com/realm/realm-kotlin/commit/10acd38d4c3a8dae6c3ed9ba27aeb709df4965bd
What Atlas App Services are you using?
Local Database only
Are you using encryption?
No
Platform OS and version(s)
Android 10+
Build environment
Android Studio version: 2022.2.1
Android Build Tools version: 33.0.2
Gradle version: 7.6.1

Fixed by #1384


Spacebudz api and how you serve it

Hey @alessandrokonrad , just revisiting the api query -
From what I've gathered, in the spacebudz-api project you shared the program syncs with the cardano node, reads transactions relevant to the spacebudz project/given policy id and dumps the information into a db file.
So I was wondering - did you have a separate project that serves https://spacebudz.io/api/? I'm guessing this api must read from the db that spacebudz-api dumps into and has the routes /multisig/, /v2/common, /v2/spacebud/budId, /v2/listings and /v2/bids.
I hope my question makes sense, I feel like I must be missing something obvious! I was looking as trying to set up an api based off the api calls made from the marketplace but I'm not too sure what the multisig routes are doing.
Thanks again :) Naeri

Hey, yes exactly https://spacebudz.io/api/ is separate project. It's an API running on Firebase. I feed this APIs all the data coming from the open sourced spacebudz-api library.

I hope my question makes sense, I feel like I must be missing something obvious! I was looking as trying to set up an api based off the api calls made from the marketplace but I'm not too sure what the multisig routes are doing.

Some of these routes relate to something else than the on-chain marketplace data. /multisig relates to our SpaceBudz multisig wallet.

Thanks that helps heaps :)


Backport of purge cluster updates

Backport of #1235

For the record, v2.1.5 was tagged with this fix.


examples: code examples for logger and metrics have compile errors.

Bug description
Provided code examples have compile errors. And the to the Tracer from the readme is broken
Expected Behavior
Code examples should compile
Current Behavior
Code examples don't all compile
Possible Solution
Fix code examples
Steps to Reproduce

Click on Tracer link goes to 404
metrics / logger examples have compile errors in webstorm and vscode

cd packages/metrics
npx tsc examples/*.ts

Environment

Powertools version used: main
Packaging format (Layers, npm):
AWS Lambda function runtime:
Debugging logs:

Related issues, RFCs

Indeed, thx for reporting this, we were planning to potentially remove them since they are duplication of unit tests and global examples.

Ok. then there is also uploading the Readme, of whether or not to point out these examples or not.
Example projects are really handy if they work :)

The Tracer example also goes to a 404:

Example projects are really handy if they work :)

The issue with current per package examples as they are today is that they can't really be copied and paste since their imports point to local implementation (for instance: import { Logger } from '../src';) so we were tempted to go more for what is in root examples folder to have only "compilable"/"deployable" examples and remove the packages' level one.

The Tracer example also goes to a 404:

Indeed that bad ...


WIP?

This is very interesting. Is it ready to run?
I dont get anything in browser except a line at the top of the screen when i run and serve.
Thanks for doing this work!

I stumbled upon an error about the types being opaque, being unusable for a demo.
Anyways, I have fixed it by letting the type system do the inference work, and I have added an example code in Index.re for a Counter app :)

Also, I am trying out some implementations on how to mimic the current features of my other state management library for React which has more fine-grained and reasonable component updates than what it is right now.

A 1:1 implementation has been made and released! 🎉

Example in the folder throws error. FYI.
➜  re-scoped-model-master yarn
yarn install v1.19.1
warning ../../package.json: No license field
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 🔨  Building fresh packages...
✨  Done in 5.41s.
➜  re-scoped-model-master cats
{
  "build": "bsb -make-world",
  "start": "bsb -make-world -w -ws _ ",
  "clean": "bsb -clean-world",
  "server": "moduleserve ./ --port 8000",
  "test": "echo \"Error: no test specified\" && exit 1"
}
➜  re-scoped-model-master yarn start
yarn run v1.19.1
warning ../../package.json: No license field
$ bsb -make-world -w -ws _
Different compiler version: clean current repo
Cleaning... 46 files.
[37/37] Building src/ReactDOMRe.cmj
>>>> Start compiling
[14/16] Building src/Example-ReScopedModel.cmj
FAILED: src/Example-ReScopedModel.cmj src/Example-ReScopedModel.cmi /Users/prisc_000/Downloads/re-scoped-model-master/src/Example.bs.js
/Users/prisc_000/Downloads/re-scoped-model-master/node_modules/bs-platform/lib/bsc.exe -nostdlib -bs-package-name @lxsmnsyc/re-scoped-model -bs-ns ReScopedModel  -bs-package-output commonjs:src -color always -bs-suffix -I . -I src -I /Users/prisc_000/Downloads/re-scoped-model-master/node_modules/reason-react/lib/ocaml -I /Users/prisc_000/Downloads/re-scoped-model-master/node_modules/bs-platform/lib/ocaml -w -30-40+6+7+27+32..39+44+45+101 -bs-super-errors -bs-no-version-header -o src/Example-ReScopedModel.cmj src/Example.reast

  We've found a bug for you!
  /Users/prisc_000/Downloads/re-scoped-model-master/src/Example.re 27:18-47

  25 │ };
  26 │
  27 │ module Counter = ReScopedModel.ScopedModel.Make(CounterHook);
  28 │
  29 │ module Count {

  The module or file ReScopedModel.ScopedModel.Make can't be found.
  - If it's a third-party dependency:
    - Did you list it in bsconfig.json?
    - Did you run `bsb` instead of `bsb -make-world`
      (latter builds third-parties)?
  - Did you include the file's directory in bsconfig.json?

[15/16] Building src/Emitter-ReScopedModel.cmj
FAILED: subcommand failed.
>>>> Finish compiling(exit: 1)

I am not aware of that, I tested the Example as well as created a new ReasonReact project and seems to work fine with me.
I'll check more about the issue and see if still persists, thanks for letting me know!

I have removed the Example.re and replaced it with an example folder which is a demo for the re-scoped-model. Let me know if the issue persists

Hey boss, just tried it. Compiles with warnings but throw following error in the browser. Renders a blank screen. Your previous version worked fine!

 Warning: render(): Rendering components directly into document.body is discouraged, since its children are often manipulated by third-party scripts and browser extensions. This may lead to subtle reconciliation issues. Try rendering into a container element created for your app. /node_modules/react-dom/cjs/react-dom.development:530:32
23:06:42.577
Error: Invalid hook call. Hooks can only be called inside of the body of a function component. This could happen for one of the following reasons:
1. You might have mismatching versions of React and the renderer (such as React DOM)
2. You might be breaking the Rules of Hooks
3. You might have more than one copy of React in the same app
See https://fb.me/react-invalid-hook-call for tips about how to debug and fix this problem. load.js line 68 > eval:1590:13
23:06:42.583
The above error occurred in the <ScopedModel$Make$EmitterProvider> component:
    in ScopedModel$Make$EmitterProvider
    in ScopedModel$Make$Provider (created by Example$App)
    in Example$App

Consider adding an error boundary to your tree to customize error handling behavior.
Visit https://fb.me/react-error-boundaries to learn more about error boundaries. /node_modules/react-dom/cjs/react-dom.development:21810:13
23:06:42.584
Error: Invalid hook call. Hooks can only be called inside of the body of a function component. This could happen for one of the following reasons:
1. You might have mismatching versions of React and the renderer (such as React DOM)
2. You might be breaking the Rules of Hooks
3. You might have more than one copy of React in the same app

hmm, strange. In reality tho, that error happened to me as well, but when I run yarn start and yarn server in parallel, the error went away.

Updated the example's package.json to use the react and react-dom modules re-scoped-model is currently using.


Get rid of WordForm interface

Fixes #164

@rultor merge

@rultor merge

@suseika OK, I'll try to merge now. You can check the progress of the merge here

@rultor merge

@suseika Done! FYI, the full log is here (took me 8min)


fix: update tensor in stack mode

Context
wip, fix : https://github.com/docarray/docarray/issues/1081 I will add more info once I am done with the implementation
What this pr do ?
each time we modify a field in a Document that is inside document array stacked, the change will apply back to the document array stacked

    class Image(BaseDocument):
        tensor: TorchTensor[3, 224, 224]

    class MMdoc(BaseDocument):
        image: Image

    batch = DocumentArray[MMdoc](
        [MMdoc(image=Image(tensor=torch.zeros(3, 224, 224))) for _ in range(10)]
    )

    batch = batch.stack()

    batch[0].image = Image(tensor=torch.ones(3, 224, 224))

    assert (batch.image[0].tensor == torch.ones(3, 224, 224)).all()

same for tensor

    class Image(BaseDocument):
        tensor: TorchTensor[3, 224, 224]

    batch = DocumentArray[Image](
        [Image(tensor=torch.zeros(3, 224, 224)) for _ in range(10)]
    )

    batch = batch.stack()

    batch[0].tensor = torch.ones(3, 224, 224)

    assert (batch.tensor[0] == torch.ones(3, 224, 224)).all()

For archival purposes I'm adding the Discord discussion around this issue: https://discord.com/channels/1062677585999429653/1078671075761725560/1078685668617769060

What happens if we put the Document into another DocumentArrayStacked? Then to whom does it belong?

I would like to understand better how the view is implemented in Pytorch. Keeping a circular dependency does not seem to me as appropiate

In Pytorch they have a circular dependency as well. The view and the original object share the same storage object.
Btw I wanted to use weakref originally to avoid memory problem. Turn out it messed up the multiprocessing part (not sure why). But I found out that python is able to deal with cyclic reference so it is not a problem

I would like to understand better how the view is implemented in Pytorch. Keeping a circular dependency does not seem to me as appropiate

In Pytorch they have a circular dependency as well. The view and the original object share the same storage object.
Btw I wanted to use weakref originally to avoid memory problem. Turn out it messed up the multiprocessing part (not sure why). But I found out that python is able to deal with cyclic reference so it is not a problem

But maybe @JoanFM you are right I should be more careful. Should I add a test to see if the GC handles it properly?

@JoanFM what would be an alternative according to you (I am open to any alternative tbh). The thing is that conceptually if a document update triggers a da update then they need to somehow be linked right ?

What happens if we put the Document into another DocumentArrayStacked? Then to whom does it belong?

This is not possible. The new DocumentArray will check that there are no _da_ref in the document before allowing him.
Basically, oncee a Document is appended to a DocumentArrayStack it is the da that has full ownership of the doc

new PR : https://github.com/docarray/docarray/pull/1183


chore(deps): update terraform terraform-ibm-modules/secrets-manager/ibm to v1.0.2

This PR contains the following updates:

Package
Type
Update
Change

terraform-ibm-modules/secrets-manager/ibm (source)
module
patch
1.0.1 -> 1.0.2

Release Notes

terraform-ibm-modules/terraform-ibm-secrets-manager (terraform-ibm-modules/secrets-manager/ibm)
v1.0.2
Compare Source
Bug Fixes

deps: update terraform terraform-ibm-modules/cbr/ibm to v1.17.1 (#11) (b2955b5)

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

/run pipeline

:tada: This PR is included in version 1.1.2 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Detaching from RDP session from elevated command prompt

Hi all
When using regular RDP - I use the little batch file below that allows me to detach from an RDP session.  It is helpful because when I get physically back to the machine, I can go straight to the desktop rather than logging again.
Finally got RDPWrap working on Win Home machine and it is great but found that the batch file doesn't work anymore --- any ideas on how I can tweak to achieve the same in an RDPWrap session

for /f "tokens=3-4" %%a in ('query session %username%') do @if "%%b"=="Active" set RDP_SESSION=%%a
tscon %RDP_SESSION% /dest:console

Closing because found out how to enable shadowing to achieve similar outcome


feat: Add FirebaseAppCheck and AppIntegrity

Ticket Link:
DCMAW-10312
Description Of Changes:
Update authentication and wallet modules to enable consuming AppIntegrity API (in authentication) and implementing the Firebase AppIntegrityCheck to get a firebase token.
Evidence:
AC1 & AC2
https://github.com/user-attachments/assets/0c7af675-f176-4171-b1c2-6a82c5af83bd
Definition Of Done Checklist:

[ ] Unit tests written to at least 80% code coverage (unless explicit approval given from TL that this is not necessary)
[ ] Relevant integration and end-to-end tests are written
[ ] Tests cover each acceptance criteria on the user story
[ ] Demo completed by running code locally, demoing to a code reviewer & designer
[ ] Sonar coverage gates met
[ ] If feature flagged, Feature is enabled in staging and manually tested to ensure integration

Resolves: DCMAW-10312, DCMAW-10311

DCMAW-10311
Evidence
Screenshot of Attestation call with X and Y values from the private key generated in 10311

2024-11-08 16:48:43.220  2318-2802  System.out              uk.gov.onelogin.build                I  {"keys":[{"kty":"EC","x":"nfKPgSUMcrJ96ejGHr-tAvfzZOgLuFK-W_pz3Jjcs-Y","y":"Z7xBQNM9ipvaDp1Lp3DNAn7RWQ_JaUBXstcXnefLR5k","crv":"P-256","use":"sig","alg":"ES256","kid":"16db6587-5445-45d6-a7d9-98781ebdf93d"},{"kty":"RSA","n":"rhd_K6nAaBtW03UFkHuFlCZVbBm0-3qr5YF0UwPw7NECjGigoRNRE5UJ7oOLPOJr04Ju5vPqbyuTWLTqI_cjcnL68H2EUgEsunFHsexThk_lK5B5OpjYPbcqNVhPszwFHrqdzsFAjbFBN3EPDv90Lf8ZjnKXKzpQ0aMdD66n5jx8Av2dZ6H-63R5mGlOrpzyu5x31AcJ_N0YFpTyqy3a2nC--taagBpfq8rL4ypg7egfq0irRglfsXhbkt1p0IwI516jWSlvdhe4Z1yJbZU_dGt4bZemf7BWs7KfmPHQRkmRg09-VXZo80K5BPWRIXrYCurA-2wNafnRoO6KmDYuXw","e":"AQAB","use":"enc","alg":"RSA-OAEP-256","kid":"849bb6a3-eb58-471a-b279-75be3c60601b"}]}

DCMAW-10311
Evidence
AC1: Skip key pair check If appCheckEnabled flag is disabled

AC2: Keypair does not exist
Screenshot of Attestation call with X and Y values from the private key generated in 10311

2024-11-08 16:48:43.220 2318-2802 System.out uk.gov.onelogin.build I {"keys":[{"kty":"EC","x":"nfKPgSUMcrJ96ejGHr-tAvfzZOgLuFK-W_pz3Jjcs-Y","y":"Z7xBQNM9ipvaDp1Lp3DNAn7RWQ_JaUBXstcXnefLR5k","crv":"P-256","use":"sig","alg":"ES256","kid":"16db6587-5445-45d6-a7d9-98781ebdf93d"},{"kty":"RSA","n":"rhd_K6nAaBtW03UFkHuFlCZVbBm0-3qr5YF0UwPw7NECjGigoRNRE5UJ7oOLPOJr04Ju5vPqbyuTWLTqI_cjcnL68H2EUgEsunFHsexThk_lK5B5OpjYPbcqNVhPszwFHrqdzsFAjbFBN3EPDv90Lf8ZjnKXKzpQ0aMdD66n5jx8Av2dZ6H-63R5mGlOrpzyu5x31AcJ_N0YFpTyqy3a2nC--taagBpfq8rL4ypg7egfq0irRglfsXhbkt1p0IwI516jWSlvdhe4Z1yJbZU_dGt4bZemf7BWs7KfmPHQRkmRg09-VXZo80K5BPWRIXrYCurA-2wNafnRoO6KmDYuXw","e":"AQAB","use":"enc","alg":"RSA-OAEP-256","kid":"849bb6a3-eb58-471a-b279-75be3c60601b"}]}
AC3: Success (keypair already exists)


InsertModelTest::testInsertWithSetAndEscape depends on time

There was 1 failure:

1) CodeIgniter\Models\InsertModelTest::testInsertWithSetAndEscape
Failed asserting that 2 is equal to 1 or is less than 1.

/home/runner/work/CodeIgniter4/CodeIgniter4/system/Test/CIUnitTestCase.php:436
/home/runner/work/CodeIgniter4/CodeIgniter4/tests/system/Models/InsertModelTest.php:296

https://github.com/codeigniter4/CodeIgniter4/actions/runs/3556450130/jobs/5973847821

Here is the code:
    /**
     * @see https://github.com/codeigniter4/CodeIgniter4/issues/4087
     */
    public function testInsertWithSetAndEscape(): void
    {
        $userData = [
            'name' => 'Scott',
        ];

        $this->createModel(UserModel::class);

        $this->setPrivateProperty($this->model, 'useTimestamps', true);

        $this->model->set('country', '1+1', false)->set('email', '2+2')->insert($userData);

        $this->assertGreaterThan(0, $this->model->getInsertID());
        $result = $this->model->where('name', 'Scott')->where('country', '2')->where('email', '2+2')->first();

        $this->assertCloseEnough(time(), strtotime($result->created_at));
    }

Referenced: https://github.com/codeigniter4/CodeIgniter4/issues/4087

Seems like this:
$this->assertCloseEnough(time(), strtotime($result->created_at));

Should be:
$this->assertNotNull($result->created_at)


Client-sided crash with Create and Quark

Hi! I'm using this on my server and every once in a while I get a client crash. Trawling through the log, it says something about Quark, Create, and Flywheel. I'll attach the crash log below. Thank you in advance for any help!
crash-2021-08-06_09.11.04-client.txt

This is fixed in the latest version of Flywheel


Allow bypassing babel

It is possible to write TS code such that Babel is not required to post-process TS compilation results. It would be a nice perf win for people in those situations if they could somehow prevent babel from being used.
For this feature, it doesn't make sense to inspect configuration options in tsconfig because they are not concrete enough to reliably express user intentions regarding babel usage.
I'm thinking that the best thing to do in the long run would be to expose an api to allow users to wrap and configure the transformer. Then they could point the bundler to their own transformer implementation.

@ds300 What about Typescript transformer support in Babel 7?

Yes! Babel 7 can transpile TypeScript, which is amazing and makes this library obsolete. Couldn't be happier to have a thing I've built become useless :D

Hey @ds300  do you recommend using this or babel 7 with typescript-plugin if I want to start with a new expo project right now? Also, do you know where I can find good documentation for babel 7 typescript & expo?

@karsens I think you will save some time if you use this library still, however I know of one team who are using babel 7 successfully, with the caveat that babel can't have both the typescript and flow plugins at the same time, so you end up needing to strip all the flow types from node_modules in a separate  step.
See here: https://github.com/artsy/emission/pull/780

Seems like the newest React Native RC does use Babel 7. https://github.com/facebook/react-native/releases/tag/v0.56.0-rc.2
Did anyone already try using TypeScript directly with that version?

Babel has some limitations - in regards to e.g. things like const enums, as it only transpiles one file at a time. For many things, babel should work fine, but not for all.
(Though if this transformer uses transpileModule internally, I think it has the same limitations)


Equality not decidable on the real numbers

Hello,
Admitted theorem Req_neq in file Order.v is probably false in constructive mathematics, because equality is not decidable on the real numbers. Do you rely on some non-constructive axioms, or do you have a proof of it?
Also I think that this implementation of Dedekind reals would also be Cauchy-complete, if the located hypothesis was a sumbool instead of a disjunction in sort Prop. Is there a reason for using Prop here?

The files are in whatever state they were left by my students when they attempted to formalize the reals. They did not have a good understanding of the role of excluded middle and they naturally assumed it in several places. I would be very happy if anyone cleaned it up.
As for Cauchy-completeness: yes, the Dedekind reals are Cauchy complete, and the proof of this has nothing to do with what form the locatedness axiom takes. Given a Cauchy sequence, we can simply define its limit in terms of a cut. Recent work of @abooij shows that putting locateness into Type (with a sum) gets us precisely the Cauchy reals, instead of the Dedekind reals, at least in the context of homotopy type theory.

If I have time I'll send you a pull request with constructive versions of your theorems.
I actually meant isomorphic to the Cauchy reals. You do not need HOTT, or univalence or any axiom outside plain Coq. You almost have the isomorphism in your lemma archimedean. It asserts in sort Prop that there exist rational numbers arbitrarily close to any real number. If you put the sumbool in located, lemma archimedean lifts to sort Set and any real number gets a converging sequence of rational numbers.
Then why not put the sumbool ? It seems a low price to pay to get many usual properties of the real numbers. Or if you want something bigger than Cauchy reals, then why use locatedness instead of 1-sided (lower) reals ?

@VincentSe: If one uses sumbool in the definition of locatedness, then your isomorphism still doesn't work out, as the maps constituting the isomorphism wouldn't preserve the intended equality on Dedekind reals, as specified by equivalence relation Req. (Bishop would call them "operations" rather than "functions".)
The Cauchy reals and the Dedekind reals are different objects, and this is a well-known phenomenon that I don't think we fully understand.

@abooij Yes, I have the full proof in standard Coq that the isomorphism works and respects the setoid equalities, on both the Dedekind and Cauchy sides. I was about to send it to you :)
So we all agree that the difference between Dedekind and Cauchy reals is the difference between or and sumbool. Then again, why not put the sumbool ? I mean in terms of the properties we can prove about the real numbers.
To me it looks similar to moving from classical to constructive logic : you can prove less things, but you are more confident in the things you do prove, and you have more tools that operate on the proofs, such as extraction. But here I don't see what we gain from weakening a sumbool to an or. More confidence that a Dedekind real is a number ? I hardly think so : you cannot extract its digits.

@VincentSe I believe that with sumbool this gives the Cauchy reals, yes, although I'm not entirely sure about some details (e.g. universe level), but it seems to work out.
The Dedekind reals are preferable over Cauchy reals for some purposes, such as their less intensional definition. (The Cauchy reals, in their usual presentation, are given by Cauchy sequences of rationals, which one may have philosophical objections to.) In non-setoid approaches (e.g. univalent foundations), there are properties that hold for the Dedekind reals but not for the Cauchy reals. For example, ironically, the Cauchy reals are not Cauchy complete in general, whereas the Dedekind reals are. In some foundational systems, the Cauchy reals are generally preferred even if the Dedekind reals can be defined. But then one shouldn't call such Cauchy reals "Dedekind reals", as the name of this repository does. We want to study in detail how the Cauchy and Dedekind reals are related, and so the Coq definitions should be accurate.
You mention extracting digits. In fact, extracting digits from your sumbool reals would violate the equivalence relation. It cannot be done in a way that respects the intended equality of the reals.
Yes, certain modifications to basic definitions allow proving more things. But that does not imply that they are better, because the definitions (i.e. assumptions) are also stronger. A whole lot of interesting math can be developed for Dedekind reals, so why focus on Cauchy reals?

@abooij I guess it means that the difference between sumbool and or is deeper than it looks. And this is interesting already. I do not see why extracting digits would violate an equivalence relation : the digits of a real number are unique (although there are different Cauchy sequences of rational numbers that converge to the same real number).
I won't argue this issue further. I have sent a pull request that corrects a constructive inconsistency in this repo, and proves 10 admitted lemmas. I will close this issue when it is merged.

@VincentSe Consider the digit expansion of the number 3/3. What is the integer part of this number? Is it 0, to obtain the digit expansion 0.999...? Or is it 1, to obtain the digit expansion 1.000...? Well, if you want to make the digit expansions unique, you probably want to exclude a tail of 9's. So you'd have to decide whether the digits are all 9's from some point onwards, or not, which sounds like an instance of LLPO.
This game shows that to compute digit expansions requires deciding an inequality (in the above example, to decide x<1), which cannot be done for arbitrary reals.
So if you want to output one sequence which is the digit expansion of a given real, you have to decide inequalities. Not as bad as deciding equalities, but still not good enough.
One solution is to work instead with signed digit expansions, as is often done in computable and constructive analysis (and in fact Turing's famed paper which introduced computability made this same mistake, and a later correction used exactly this solution). In signed digit expansions, the digits of the (decimal, say) digit expansion range from -9 to +9. So the number 1 can be represented as 1.000... or 0.999... or 2.(-9)(-9)(-9)... or 1.1(-9)(-9)(-9)..., etc.

@abooij ok fair point

@abooij Can you develop how you decide equalities if you know how to decide inequalities ?

@VincentSe: Suppose we can decide x<y and y<x. If one of them is true, then certainly x and y are apart. The remaining case is that both of them are false, that is, ¬(x<y)∧¬(y<x). But this is equivalent to (x≤y)∧(y≤x), i.e., x=y.
This argument shows that if x<y is decidable for all x,y, then apartness is decidable, which iin particular implies that equality is decidable (since the negation of apartness is equality, and apartness implies the negation of equality).

@abooij thanks

@abooij I found a question more interesting about Dedekind reals and Cauchy reals : can this implementation of Dedekind reals construct a real number and prove that it is not a Cauchy real ?
I suspect it cannot, because the effective topos is a model of constructive logic where all real numbers are computable, ie Cauchy reals.
To define a non computable Dedekind real, we would need an axiom on top of standard Coq, for example the excluded middle. I read you work on HoTT, so maybe univalence allows to construct such a number too. Do you know what axioms work ?

@VincentSe: Univalence does not suffice, because it is compatible with excluded middle, which implies that the Dedekind reals and the Cauchy reals coincide. You would need what some refer to as a constructive fetish, namely an axiom that contradicts classical axioms.

This issue was more or less closed by #9. I also cleaned up the repository a bit and included in Cut.v a reference to @abooij's paper.


Timeframe for CVE-2023-22102 resolution?

Discussed in https://github.com/jruby/activerecord-jdbc-adapter/discussions/1161

Originally posted by atziatzios-ccycloud October 29, 2024
Hi. I just wanted to highlight an already raised issue (https://nvd.nist.gov/vuln/detail/CVE-2023-22102) with the mysql adapter version in use in the latest release.
How complex/problematic is an update to a non vulnerable version of the adapter?
Thanks to anyone taking the time to answer.

I have released jdbc-mysql-<IP_ADDRESS> (smallest update to avoid CVE, based on 8.2.0 but I botched the first attempt), jdbc-mysql-8.4.0 (last in 8.x series), and jdbc-mysql-9.1.0 (latest overall).


SortList is read-only in CSOM library

Currently , the SortList attribute of the KeywordQuery class is read-only in the CSOM library (16.1.20912.12000). Will this property will be updated to also be SET-able like the REST implementation as mentioned in this article?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 59d82d79-2ecd-e278-a7e4-44ec8a7a5a90
Version Independent ID: c2588f4f-c665-f863-79ca-26cd33ba9cf4
Content: Pagination for large result sets
Content Source: docs/general-development/pagination-for-large-result-sets.md
Product: sharepoint
Technology: sharepoint-framework
GitHub Login: @VesaJuvonen
Microsoft Alias: vesaj

Never mind, you can use keywordQuery.SortList.Add("DocId", SortDirection.Ascending); to enable the sorting.