[Bug]: NPE in SolaceIO getWatermark

What happened?
With our Solace broker instance, it sometimes happens during maintenance windows that it resets the connections and our Beam jobs have to reconnect. Normally this is not a big issue but I have noticed this in one of our SolaceIO-based jobs that we are currently testing:
java.lang.NullPointerException
	at org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.base.Preconditions.checkNotNull(Preconditions.java:903)
	at org.apache.beam.sdk.io.solace.read.UnboundedSolaceReader.getWatermark(UnboundedSolaceReader.java:128)
	at org.apache.beam.sdk.io.Read$UnboundedSourceAsSDFWrapperFn$UnboundedSourceAsSDFRestrictionTracker.currentRestriction(Read.java:953)
	at org.apache.beam.sdk.io.Read$UnboundedSourceAsSDFWrapperFn$UnboundedSourceAsSDFRestrictionTracker.getProgress(Read.java:1017)
	at org.apache.beam.sdk.fn.splittabledofn.RestrictionTrackers$RestrictionTrackerObserverWithProgress.getProgress(RestrictionTrackers.java:105)
	at org.apache.beam.fn.harness.FnApiDoFnRunner.getProgress(FnApiDoFnRunner.java:1186)
	at org.apache.beam.fn.harness.FnApiDoFnRunner.access$1700(FnApiDoFnRunner.java:145)
	at org.apache.beam.fn.harness.FnApiDoFnRunner$SplittableFnDataReceiver.getProgress(FnApiDoFnRunner.java:1171)
	at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$SplittingMetricTrackingFnDataReceiver.getProgress(PCollectionConsumerRegistry.java:482)
	at org.apache.beam.fn.harness.BeamFnDataReadRunner.trySplit(BeamFnDataReadRunner.java:266)
	at org.apache.beam.fn.harness.control.ProcessBundleHandler.trySplit(ProcessBundleHandler.java:741)
	at org.apache.beam.fn.harness.control.BeamFnControlClient.delegateOnInstructionRequestType(BeamFnControlClient.java:150)
	at org.apache.beam.fn.harness.control.BeamFnControlClient$InboundObserver.lambda$onNext$0(BeamFnControlClient.java:115)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at org.apache.beam.sdk.util.UnboundedScheduledExecutorService$ScheduledFutureTask.run(UnboundedScheduledExecutorService.java:163)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)

This exception happens somewhere around reconnection failures (network reasons - connection refused, broker was probably down briefly).
Afterwards, the pipeline seems to recover (subsequent reconnection to the broker is successful) but still I guess it would be nice to handle such event more gracefully than NPE.
FYI @bzablocki
Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
Issue Components

[ ] Component: Python SDK
[X] Component: Java SDK
[ ] Component: Go SDK
[ ] Component: Typescript SDK
[X] Component: IO connector
[ ] Component: Beam YAML
[ ] Component: Beam examples
[ ] Component: Beam playground
[ ] Component: Beam katas
[ ] Component: Website
[ ] Component: Infrastructure
[ ] Component: Spark Runner
[ ] Component: Flink Runner
[ ] Component: Samza Runner
[ ] Component: Twister2 Runner
[ ] Component: Hazelcast Jet Runner
[X] Component: Google Cloud Dataflow Runner

.take-issue

Thanks for reporting, I'll handle it next week 👌


Subscriber not called after action dispatch

Component
logout() {
	this.ngxs.dispatch(new Logout()).subscribe(() => {
		this.router.navigate(['']); // <-- This code never runs
	});
}

Store
@Action(Logout)
logout() {
	this.loginService.logout();
}

@perjansson thanks for reporting will look into it

I am also doing similar code, I thing you may have to return in the action method.
Reference here

I'm seeing this in my 2.0 code too @deebloo ... Its def a bug.

I will investigate the issue and fix in 2.0

Confirmed fixed in 2.0 branch. Closing this, 2.0 will be released this weekend.


Icon tree shake for 3rd party library not working in windows build

Steps to reproduce
Create and build template project

flutter create flutter_project
cd flutter_project
flutter pub add material_symbols_icons
Apply patch below
flutter build windows --tree-shake-icons
View build assets build\windows\x64\runner\Release\data\flutter_assets\packages\material_symbols_icons\lib\fonts

--- a/main.orig.dart
+++ b/main.dart
@@ -1,4 +1,5 @@
 import 'package:flutter/material.dart';
+import 'package:material_symbols_icons/material_symbols_icons.dart';

 void main() {
   runApp(const MyApp());
@@ -112,6 +113,7 @@ class _MyHomePageState extends State<MyHomePage> {
               '$_counter',
               style: Theme.of(context).textTheme.headlineMedium,
             ),
+            const Icon(Symbols.abc),
           ],
         ),
       ),

Expected results
As a comparison, fonts size in android build.
Build command flutter build apk --tree-shake-icons
Extract from build\app\outputs\apk\release\app-release.apk!/assets/flutter_assets/packages/material_symbols_icons/lib/fonts
-rw-rw-r--    1 kr328    kr328       3.8K Jan 01  1981 MaterialSymbolsOutlined.ttf
-rw-rw-r--    1 kr328    kr328       2.6K Jan 01  1981 MaterialSymbolsRounded.ttf
-rw-rw-r--    1 kr328    kr328       2.3K Jan 01  1981 MaterialSymbolsSharp.ttf

Actual results
Fonts size in windows build.
Files in build\windows\x64\runner\Release\data\flutter_assets\packages\material_symbols_icons\lib\fonts.
-rw-rw-r--    1 kr328    kr328       8.6M Oct 19 01:57 MaterialSymbolsOutlined.ttf
-rw-rw-r--    1 kr328    kr328      12.3M Oct 19 01:57 MaterialSymbolsRounded.ttf
-rw-rw-r--    1 kr328    kr328       7.1M Oct 19 01:57 MaterialSymbolsSharp.ttf

Code sample
Code sample
import 'package:flutter/material.dart';
import 'package:material_symbols_icons/material_symbols_icons.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        // This is the theme of your application.
        //
        // TRY THIS: Try running your application with "flutter run". You'll see
        // the application has a purple toolbar. Then, without quitting the app,
        // try changing the seedColor in the colorScheme below to Colors.green
        // and then invoke "hot reload" (save your changes or press the "hot
        // reload" button in a Flutter-supported IDE, or press "r" if you used
        // the command line to start the app).
        //
        // Notice that the counter didn't reset back to zero; the application
        // state is not lost during the reload. To reset the state, use hot
        // restart instead.
        //
        // This works for code too, not just values: Most code changes can be
        // tested with just a hot reload.
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
        useMaterial3: true,
      ),
      home: const MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({super.key, required this.title});

  // This widget is the home page of your application. It is stateful, meaning
  // that it has a State object (defined below) that contains fields that affect
  // how it looks.

  // This class is the configuration for the state. It holds the values (in this
  // case the title) provided by the parent (in this case the App widget) and
  // used by the build method of the State. Fields in a Widget subclass are
  // always marked "final".

  final String title;

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  int _counter = 0;

  void _incrementCounter() {
    setState(() {
      // This call to setState tells the Flutter framework that something has
      // changed in this State, which causes it to rerun the build method below
      // so that the display can reflect the updated values. If we changed
      // _counter without calling setState(), then the build method would not be
      // called again, and so nothing would appear to happen.
      _counter++;
    });
  }

  @override
  Widget build(BuildContext context) {
    // This method is rerun every time setState is called, for instance as done
    // by the _incrementCounter method above.
    //
    // The Flutter framework has been optimized to make rerunning build methods
    // fast, so that you can just rebuild anything that needs updating rather
    // than having to individually change instances of widgets.
    return Scaffold(
      appBar: AppBar(
        // TRY THIS: Try changing the color here to a specific color (to
        // Colors.amber, perhaps?) and trigger a hot reload to see the AppBar
        // change color while the other colors stay the same.
        backgroundColor: Theme.of(context).colorScheme.inversePrimary,
        // Here we take the value from the MyHomePage object that was created by
        // the App.build method, and use it to set our appbar title.
        title: Text(widget.title),
      ),
      body: Center(
        // Center is a layout widget. It takes a single child and positions it
        // in the middle of the parent.
        child: Column(
          // Column is also a layout widget. It takes a list of children and
          // arranges them vertically. By default, it sizes itself to fit its
          // children horizontally, and tries to be as tall as its parent.
          //
          // Column has various properties to control how it sizes itself and
          // how it positions its children. Here we use mainAxisAlignment to
          // center the children vertically; the main axis here is the vertical
          // axis because Columns are vertical (the cross axis would be
          // horizontal).
          //
          // TRY THIS: Invoke "debug painting" (choose the "Toggle Debug Paint"
          // action in the IDE, or press "p" in the console), to see the
          // wireframe for each widget.
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            const Text(
              'You have pushed the button this many times:',
            ),
            Text(
              '$_counter',
              style: Theme.of(context).textTheme.headlineMedium,
            ),
            const Icon(Symbols.abc),
          ],
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: _incrementCounter,
        tooltip: 'Increment',
        child: const Icon(Icons.add),
      ), // This trailing comma makes auto-formatting nicer for build methods.
    );
  }
}

Screenshots or Video
Just build issue, not related to UI.
Logs
Log Files
build.apk.log
build.windows.log

Flutter Doctor output
Doctor output
PS D:\Workspace\flutter_project> flutter doctor
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.24.3, on Microsoft Windows [Version 10.0.22631.4317], locale zh-CN)
[✓] Windows Version (Installed version of Windows is version 10 or higher)
[✓] Android toolchain - develop for Android devices (Android SDK version 35.0.0)
[✓] Chrome - develop for the web
[✓] Visual Studio - develop Windows apps (Visual Studio Community 2022 17.10.5)
[✓] Android Studio (version 2024.2)
[✓] IntelliJ IDEA Ultimate Edition (version 2024.2)
[✓] VS Code (version 1.94.2)
[✓] Connected device (4 available)
[✓] Network resources

• No issues found!

I can reproduce this issue on Flutter master channel as well.

flutter doctor versions (master)
[!] Flutter (Channel master, 3.27.0-1.0.pre.120, on Microsoft Windows [Version 10.0.19045.5011], locale en-US)
    • Flutter version 3.27.0-1.0.pre.120 on channel master at C:\WIP\flutter_master
    ! Warning: `flutter` on your path resolves to C:\WIP\flutter\bin\flutter, which is not inside your current Flutter
      SDK checkout at C:\WIP\flutter_master. Consider adding C:\WIP\flutter_master\bin to the front of your path.
    ! Warning: `dart` on your path resolves to C:\WIP\flutter\bin\dart, which is not inside your current Flutter SDK
      checkout at C:\WIP\flutter_master. Consider adding C:\WIP\flutter_master\bin to the front of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 47035a0102 (19 hours ago), 2024-10-20 05:27:31 -0400
    • Engine revision 53e187a1d4
    • Dart version 3.7.0 (build 3.7.0-41.0.dev)
    • DevTools version 2.40.1
    • If those were intentional, you can disregard the above warnings; however it is recommended to use "git" directly
      to perform update checks and upgrades.

@bkonyi @andrewkolos Hey there, from a cursory glance this looks like a tool issue. Feel free to reassign back to the Windows team if I'm wrong!

Not only Windows has this issue, all desktop platforms, including Linux, mac OS also have the issue.


Create factor.js

Factor language simple mode.

Wonderful! Could you add an index.html demo file for the mode (copy-paste one from another mode and adjust)

I've created another pull request -- with factor.js and index.html. This one turned out to be glitchy.


GH-2451: Fix Class Level Listener Multi Instances

Resolves https://github.com/spring-projects/spring-kafka/issues/2451
Classes with class level @KafkaListener were incorrectly added to the nonAnnotatedClasses set, preventing multiple instances of the same class to be registered as listeners.
cherry-pick to 2.9.x, 2.8.x

... and cherry-picked to 2.9.x & 2.8.x


When do you merge develop with master?

Hey, i need the fix with token.expires in auth.py.
When will you merge it into master?

just did it :)

@belugame thank you!

@belugame i forgot, could you push it to pypi?

@bobo-le we are currently discussing the release in #168 if you want to follow along.


Doesn't handle encoded query params

This got me very confused for a while.
I was doing
nock 'https://ci.jenkins.com'
  .get '/api/json?tree=jobs[name]'

The actual request goes to https://ci.jenkins.com/api/json?tree=jobs%5Bname%5D, so I finally figured out you have to encode query parameters for nock:
nock 'https://ci.jenkins.com'
  .get '/api/json?tree=jobs%5Bname%5D'

Is there a possibility nock could be smart about this?

@raine I added this test and I couldn't make it fail:
https://github.com/pgte/nock/blob/50912da57c881fd5fbb1a824f9514ea7cc3b7496/tests/test_url_encoding.js
Is it possible that some library you're using transforms the URL before it reaches the native Node HTTP client?
If not, can you make some change to the test that makes it fail?

Related: #82

I'm using qs to create a query string which is appended to a url with a path, and the whole thing is passed to request.
So the URL actually already looks like ... ?tree=jobs%5Bname%5D before it reaches request. It appears qs does encoding.

request already supports a qs argument our of the box, no need to use qs directly.
Anyway, closing this as it's not a Nock problem.

Isn't it an unnecessary inconvenience if user of Nock needs to manually encode the query string?

The point is that you wouldn't have to encode the URL if a lib didn't encode it (unnecessarily since those are valid characters).
What do you think nock could do to help?

Instead of
var qs = require('querystring').stringify;
nock('https://ci.jenkins.com')
  .get('/api/json?' + qs({ tree: 'jobs[name]' })

I'd rather do
nock('https://ci.jenkins.com')
  .get('/api/json')
  .query({ tree: 'jobs[name]' })

But I can see why it would be tricky to implement because Nock has no idea about how user's code encodes the querystring, right? Best it could do is guess.

The issue still exists now - when combining .post() with .query(...encoded_params)

@pgte I think the proper way to handle this would be make it so that nock optionally consume a query param serializer.
e.g. something like:
const querystring = require('querystring');
const serializeQuery(params) {
  return querystring.stringify(params);
}

nock('https://example.com')
.setQuerySerializer(serializeQuery)
.query({ tree: 'jobs[name' })

Would be nice to have the option to set it on the nock object and also just set it for specific one off requests.
If you'd accept a PR like this let me know and I can put one up.

@peteratticusberg Hey Peter, I’m currently maintaining nock but still finding my ways around, happy to help.
I think encoding of query parameters is out of scope of nock, as different request libraries handle encoding differently ... I’m reluctant to add another feature that we’d need to document and support for forever. If you feel strong about it, can you create a separate issue and we see if more folks run into problems that the additional API would help with? Maybe the right solution for this is a better documentation?

Thanks!

I think encoding of query parameters is out of scope of nock, as different request libraries handle encoding differently

Agreed. The idea behind the feature would be to allow users to optionally configure nock to use an arbitrary query serializer (e.g. the one that their request library is using). Nock itself would not ship with a query serializer.
Does that make any more sense?

I know what you mean, thanks. I’m adding the .setQuerySerializer() at this point though. It’d be easy to add, that’s not the problem, it’s all about maintainability & support in the long run, which is the biggest challenge nock has right now, but usually not what users of the project ask for, so i have to stand up for it :)

I see totally makes sense. Let's hold off on this then. If we see more activity in this thread / demand for this feature elsewhere I'd be happy to come back and put up a PR.


Update sbt-idea-plugin to 3.20.0

About this PR
📦 Updates org.jetbrains:sbt-idea-plugin from 3.18.3 to 3.20.0
📜 GitHub Release Notes - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.jetbrains", artifactId = "sbt-idea-plugin" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "org.jetbrains", artifactId = "sbt-idea-plugin" }
}]

labels: sbt-plugin-update, early-semver-minor, semver-spec-minor, commit-count:1

Superseded by #633.


ci: switch Docker image to nvidia/cuda:12.6.0-devel-rockylinux9

cupy currently requires development headers that aren't in the "runtime" CUDA Docker images (https://github.com/cupy/cupy/issues/8466). It seems like eventually the plan is to support CUDA from pip (https://github.com/cupy/cupy/issues/8013), and indeed the headers are already available from the nvidia-cuda-runtime-cu12 on PyPI, but let's not mix and match CUDA sources unless we need to.
(Also, let's use Rocky instead of Ubuntu, since most users will probably be on a GPU cluster running a RHEL variant.)

Also, the Jenkins tests weren't running properly on PRs before, but they should be now.


SpanKind support

SpanKind is now part of OpenCensus trace proto. We should support it as a first class citizen in Go. See: https://github.com/census-instrumentation/opencensus-proto/pull/51

Is this API breaking anymore? We are going to make it an option.

Guess not

fwiw in zipkin-go @basvanbeek made lack of kind a special kind (at the api level which is not strictly bound to the format) https://github.com/openzipkin/zipkin-go/blob/master/model/kind.go

Closed via https://github.com/census-instrumentation/opencensus-go/pull/610 and https://github.com/census-instrumentation/opencensus-go/pull/616.


Change the return type of RegionEngine::handle_request()

What type of enhancement is this?
Tech debt reduction
What does the enhancement do?
This interface inherits common_query::Output from the previous table engine.
pub enum Output {
    AffectedRows(usize),
    RecordBatches(RecordBatches),
    Stream(SendableRecordBatchStream),
}

It combines the result of query (RecordBatches and Stream) and non-query (AffectedRows) together. After refactoring to region engine and region server, those requests are handled separately in two APIs. RegionEngine::handle_request() only handles non-query requests, thus the return type can be simplified to only contains the AffectedRows branch.
Implementation challenges
No response

Can I have a try?


Accept Arrayable as props for modals

In Inertia you can pass an Arrayable object as props, which is useful if you use something like spatie/laravel-data since you don't have to convert the data objects to array yourself. This PR updates this package to also accept Arrayable objects as props.

Merged, thanks!


Duplicate named registrations fails silently when using BasedOnDescriptor.Configure

In my application, I'm using a custom attribute to provide info to Windsor that it can use to generate named registrations for different implementations of the same underlying service type.
The issue is that if a scenario is encountered where the same name would be used to register multiple components, then it creates a race condition, where the first type that Windsor discovers and registers will get added to the container, while the other types that try to register with the same name, simply do not get added to the container. No exception is triggered, which is the behavior I expected.
Unfortunately, I can't share the exact code, but here is some similar code that should reproduce the same issue. Whichever type is discovered by Windsor first , either ServiceImplementation1 or ServiceImplementation2, will get registered, while the attempt to register the 2nd component with the same name will fail silently.
namespace MyApp
{
    [AttributeUsage(AttributeTargets.Class)]
    public class CustomAttribute : Attribute
    {
        public string Key { get; }

        public CustomAttribute(string key)
        {
            Key = key;
        }
    }

    // the service type registered with the container
    public interface IServiceInterface { }

    // implementation #1
    [Custom("implementation1")]
    public class ServiceImplementation1 : IServiceInterface { }

    // implementation #2: this uses the same key as #1, to trigger a registration failure
    [Custom("implementation1")]
    public class ServiceImplementation2 : IServiceInterface { }

    public class WindsorInstaller : IWindsorInstaller
    {
        public void Install(IWindsorContainer container, IConfigurationStore store)
        {
            container.Register(
                Classes.FromAssemblyContaining<WindsorInstaller>()
                    .Where(t => typeof(IServiceInterface).IsAssignableFrom(t) &&
                                Component.HasAttribute<CustomAttribute>(t))
                    .Configure(reg =>
                    {
                        var attribute =
                            (CustomAttribute)Attribute.GetCustomAttribute(c.Implementation, typeof(CustomAttribute));
                        reg.Named($"{typeof(IServiceInterface).FullName}_{attribute.Key}");
                    })
            );

        }
    }

    public class Program
    {
        public static int Main(string[] args)
        {
            var container = new WindsorContainer().Install(FromAssembly.Containing<Program>());

            // resolves to whichever component happened to be discovered first
            var service = container.Resolve<IServiceInterface>($"{typeof(IServiceInterface).FullName}_implementation1");
            Console.WriteLine($"Resolved {service.GetType().Name}");

            // returns a single component (whichever one was successfully registered)
            var allServices = container.ResolveAll<IServiceInterface>();
            Console.WriteLine($"Discovered {allServices.Length} services that implement IServiceInterface");

            return 0;
        }
    }
}

FWIW... I discovered a way to deal with this issue by using the IKernel.HasComponent method to explicitly check for a duplicate named component. This still feels like something that Windsor should do by default though.
From the sample code I included earlier, here's how I changed the registration to throw an exception
container.Register(
    Classes.FromAssemblyContaining<WindsorInstaller>()
        .Where(t => typeof(IServiceInterface).IsAssignableFrom(t) &&
                    Component.HasAttribute<CustomAttribute>(t))
        .Configure(reg =>
        {
            var attribute = (CustomAttribute)Attribute.GetCustomAttribute(c.Implementation, typeof(CustomAttribute));

            var componentName = $"{typeof(IServiceInterface).FullName}_{attribute.Key}";

            // throw an exception when trying to register another component with same name
            if (container.Kernel.HasComponent(componentName))
                throw new ComponentRegistrationException("some useful message about registration failure");

            reg.Named(componentName);
        })
);


API Update

Updated scripts and code for the new caching API.

@phajy @kyle8mcknight @dangoh123 see here.
The diffs are a little difficult to see, since I also ran the Julia style formatter over the repository to make things a bit more standard to read.
I've left review comments on the main changes -- please update your remotes when convenient.

The plot produced by the code I pushed looks like this:

Just to compare it with Andy's script:

where the y axis has been arbitrarily rescaled. Also I got rid of the check to see if we're within a certain radius, since the filter function now only returns a real number if the geodesic terminated through disc intersection, so shouldn't need to do that check anymore.

Looks good! Thanks for sorting out the cache and garbage collection.
Would it be helpful if I generated some test line profiles from XSPEC that we can use for testing purposes?

@phajy that would be brilliant!


Config flags size > 56 with Moto Z android 7.0

Information

Apktool Version (apktool -version) -2.2.3-6cc251-SNAPSHOT
Operating System (Mac, Linux, Windows) -Windows 10
APK From? (Playstore, ROM, Other) - Moto Z  android 7.0 framework-res.apk (CN Retail Stock firmware)

Stacktrace/Logcat
I: Loading resource table...
W: Config flags size > 56. Size = 64. Exceeding bytes: 0x1666221400000000.
W: Invalid config flags detected. Dropping resources: string-mcc204-mnc4-ERR0
W: Config flags size > 56. Size = 64. Exceeding bytes: 0x1666221400000000.
W: Invalid config flags detected. Dropping resources: array-mcc204-mnc4-ERR1
W: Config flags size > 56. Size = 64. Exceeding bytes: 0x1666221400000000.
W: Invalid config flags detected. Dropping resources: bool-mcc204-mnc4-ERR2
I: Decoding AndroidManifest.xml with resources...
I: Regular manifest package...
I: Decoding file-resources...

Include stacktrace here

Steps to Reproduce

apktool

Frameworks
If this APK is from an OEM ROM (Samsung, HTC, LG). Please attach framework files
(.apks that live in /system/framework or /system/priv-app)
APK
If this APK can be freely shared, please upload/attach a link to it.
lenovo-res.zip.txt
framework-res.zip.txt
Questions to ask before submission

Have you tried apktool d, apktool b without changing anything?yes
If you are trying to install a modified apk, did you resign it?no
Are you using the latest apktool version?yes

I'm tried recompile framework-res.apk, but it show this error, I did not change anything, then I replaced the original file, the phone will be in black, did not show anything, it can't enter the launcher. 😢

Sorry for the delay. Cleaning up tickets that have no response/activity. Dropped the ball on getting to this.
This issue is because there is a new qualifier known as spn with an insane value range. The value I see now is 895679974. It seems connected to mcc/mnc. I'll do some research, but this issue much like other OEMs who break AOSP qualifiers tend to cause problems with apktool.

I can't patch this safely. This is another qualifier and future AOSP updates may use the same space this "3rd party" qualifier is using and thus if I pack this qualifier into apktool, then AOSP update comes out everything will be decoded incorrectly.
I do not know the best way to patch this. One way is a patch/plugin/parameter system that allows you to specify different qualifiers. That works great for apktool, but adding that functionality into aapt is another story.

Years later. This is being added into backlog of ROADMAP. My guess is newer OEMs aren't doing this because its insane and clashes with CTS and AOSP merges.
https://github.com/iBotPeaches/Apktool/commit/d61e33048f8bf7aa28a69f40ee6fc3624534f68d


Add Build and Release actions

In case the release proposal gets approved, here's a PR for:

Add Build and Create Release action jobs:
Build

Setup Go
Install Dependencies
build
Run vet & lint
Test
Clean up processes

Create Release

Get version and tag numbers
Create release and push to GitHub
Create tag and push to GitHub

Modified Readme to include notes about changing the custom logger and adding custom workflows and Github actions.

For testing you can checkout addBuildAndReleaseActions branch and add it to on push branches in .github/workflows/go.yml file.
on:
  push:
    branches:
      - main
      - addBuildAndReleaseActions
  pull_request:
    branches:
      - main
      - addBuildAndReleaseActions

For testing the "Create Release" job, you can comment out line #44 in go.yml file:
# if: ${{ github.ref == 'refs/heads/main' && github.event_name == 'push' }} # Only run if on main branch on push event

.

.

.

.

LGTM


Today indicator

Sample (which sample are you talking about)
project-gantt-chart
Suggestion (the more details, the better)
Would it be possible to add a marker to indicate the current date if it falls within the time frame of the project?

Hi @GerryHGraham,
Yes, we could do that.
The code that I published here, is already doing that.
I have more ideas to be added, like:

Status indicator.
Startdate - enddate at the top of the of chart


Update of Docu required

The Documentation mentions a WORKSPACES folder  right under the root folder. In the current version you will find the folder under samples and under training-materials. The LIBRARY folder does not exist here. Instead a BOMS folder was added. I would appreciate an update of the documentation.
Thank you
Stefan Netzer
[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6e8b503d-0faa-67bd-3461-0290c87a05a7
Version Independent ID: 4f78a94e-5518-cdc1-97d7-2a3fc0202f14
Content: Plan your SAP deployment with the automation framework on Azure - Azure Virtual Machines
Content Source: articles/virtual-machines/workloads/sap/automation-plan-deployment.md
Service: virtual-machines-sap
GitHub Login: @KimForss
Microsoft Alias: kimforss

@StefanNetzer Thank you for pointing this out. We will engage with the content team to pass on the feedback

@StefanNetzer
Thanks for the feedback! I have assigned the issue to content author to check and update the document as appropriate.

@KimForss
Can you please check and add your comments on this doc update request as applicable.


TPC-DS: Fix CSV format

Quick fix to make it working with trailing delimiters

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

R: @mosche

👍 lgtm, though spotless is failing. pls apply formatting changes

Fixed. merging


Secret Key Rotation

Hi everyone,
I have started working with Flask in the last month and I noticed that there could be an improvement on secret key managment for sessions; specifically I am taking as reference OWASP Top 10 Cryptographic Failures on the point where it talks about the importance of proper key managment (that isn't flask responsibility) and rotation.
I saw that there is a closed issue on that topic, however that was from 10 years ago, a lot of time passed so I thought it would be a nice thing pointing that out; I understand that this implies session invalidating when the secret key changes, but I think that this could be a choice left to developers based on security requirements of the specific system, therefore adding more flexibility.
If I am wrong in any way or it isn't feasible I hope that someone can provide me a good explanation about why.

Same previously closed discussions still apply.


update RTO/RPO

updated backup-restore RTO/RPO text
updated replication-failover RTO/RPO text

@hectoralinares : Thanks for your contribution to the Azure documentation! The author, @mattbriggs, has been notified to review your proposed change.


Qai 443: added record id and raw_data in some objects that were missed

Some objects were not extending object.json because of which record_id and raw_data were not present in the attributes. We want all the objects to extend object.json
We are using v20230317 tag in queryai_ocsf repo. To compare the changes simply create a new pr with destination branch as tag. While we cant merge in tags,  file diff can be checked

Great job tracking down the cause of this! Despite editing schema files, I'm on board with this fix.
The only alternative I can think up is to add an optional argument to ewent and object so a default parent class could be provided when the JSON file doesn't specify one. This would let us not edit schema files, but the code generator is already hard to understand and this change is easy to understand.
GitHub is showing the top-level query/ directory as all new files, presumably because it was renamed from Query/, but it doesn't show Query/ being deleted. Were the only changes in that directory to add the "extends" attribute? Will changing the capitalization here have any affect on downstream generated code?
Lastly, could you create a v20230428 tag once this merges and update the git submodules in masala and queryai-codegen to point to this tag? IIRC, you just cd into the submodule directory and do git checkout v20230428, then cd to the parent directory and commit this as a change.

PR shows more changes than were made. Even prior to this, when I worked on query extension, I had noticed, that the tag we had used was not latest main. We cannot use latest main because it has some changes which are not there in these tags


Emphasize cloud shell

@allclark,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

Acrolinx Scorecards
Click the scorecard links for each article to review the Acrolinx feedback on grammar, spelling, punctuation, writing style, and terminology:

Article
Score
Issues
Scorecard

docs-ref-conceptual/get-started-with-azure-cli.md
90
10
link

docs-ref-conceptual/overview.md
90
2
link

More info about Acrolinx

The scorecard links are good for one hour. If you're prompted to authenticate, you can close and reopen the PR to get new links. Or, use these credentials:
Account: report-viewer
Password: kermit$
Report false positive results or other issues

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs-ref-conceptual/get-started-with-azure-cli.md
:white_check_mark:Succeeded
View

docs-ref-conceptual/media/get-started-with-azure-cli/shell-icon.png
:white_check_mark:Succeeded
View

docs-ref-conceptual/media/get-started-with-azure-cli/storage-prompt.png
:white_check_mark:Succeeded
View

docs-ref-conceptual/overview.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Acrolinx Scorecards
Click the scorecard links for each article to review the Acrolinx feedback on grammar, spelling, punctuation, writing style, and terminology:

Article
Score
Issues
Scorecard

docs-ref-conceptual/get-started-with-azure-cli.md
90
10
link

docs-ref-conceptual/overview.md
90
2
link

More info about Acrolinx

Install Acrolinx locally for VSCode

The scorecard links are good for one hour. If you're prompted to authenticate, you can close and reopen the PR to get new links. Or, use these credentials:
Account: report-viewer
Password: kermit$

False positives or issues

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs-ref-conceptual/get-started-with-azure-cli.md
:white_check_mark:Succeeded
View

docs-ref-conceptual/media/get-started-with-azure-cli/shell-icon.png
:white_check_mark:Succeeded
View

docs-ref-conceptual/media/get-started-with-azure-cli/storage-prompt.png
:white_check_mark:Succeeded
View

docs-ref-conceptual/overview.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Update index.md

We need to replace Windows Virtual Desktop with Azure Virtual Desktop

@TomHickling : Thanks for your contribution! The author(s) have been notified to review your proposed change.


[Suggestion] Option to align deciamal places

Could you add an option a column that aligns decimal places? This is fairly common in social sciences.
See: https://tex.stackexchange.com/questions/2746/aligning-numbers-by-decimal-points-in-table-columns
It would be nice if tably could automatically figure out:

The number of places before the decimal point, i.e., the maximal number column.
The number of places after the decimal point, i.e., the maximal number in a particular column.

That way, [table-format=3.2] doesn't have to be specified manually.
In addition, it would be nice if it can handle the 'header column' and automatically wrap the contents
around braces to prevent aligning the decimal point. The comment at the TeX.SX link above provides the context:
in response to a comment by daniel kullmann Nov 15 '11 at 10:31 The official way of getting siunitx to ignore bits of text, like headers, is to just wrap them in braces; i.e. {name} will work just as well as \multicolumn{1}{c}{name}. See for instance tex.stackexchange.com/questions/3709/… (would still like newbies to be able to post comments!) – Sam Mason Sep 23 '12 at 17:41

Could you add an option a column that aligns decimal places?

Thank you for the suggestion! I'll see what I can do.

It would be nice if tably could automatically figure out (...) That way, [table-format=3.2] doesn't have to be specified manually.

I'm thinking maybe an option to specify it manually (something like -d 3.2) might be useful in cases where you want to have the control in your hands (e.g. specify more places before the decimal point than the biggest number has).
Thoughts?

Yes, on further thought, it actually would make more sense for the [table-format=#.#] to be specified manually. The idea is that the CSV file is generated by some external program (e.g., MATLAB) and its numbers have many decimal places. Then, specifying -d 3.2 allows the user to choose how many in the decimal places there should be.
Though, when specifying -d 3.2, it would make sense for tably to directly truncate the output LaTeX source output as well, e.g. using print()'s format specifiers.
Thoughts?


ViewCube Display And Zoom

Description 📝
when model is far, tooeen does not show the model, or perhaps it loads the model but we cannot find it or zoom to the model.

Suggested solution 💡
perhaps, view cube will help us to find the element like most of 3D software does... when we select item from the model tree, we could click on the viewCube for allowing to zoom it to the selected item(s)
Alternative ⛕
No response
Additional context ☝
No response
Validations ✅

[X] Read the docs.
[X] Check that there isn't already an issue that requests the same feature to avoid creating a duplicate.

@ThatMetaverseArchitect can you please also resend this one? Thanks!

We now have features to zoom to specific model or models. You can check a tutorial here. We also now have a navigation cube in the UI library.

@agviegas
Thanks!! great work!


[1.1.5] Cannot finish indexing when using this plugin for EU4 modding

Hello, I have been corresponding with you already regarding this issue, but I wanted to post it here, so it does not go missing.
The indexing in pycharm/IntelliJ never finishes after adding the directory of the EU4 game. Probably there are too many files in that game compared to Stellaris. Indexing for Stellaris works fine, but not for EU4.
I even left it overnight, and it did not finish. CPU and RAM pegged at 100% used. I've got a Ryzen 5950X and 32 GB 3800mhz RAM, so I guess it should be enough?
I've checked it on a different machine and it's the same. Files are getting indexed, but very slowly.

Hi, I have made some optimization, please install the following plugin package manually, check and profile indexing for eu4 game directory again, and then send me the .jfr file for about 5~10 minutes.
This update should optimize the indexing logic for definitions, but there are many other things to be indexed.
Paradox-Language-Support-1.1.6.zip

Will try soon. Thank you so much!!
pt., 11 sie 2023, 06:57 użytkownik 微风的龙骑士 @.***>
napisał:

Hi, I have made some optimization, please install the following plugin
package manually, check and profile indexing for eu4 game directory again,
and then send me the .jfr file for about 5~10 minutes.
This update should optimize the indexing logic for definitions, but there
are many other things to be indexed.
Paradox-Language-Support-1.1.6.zip
https://github.com/DragonKnightOfBreeze/Paradox-Language-Support/files/12318685/Paradox-Language-Support-1.1.6.zip
—
Reply to this email directly, view it on GitHub
https://github.com/DragonKnightOfBreeze/Paradox-Language-Support/issues/49#issuecomment-1674211960,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOBIFXDW3DVTWPM4VAX7X23XUW3RZANCNFSM6AAAAAA3IKN4MY
.
You are receiving this because you authored the thread.Message ID:
@.***
.com>

Will try it out in a second, thank you very much!!

It did not help unfortunately. Looks about the same. One difference I noticed is that RAM usage is lower. It capped out around 4.5 GB even tho I have 8 GB limit in the IDE settings.
Here is the jfr file:
idea64_exe_2023_08_11_084202.zip

During this jfr, it takes 60% time (4000s+) to parse script files, 30% time (2000s+) to indexing. at max 20% time (1400s+) can be optimized by optimizing indexing logic.

So this is just because there are a lot of files in the EU4 game

Do you think it is possible for the plugin to work for this game?

Not sure, optimize indexing logic helps nearly nothing, but perhaps scripts files to parse during indexing can be optmized and reduced. Should install eu4 and check manually.

This finally should be a out of memory problem. Over 60,000,000+ config objects are generated by deep copy, deep merge and inline. Approximately 4G of memory is occupied. There should be no other problems. Indexing process should be finished in 20 minutes If this problem are resolved.

So you are going to be able to fix this issue?

Not sure for those generated config objects are hard to cache.

After lastest commit IDE will freeze during indexing event files.

Does it go through folders alphabetically?

So there is quite a lot more to index.

Cwtools shall not index expressions (e.g. a event target) but only almost definitions and localisations. So it's enough fast. But for my plugin, it shall index expressions. It take too much time to get configs from expressions, to infer it's a event target or some other.

It should only use the memory you give to IDEA. For Stellaris, 4G must be ok.

Until this problem is resolved, maybe thoese additional indices can be disabled, to use PLS together with eu4.

It would be spectacular if you managed to fix this plugin for EU4.

Should be very hard after many updates.

Indexing for eu4 game directory is almost finished after 15 minutes (CPU: 8 cores, RAM: 8G)

Does it work normally after the process is finished? Last time I waited for it to finish, the process would start over every time I had added a new file to the project structure. Does it reindex quickly now (15 minutes is quick)?

If you'd like to, I could test it on my end, but you would need to send the new plugin version.

I am also interested why does it index these files?

1.1.7 still does not work fine for me.

Need some more optimization

The config objects generated during indexing process are too many (3000,000+, 3G+), also the time to consume.
The eu4 game directory is just too crazy. Plugin should parse all script files and localisation files to index definitions, localisations, flags, event targets, and more.
Other files should not be indexed by plugin ,they are just shown by IDEA during scanning.

I'm sorry but this time I cannot resolve this performance problem. Since there are few remaining features to be implemented, Next several versions should try to do some optimization.

Sure, I hope someday it will be fixed. Thank you!

Indexing process for eu4 game directory is once finished over 1 hour under 8 cores CPU & 8 G RAM from my last test.
Note that cwt configs exclude Stellaris is not very perfect, so it may show a lot of errors when open a script file. You may need to send PR by yourself as I don't have so much time and vigor.

although it was once done, since it's too slow, if you open game directory by IDE twice or make lots of changes, it may still cause a soft IDE freeze (you can operate anything but some IDEA background process is just cannot be finished for a long time)

Indexing process for eu4 game directory is finished over about 4 minutes at 1.1.12.
This version I merge several indices (that require resolving cwt configs) into one index. So the speed to indexing should be much faster when there are too many files to indexing. but the speed to access index should be slower since there are more index data to visit.
Please still note that cwt configs exclude Stellaris is not very perfect, so it may show a lot of errors when open a script file.
Since loading local cwt configs is not supported by PLS currently, you may still need to send PR to cwtools-eu4-config by yourself once 1.1.12 is released.


Issue reading the m4a format.

Hi all,
most of the .m4a files I am trying to read with the c2pa.js library have mime-types: audio/x-m4a, this one doesn't seem to be supported
Do you have any idea to be able to read this kind of file?
Is there an option to add the audio/x-m4a in the VALID_MIME_TYPES array?

thanks

Hi there, any news on this issue? Many thanks.


Everything goes wrong in installing. Nothing works.
^--- This feedback is as useful as your documentation.

There's a ticket making great suggestions for the docs which has been open since early 2021. That took me a step further, but even with that extra information I couldn't get it working.
I would recommend creating a fresh project and then doing what ever you need to do to make it work. Assume zero knowledge and prior experience. And write down what you have to do to get things working - with examples - and update the readme.md.
Good luck.

Thank you, that's a great suggestion.
For your information (and for other that may stumble on this), I'm planning on doing just that when godot 4 is stable.

@Pilvinen
We now have working Godot4 port


Introduce a new BinaryLogger logger.

/binaryLogger[:output.binlog]
Serializes all build events to a compressed binary file. By default the file is in the current directory and named "msbuild.binlog". The binary log is a detailed description of the build process that can later be used to reconstruct text logs and used by other analysis tools. A binary log is usually 10-20x smaller than the most detailed text diagnostic-level log, but it contains more information. (Short form: /bl or /binlog)
Examples:
/bl
bl:output.binlog
binlog:....\custom.binlog
binaryLogger
For now this commit only includes the logger (writer), but the reader is disconnected. In subsequent PRs we will add an option to playback the binary log into any other set of existing loggers.
Support for the new format will be added to MSBuild Structured Log Viewer so that you can view the .binlog files.
BuildEventArgsWriter and BuildEventArgsReader are general purpose custom binary serializers for BuildEventArgs objects.
BinaryLogReplayEventSource is a way to playback events from the log to subscribers of IEventSource.
A set of unit-tests is to follow in a separate PR.

Looks like you're not far off on the .NET Core build. Close() isn't supported on the stream (but you can use Dispose()), and Delegate.CreateDelegate() isn't supported until we move to .NET Standard 2.0. So have to find another way to implement that unfortunately.

Looks like a great start! My biggest worry here is that the log format is extremely terse. Obviously this is good for this purpose, but very difficult to debug. You did mention a second PR for unit tests, so hoping you have some dedicated to reader/writer specifically.

@dotnet-bot test Windows_NT Build for Desktop please

If you guys are OK with it, I'd like to get this in. I'll continue iterating, adding more tests and will hook up the reader, but I'd like to do that in a separate PR.


[Storage] 100continue on RST

Closed #41037 as we don't currently have reliable tooling to test. Needs to be readdressed in the future. 100continue still works for intended status code responses.

Related https://github.com/Azure/azure-sdk-for-net/pull/40611


Vertical list instead of horizontal list in web editor

Hi
I love the web editor but I'd love an option to have the list of groups to appear in a vertical list to the left instead of a horizontal tab on the top.
And options to sort that list by name would be nice too, as of now i notice it goes by the weight of the group and then alphabetical.
Thanks for an amazing plugin!
//Etrisa

Added in v2 (please install LuckPerms v5 to use it).


ignore_transaction stopped working on <IP_ADDRESS>

I was using version <IP_ADDRESS> and this was working fine:
@app.get('/')
def index():
    newrelic.agent.ignore_transaction(flag=True)
    return 'Hello World!'

By updating to <IP_ADDRESS>, it started showing up in New Relic dashboard.
Tested in <IP_ADDRESS> and the bug persists.
Using Uvicorn / FastAPI, Python 3.8.
newrelic-admin run-program uvicorn server:app

Hi @danielapsmaior , thanks for filing this issue! I believe this is related to a known issue with the agent's monitoring of starlette/fastapi that the team is currently working on (#5)!
TL/DR
The agent doesn't see any transaction running when newrelic.agent.ignore_transaction() is called and therefore, there is no transaction to be ignored. To work around this problem while we address it in the agent code (next release), one of the following should be done:

Disable uvicorn instrumentation by adding the following to your newrelic.ini:[import-hook:uvicorn.config]
enabled=false

Change the signature of def index to async def index

Could you also confirm if removing newrelic.agent.ignore_transaction still results in the transaction being missing on <IP_ADDRESS>?
Details
The background is related to how the agent internally tracks the "current transaction" or "current trace/segment". The implementation for this tracking is in trace_cache.py.
The agent's data structures around timing, error tracking, etc. are not considered thread safe. The agent is designed so that once a transaction is started on a particular thread, it tracks the transaction only within that thread. This design limitation (until addressed) means that any time a transaction crosses a thread boundary, no additional details / tracking of the transaction can be done on the new thread.
So, what's the deal here, and how is the above relevant? Well, starlette runs all non-coroutine routes on a separate thread and so does fastapi. That currently means that any non-async routes lose the "current transaction" anywhere it is needed. Timing information is still accurate, but details will be omitted.
I suspect that the call to newrelic.agent.ignore_transaction() didn't actually work in <IP_ADDRESS> either. Before 5.20.0, the agent didn't include automatic instrumentation for uvicorn, so a transaction must have been started manually. What likely happened is the transaction wasn't started for the function being "ignored", so it gave the perception that the ignore call was actually working. 😄

@a-feld Thank you for your answer!
I confirm that putting async in all signatures worked! And also that removing newrelic.agent.ignore_transaction from <IP_ADDRESS> still keeps ignoring transaction!
Thank you again and I'm very excited with the agent support to FastAPI :🌟

@danielapsmaior We've released starlette / FastAPI support, so this issue should be resolved! 🎉


Enable usign external images in the report

This enable using external images passing by parameter into report

Thank you for your help!


[Event Request] Table 5965 Service Contract Header - Next Invoice Date - OnValidate

Issue #3704 only has been partly implemented. Could you also add the second requested publisher of this issue? Many thanks.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.


ws client duplicate transactions

Im using the websocket client on MagicEden program logs
from solana.rpc.websocket_api import (
   connect,
   RpcTransactionLogsFilterMentions,
)

async def main():
   async with connect(RPC_WS_ENDOINT) as wsClient:
      await wsClient.logs_subscribe(filter_=RpcTransactionLogsFilterMentions(MagicEden.program_address))
      first_resp = await wsClient.recv()
      me_log_sub_id = first_resp[0].result
      async for _, res in enumerate(wsClient):
         data = res[0]
         print(data.result.value.signature)
      await wsClient.logs_unsubscribe(me_log_sub_id)

asyncio.run(main())

This generates  lots of duplicate responses

am I missing something?

You're most likely getting the 1st message when the transaction reached "committed" status, then another when it is "finalized". Try setting: commitment='confirmed' in logs_subscribe()

My bad, I haven't included the Confirmed configuration, and I'm still getting duplicated transactions. As you said, it seems that I'm receiving the transaction when it's confirmed and then finalized.


YoutubeDL - How to get a status object after download has completed

I'm trying basically to get information out of what seems to be a status object that's hitting the hook in Youtube-DL, and then I'm trying to save it to the db. I've got a 'song' object with attributes such as "filename" I'm trying to save once the download is complete, and maybe even continually update the database with progress.
There's three ways I can think of to do this but I've not been able to get them to work

Send the my_hook function a db and song object and then save it all in there once status == finished. Problem is I'm unable to pass additional parameters to the hook unless I'm missing something
Get the my_hook function to return d and then save that, problem is that I don't think I can access that it would return to (youtube-dl source)
Get ydl.download([song.url]) to return a status object that I can process, I don't think it does this though
I don't want to do this, but I can output a .json file and get it from there, or just simply guess the name of the file given that I'm dictating it initially :-1:

def my_hook(d):
    if d['status'] == 'finished':
        file_tuple = os.path.split(os.path.abspath(d['filename']))
        print("Done downloading {}".format(file_tuple[1]))
    if d['status'] == 'downloading':
        print(d['filename'], d['_percent_str'], d['_eta_str'])

class MyLogger(object):
    def debug(self, msg):
        pass

    def warning(self, msg):
        pass

    def error(self, msg):
        print(msg)

class Downloader(object):
    def get_opts(self):
        ydl_opts = {
            'format': 'bestaudio/best',
            'outtmpl': os.path.join(app.config['VIDEOS_FOLDER'], '%(id)s.%(ext)s'),
            'logger': MyLogger(),
            'progress_hooks': [my_hook],
        }
        return ydl_opts

    def download(self, song):
        ydl = youtube_dl.YoutubeDL(self.get_opts())
        ydl.download([song.url])

I don't see any problem with first approach. When download is finished my_hook is fired with finished status and you can put here any logic either writing to db right here or putting it to dbmanager queue or whatever you have. You can do anything in my_hook since it's your code.

@dstftw I don't see how to pass the hook the relevant variables it needs in order to update the db or even get the row ID in order to do so.
I've tried altering this def my_hook(d): to be more like this def my_hook(d, song, db): but that causes problems because other parts of the youtube-dl code only expect it to be taking the status object.
Not only that, but then the line 'progress_hooks': [my_hook], expects me to detail out what my_hook should be expecting, but that also fails also.
Can you show an example where I can pass parameters to my_hook?

You don't need to pass them to hook. You should resolve song, db and so on in your code.
def handle_finished(d):
    song = get_song_by_filename(d['filename'])
    db = get_db()
    db.write(song)

def my_hook(d):
    if d['status'] == 'finished':
        file_tuple = os.path.split(os.path.abspath(d['filename']))
        print("Done downloading {}".format(file_tuple[1]))
        handle_finished(d)
    if d['status'] == 'downloading':
        print(d['filename'], d['_percent_str'], d['_eta_str'])

Yes, sorry I absolutely agree that I can create a new db object from the hook but the issue is that the 'filename' is not unique so I actually need to get the originating unique song.id known to my program and not youtube-dl - the only way I see that to be possible is to somehow pass it from def download(self, song): to the hook

filename must be unique otherwise how do you expect it to behave when downloading two videos in the same file? id in output template provides well enough unicity. However If you want guaranteed unicity just add a guid to output template.

how do you expect it to behave when downloading two videos in the same file?
I didn't explain that well, the song file is not unique within the "song" table, it's possible for multiple entries in that table to be pointing at the same file. It's actually more like a "song request" that has a "song file" entry but all in one table.

Anyway thinking about it from the POV of your comments this may point to problems in the model itself, because of the one to many nature of "song requests" to "song files" I should break it down, this will make the file name unique in the "song file" entry, and I can fetch by youtubeID.

Hi Michael,
Would you be willing to share your code how to add downloaded descriptions to the database? Did you use MySQL?
Thanks

Hi, the database I was using doesn't matter - I was using http://www.sqlalchemy.org/ which would allow me to connect the code to any supported database. It's usually quickest to simply create and connect to a SQLite DB for dev, then test on the target prod type DB, then deploy.
Have a read up on how sqlalchemy works, you simply define a model, objects with attributes represent tables and columns, you can then populate that model into a connected DB.
The code it out in the open if you want to take a look:
https://github.com/MichaelJAndy/flask-wallboard-music


feat: enable CMYK responsive

I enable responsive behavior for CMYK page
Remove the package-lock.json since there was a yarn.lock file

LGTM! Thank you, @glrodasz 🙏


Python 3.5 Bug: No need to import izip

Firstly, thanks for the wonderful reference implementation!
dvf@ubuntu-us-nyc-01:~ twistd -noy server.tac
Unhandled Error
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/twisted/application/app.py", line 662, in run
    runApp(config)
  File "/usr/local/lib/python3.5/dist-packages/twisted/scripts/twistd.py", line 25, in runApp
    _SomeApplicationRunner(config).run()
  File "/usr/local/lib/python3.5/dist-packages/twisted/application/app.py", line 380, in run
    self.application = self.createOrGetApplication()
  File "/usr/local/lib/python3.5/dist-packages/twisted/application/app.py", line 445, in createOrGetApplication
    application = getApplication(self.config, passphrase)
--- <exception caught here> ---
  File "/usr/local/lib/python3.5/dist-packages/twisted/application/app.py", line 456, in getApplication
    application = service.loadApplication(filename, style, passphrase)
  File "/usr/local/lib/python3.5/dist-packages/twisted/application/service.py", line 412, in loadApplication
    application = sob.loadValueFromFile(filename, 'application')
  File "/usr/local/lib/python3.5/dist-packages/twisted/persisted/sob.py", line 177, in loadValueFromFile
    eval(codeObj, d, d)
  File "server.tac", line 7, in <module>
    from kademlia.network import Server
  File "/usr/local/lib/python3.5/dist-packages/kademlia/network.py", line 13, in <module>
    from kademlia.storage import ForgetfulStorage
  File "/usr/local/lib/python3.5/dist-packages/kademlia/storage.py", line 2, in <module>
    from itertools import izip
builtins.ImportError: cannot import name 'izip'

Since Twisted supports 3.6, how about:
try:
    from itertools import izip
except ImportError:
    from itertools import zip
...

There's a python 3.5 branch that doesn't use Twisted, I recommend just using that.  I don't think I'll be supporting 2.7 or Twisted going forward.

Awesome. I didn't see that. Thanks @bmuller
Are there any plans to add the Python 3 build to pypi?


[python/viewer] Add support of user-specified extra cameras (rgb and depth).

import numpy as np
import matplotlib.pyplot as plt

import gymnasium as gym

from panda3d.core import VBase4, Point3, Vec3
from jiminy_py.viewer import Viewer
import pinocchio as pin

Viewer.close()
#Viewer.connect_backend("panda3d-sync")
env = gym.make("gym_jiminy.envs:atlas-pid", viewer_kwargs={"backend": "panda3d-sync"})
env.reset(seed=0)
env.step(env.action)
#env.render()
env.simulator.render(return_rgb_array=True)

env.viewer.add_marker("sphere",
                      shape="sphere",
                      pose=(np.array((1.7, 0.0, 1.5)), None),
                      color="red",
                      radius=0.1,
                      always_foreground=False)

#Viewer.add_camera("rgb", height=200, width=200, is_depthmap=False)
Viewer._backend_obj.gui.add_camera("rgb", False, (200, 200))
#Viewer.add_camera("depth", height=128, width=128, is_depthmap=True)
Viewer._backend_obj.gui.add_camera("depth", True, (128, 128))
Viewer.set_camera_transform(
    position=[2.5, -1.4, 1.6],  # [3.0, 0.0, 0.0],
    rotation=[1.35, 0.0, 0.8],  # [np.pi/2, 0.0, np.pi/2]
    camera_name="depth")

frame_index = env.robot.pinocchio_model.getFrameId("head")
frame_pose = env.robot.pinocchio_data.oMf[frame_index]
# Viewer._backend_obj.gui.set_camera_transform(
#     pos=frame_pose.translation + np.array([0.0, 0.0, 0.0]),
#     quat=pin.Quaternion(frame_pose.rotation @ pin.rpy.rpyToMatrix(0.0, 0.0, -np.pi/2)).coeffs(),
#     camera_name="rgb")
Viewer.set_camera_transform(
    position=frame_pose.translation + np.array([0.0, 0.0, 0.0]),
    rotation=pin.rpy.matrixToRpy(frame_pose.rotation @ pin.rpy.rpyToMatrix(np.pi/2, 0.0, -np.pi/2)),
    camera_name="rgb")

lens = Viewer._backend_obj.render.find("user_camera_depth").node().get_lens()
# proj = lens.get_projection_mat_inv()
# buffer = Viewer._backend_obj._user_buffers["depth"]
# buffer.trigger_copy()
# Viewer._backend_obj.graphics_engine.render_frame()
# texture = buffer.get_texture()
# tex_peeker = texture.peek()
# pixel = VBase4()
# tex_peeker.lookup(pixel, 0.5, 0.5)  # (y, x normalized coordinates, from top-left to bottom-right)
# depth_rel = 2.0 * pixel[0] - 1.0   # map range [0.0 (near), 1.0 (far)] to [-1.0, 1.0]
# point = Point3()
# #lens.extrude_depth(Point3(0.0, 0.0, depth_rel), point)
# # proj.xform_point_general(Point3(0.0, 0.0, pixel[0]))
# # depth = point[1]
# depth = 1.0 / (proj[2][3] * depth_rel + proj[3][3])
# print(depth)

rgb_array = Viewer.capture_frame(camera_name="rgb")
depth_array = Viewer.capture_frame(camera_name="depth")
# depth_normalized_array = lens.near / (lens.far - (lens.far - lens.near) * depth_array)
depth_true_array = lens.near / (1.0 - (1.0 - lens.near / lens.far) * depth_array)
fig = plt.figure(figsize=(10, 5))
ax1 = fig.add_subplot(121)
ax1.imshow(rgb_array)
ax2 = fig.add_subplot(122)
ax2.imshow(depth_true_array, cmap=plt.cm.binary)
for ax in (ax1, ax2):
    ax.axis('off')
    ax.xaxis.set_visible(False)
    ax.yaxis.set_visible(False)
fig.tight_layout(pad=1.0)
plt.show(block=False)

@mwulfman This is just a POC. Applying filters to make the depth map more realistic is still necessary. Do you have any reference or even source code about this topic ? It would be very helpful.

If it can help you, here's what I started to draft. It lacks filtering capabilities on the output images. I believe, besides the approach (using class versus adding it to the viewer), it is very similar to what you've done. I've been considering using cv2 directly for doing the filter. I believe capturing the frame takes around the same amount of time than you (1-2 ms)
class DepthMapSensorPOC:
    """Depth Map Sensor Class"""

    def __init__(
        self,
        panda3d_app: Panda3dApp,
        window_size: Tuple[float, float],
        depth_range: Tuple[float, float],
        field_of_view: Tuple[float, float],
    ):
        winprops = WindowProperties(size=window_size)
        props = FrameBufferProperties()
        props.set_float_depth(True)
        props.set_depth_bits(32)
        depth_buffer = panda3d_app.graphics_engine.make_output(
            panda3d_app.pipe,
            "depth_buffer",
            -2,
            props,
            winprops,
            GraphicsPipe.BFRefuseWindow,
            panda3d_app.win.get_gsg(),
            panda3d_app.win,
        )

        depth_texture = Texture("depth_texture")
        depth_buffer.add_render_texture(
            depth_texture, GraphicsOutput.RTM_triggered_copy_ram, GraphicsOutput.RTPDepthStencil
        )

        self.depth_buffer = depth_buffer

        self.depth_cam = panda3d_app.make_camera(self.depth_buffer, camName="depth_cam")
        self.depth_min, self.depth_max = depth_range
        self.depth_cam.node().getLens().set_near_far(self.depth_min, self.depth_max)
        self.depth_cam.node().getLens().set_fov(*field_of_view)

        self.depth_cam.reparent_to(panda3d_app.render)
        self.depth_cam.set_pos(0.0, 0.0, 0.0)
        self.depth_cam.set_hpr(0.0, 0.0, 0.0)

        # for debug, don't use for real run !
        # It adds black pixels on side of depth image
        # self.depth_cam.node().showFrustum()

        panda3d_app.graphics_engine.render_frame()

        self.depth_buffer.inverted = True

        panda3d_app.bufferViewer.setInclude([self.depth_buffer])
        panda3d_app.bufferViewer.enable(1)

    def get_camera_depth_image(self, graphics_engine):
        """
        Returns the camera's depth image, which is of type float32 and has
        values between 0.0 and 1.0.
        """
        self.depth_buffer.trigger_copy()
        graphics_engine.render_frame()

        depth_texture = self.depth_buffer.get_texture()
        data = depth_texture.get_ram_image()
        depth_image = np.frombuffer(data, np.float32)
        depth_image.shape = (depth_texture.get_y_size(), depth_texture.get_x_size())
        normalize_world_depth = np.clip(
            self.depth_min / (self.depth_max - (self.depth_max - self.depth_min) * depth_image),
            0.0,
            1.0,
        )
        return normalize_world_depth

I have an unreliable setup where computation of the depth map takes only 700us on my machine. If I make find a way to achieve this speed without breaking anything else, then using depth map in RL may be an option !

I have an unreliable setup where computation of the depth map takes only 570us on my machine. If I make find a way to achieve this speed without breaking anything else, then using depth map in RL may be an option !
Edit: I found a reliable way 🎉 @mwulfman it would be nice if you could benchmark it on a machine with Nvidia GPU.
Edit2: After some in-depth profiling, this appears to be the absolute limit because now 90% of the time is spent on copying the result from the GPU to the CPU. I don't think it is possible to speed up this operation but I will have a look. The good thing is that decreasing the resolution is now improving the timing, eg it takes 390us for 64 x 64 depth map.

Just seen your message. Do you have a reliable script that I can run to benchmark the depth map acquisition speed?

Just seen your message. Do you have a reliable script that I can run to benchmark the depth map acquisition speed?

Yes, execute the script provided in the PR description in ipython, then run the following command:
%timeit Viewer.capture_frame(camera_name="depth")

I am getting the following error after installing jiminy 1.8.8 and trying to run the script in an ipython:
Cell In[3], line 1
----> 1 Viewer.add_camera("rgb", height=200, width=200, is_depthmap=False)

File ~/.pyenv/versions/3.10.9/envs/test/lib/python3.10/site-packages/jiminy_py/viewer/viewer.py:299, in _with_lock.<locals>.fun_safe(*args, **kwargs)
    297 self = kwargs.get('self', self)
    298 with self._lock:
--> 299     return fun(*args, **kwargs)

File ~/.pyenv/versions/3.10.9/envs/test/lib/python3.10/site-packages/jiminy_py/viewer/viewer.py:287, in _must_be_open.<locals>.fun_safe(*args, **kwargs)
    283 if not self.is_open():
    284     raise RuntimeError(
    285         "No backend available. Please start one before calling "
    286         f"'{fun.__name__}'.")
--> 287 return fun(*args, **kwargs)

File ~/.pyenv/versions/3.10.9/envs/test/lib/python3.10/site-packages/jiminy_py/viewer/viewer.py:1922, in Viewer.add_camera(camera_name, width, height, is_depthmap)
   1918     raise NotImplementedError(
   1919         "This method is only supported by Panda3d.")
   1921 # Add camera
-> 1922 Viewer._backend_obj.gui.add_camera(
   1923     camera_name, (width, height), is_depthmap)

File ~/.pyenv/versions/3.10.9/envs/test/lib/python3.10/site-packages/jiminy_py/viewer/panda3d/panda3d_visualizer.py:824, in Panda3dApp.add_camera(self, name, is_depthmap, size)
    822     fbprops.set_multisamples(4)
    823 winprops = WindowProperties()
--> 824 winprops.set_size(*size)
    825 flags = GraphicsPipe.BF_refuse_window
    826 buffer = self.graphicsEngine.make_output(
    827     self.pipe, f"user_buffer_{name}", 0, fbprops, winprops, flags,
    828     self.win.get_gsg(), self.win)

TypeError: WindowProperties.set_size() argument after * must be an iterable, not bool

Arf damn, the code being merged is broken... I dirty-fixed the snippet in PR description. It should be enough to run the benchmark!

Here the results on Nvidia GPU RTX 3060:
In [2]: %timeit Viewer.capture_frame(camera_name="depth")
387 μs ± 1.9 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

In [3]: %timeit Viewer.capture_frame(camera_name="depth")
388 μs ± 3.04 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

In [4]: %timeit Viewer.capture_frame(camera_name="depth")
405 μs ± 14.7 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

In [5]: %timeit Viewer.capture_frame(camera_name="depth")
388 μs ± 9.6 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

In [6]: %timeit Viewer.capture_frame(camera_name="depth")
388 μs ± 2.93 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

Nice ! These results are encouraging, thanks :)
Apparently, the same performance is the same as running on Intel integrated, which makes sense because computations are too small to take avantage of the available resources. If you have time, I'm curious to see if the timing changes after increasing the resolution:
Viewer._backend_obj.gui.add_camera("depth", True, (200, 200))

or even
Viewer._backend_obj.gui.add_camera("depth", True, (500, 500))

200x200:
In [2]: %timeit Viewer.capture_frame(camera_name="depth")
370 μs ± 61.3 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

In [3]: %timeit Viewer.capture_frame(camera_name="depth")
395 μs ± 27.1 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

In [4]: %timeit Viewer.capture_frame(camera_name="depth")
385 μs ± 2.01 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

In [5]: %timeit Viewer.capture_frame(camera_name="depth")
395 μs ± 9.23 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

500x500:
In [2]: %timeit Viewer.capture_frame(camera_name="depth")
432 μs ± 29.5 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

In [3]: %timeit Viewer.capture_frame(camera_name="depth")
430 μs ± 24.5 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

In [4]: %timeit Viewer.capture_frame(camera_name="depth")
425 μs ± 18.2 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

Ok perfect, just like I expected :) From these results, I think with a powerful GPU it should not be a problem to generate the depth map for 10 / 20 env in parallel if the resolution is small enough.


Feature Request - Update Available

Feature Request - It would be awesome if this integration was able to tell us if there is an update available. Otherwise great job on the integration works well.
I host a Minecraft Server for my brother and I usually don't find there is an update until his computer updates to a newer major version which then makes it not possible for him to play. I'd then use an automation to tell me that the current and update versions don't match and send me a notification. Probably didn't need to know this part. lol
Thanks again

Hi @mhump711, you might want to look into the bedrock server update script I use on Linux Hosted servers. how are you hosting your bedrock server?

I wasn't planning on adding this, as my server updates automatically every night. I use: https://jamesachambers.com/minecraft-bedrock-edition-ubuntu-dedicated-server-guide/
Also the bedrock API I use only pulls in a protocol version, which I am not sure directly maps to a new release...

I think that will help out @mhump711 so it's not really a feature request. they can have their server update like the above script. I use the same. Could look at borrowing a line from that script to return the published server version from the website that script uses?

I use Cubecoders AMP. I wanted something with a web interface. Admittedly I get how the Minecraft server is laid out so I don't really need it anymore but it already works. I check out the script tomorrow but most like I can do a nightly restart and have update on boot.


Jetty 9 goes EOL with April 2022

The Jetty 9 release line will go into Community EOL in April 2022. We should offer a Ringo release with a newer Jetty until then. Since this would further hold back the 3.0.0 release, I created a 3.1.0 milestone for this update.

Fixed in #449


fix search after app-router enhancement

the search is broker after https://github.com/Polymer/polymer-element-catalog/pull/188.  It was
relying on the browse page being created everytime the app-router
switches to the page.  But with the enhancement in app-router we are no
longer re-creating the page so we need to move the input focus logic to
when q changes instead of in attached

LGTM


Fix agent concurrency problems

Locks are needed at some places. High caution. Done in near death condition.

@vincentbernat please rebase? I'll review & merge


view and copy sequence diagrams

Closes #134
not 100% about the copy sequence diagrams

Codecov Report
Base: 70.03% // Head: 70.03% // No change to project coverage :thumbsup:

Coverage data is based on head (1e05b51) compared to base (b5dbbdc).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff            @@
##             master     #142   +/-   ##
=========================================
  Coverage     70.03%   70.03%
  Complexity      641      641
=========================================
  Files           100      100
  Lines          2016     2016
  Branches        221      221
=========================================
  Hits           1412     1412
  Misses          535      535
  Partials         69       69

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Timestamp is deprecated

Disclaimer: Just reviewing code at this point.
https://github.com/go-rel/mssql/blob/e7d7d121f21fde19a9e6d66c8a9f946631b9d868/mssql.go#L143-L145
The "timestamp" keyword for a data type is deprecated and will be removed in a future version.  They suggest using "rowversion".
https://docs.microsoft.com/en-us/sql/t-sql/data-types/rowversion-transact-sql?view=sql-server-ver15#remarks

hi, thanks for your feedback again 😄
hmm, in that case timestamp/rowversion has a totally different meaning compared to other database.
it seems different database have different definition for timestamp, probably better to drop this type from rel package and let the driver defines it in the driver package? 🤔

I don't know enough about the other databases to know how they handle this.  I'm just glad you're thinking about it :-)


Fix issue with launch wizard parsing

Fix issue with launch wizard parsing

Update launch wizard to parse the daffodilDebuggerClasspathTable items textContent instead of innerHTML.

Using innerHTML was causing an issue where the wizard would set the classpath as the css class name "debug-classpath-item" instead of its actual value.

@nlewis05 please make sure this PR does in fact fix this issue you were seeing an approve if it does.


PoN in gCNV

Dear @fobinf
Hi, I'm Oh.
Many thanks for this great system.
Then, I have two questions.
Q1.
In your tool and preprint (bjiorxiv., 2021), your team use PoN in gCNV calling.
However, there was no PoN making step in GATK best practices.
https://github.com/broadinstitute/gatk/blob/master/scripts/cnv_wdl/germline/cnv_germline_case_workflow.wdl
https://github.com/broadinstitute/gatk/blob/master/scripts/cnv_wdl/germline/cnv_germline_cohort_workflow.wdl
Or, the PON you are talking about is the output of the cohort mode?
What do you think?
Q2. (If PON means cohort mode is correct)
Can you make a model with COHORT MODE by putting all your WGS-derived hdf5 (or tsv) files (n=70) and execute CASE mode n times to get the final CNV result?
Many Thanks.
Oh.

Dear Oh,
We don't really cover PoN in the manuscript, and for sure not gCNV, but found it important to mention, because we (after these investigations) found that this helped us a lot - so we would not keep this information to ourselves.
We are currently using svdb https://github.com/J35P312/SVDB to store our PoN, and soft filter our results. I hope this will help you to get started.
Best,
Frederik


optimize: reduce distributed learning code complexity

This commit reduces the complexity of the distributed learning code by storing the ends who agree with the ring topology at a time, so that a trainer in a ring won't send weights message to trainers outside of the ring. Meanwhile, a new trainer's arrival message won't send to the trainers in a ring either. Hence, the moment that a new trainer fetches the global optimal weights will be when the new trainer is in the ring.

type
file
changes

modified
lib/python/flame/mode/distributed/trainer.py
store ends in a ring separately

Codecov Report

Merging #161 (e1a2629) into main (77ed492) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #161   +/-   ##
=======================================
  Coverage   21.29%   21.29%
=======================================
  Files          34       34
  Lines        1503     1503
=======================================
  Hits          320      320
  Misses       1173     1173
  Partials       10       10

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 77ed492...e1a2629. Read the comment docs.


Fixed JMH ByteBuf benchmark to avoid dead code elimination

Motivation:
The JMH doc suggests to use BlackHoles to avoid dead code elimination hence would be better to follow this best practice.
Modifications:
Each benchmark method is returning the ByteBuf/ByteBuffer to avoid the JVM to perform any dead code elimination.
Result:
The results are more reliable and comparable to the others provided by other ByteBuf benchmarks (eg HeapByteBufBenchmark)

Cherry-picked into 4.1 (1cf268724430131ce5c7f4c2068498d429125ac0).
@franz1981 thanks!


build(deps): Bump autovalue version to 1.10.2

Thank you for opening a Pull Request! For general contributing guidelines, please refer to contributing guide
Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

[x] Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
[x] Ensure the tests and linter pass
[x] Code coverage does not decrease (if any source code was changed)
[x] Appropriate docs were updated (if necessary)

google-cloud-core and api-common need shared-configs v1.5.7 to resolve the enforcer issue.
Might need to reach out to the oauth team about bumping auto-value annotations.

Error:
Error:  Rule 2: org.apache.maven.plugins.enforcer.RequireUpperBoundDeps failed with message:
Failed while enforcing RequireUpperBoundDeps. The error(s) are [
Require upper bound dependencies error for com.google.auto.value:auto-value-annotations:1.10.1 paths to dependency are:
+-com.google.cloud:google-cloud-core:2.21.1-SNAPSHOT
  +-com.google.auto.value:auto-value-annotations:1.10.1
and
+-com.google.cloud:google-cloud-core:2.21.1-SNAPSHOT
  +-com.google.api:api-common:2.14.1-SNAPSHOT
    +-com.google.auto.value:auto-value-annotations:1.10.1 (managed) <-- com.google.auto.value:auto-value-annotations:1.10.2
and
+-com.google.cloud:google-cloud-core:2.21.1-SNAPSHOT
  +-com.google.auth:google-auth-library-oauth2-http:1.19.0
    +-com.google.auto.value:auto-value-annotations:1.10.1 (managed) <-- com.google.auto.value:auto-value-annotations:1.10.1
]

Ah Tomo already has a PR for this. Closing in favor for: https://github.com/googleapis/sdk-platform-java/pull/1859


Introduce Vault feature

Q
A

Bug fix?
no

New feature?
no

API breaks?
no

Deprecations?
no

License
Apache 2.0

What's in this PR?
Introduce new Vault secrets webhook feature
Checklist

[x] Implementation tested (with at least one cloud provider)
[x] Error handling code meets the guideline
[x] Logging code meets the guideline (TODO)

I will open a new PR. In this one the Build docker image check cannot build successfully because of the name of the branch.


Extending CLI session duration & Okta+awscli setup documentation

Hello, I recently had an Okta support case (# 00465201) open for this, but after a little back and forth the case owner advised that I instead open a Github issue ticket for the dev team to address.
My initial inquiry is about extending the duration of CLI sessions.  I know there have been other questions about this, but earlier this year AWS introduced the ability to extend the CLI session for federated logins up to 12 hours (from the default of 1).  Per the case and as a test, I tried adding the OKTA_STS_SESSION parameter to my config.properties file (as described here https://github.com/oktadeveloper/okta-aws-cli-assume-role#configuring-the-application), but my federated CLI sessions continue to timeout after 1 hour.
I then got to thinking that perhaps my version of the okta-cli tool is out of date, as I originally setup our Okta+AWS integration back in 2016.   I noticed there is a new(er) version of the okta-aws-cli jar file (https://github.com/oktadeveloper/okta-aws-cli-assume-role/releases), so then thought mabye I needed to download that and/or the entire repo and start from scratch. Unfortunately, downloading just the new jar file (okta-aws-cli-1.0.2.jar) or the entire repo is not helping. I keep running into "request has expired" errors or java issues.
Additionally, the instructions on both the Github code page and the Okta support page (https://support.okta.com/help/Documentation/Knowledge_Article/Integrating-the-Amazon-Web-Services-Command-Line-Interface-Using-Okta) appear to not be correct/entirely up-to-date. In some spots they are a little vague, they seem to be missing steps for use on Windows machines, and not fully updated. For example, the latest jar file to download is named okta-aws-cli-1.0.2.jar, but everywhere in the setup/installation steps the instructions talk about the old file name (oktaawscli.jar). The instructions do not say if we are supposed to rename the file, etc. The Github instructions also mention a lib folder, as well as needing the AWS Java SDK, but the current repo does not contain a lib folder. The old installation (aka what I've been using for almost 2 years) does.
If I download the entire repo and try to start from scratch, I am not completely sure which files in the repo are absolutely needed and which I can perhaps dispose of. When I originally setup the AWS-Okta-cli integration 2 years ago, I was advised I only needed a lib and out directory and the files therein.
Thank you.
Tom

Hello, I am still looking for some assistance here.  Thank you.

Not sure why no one from Okta acted on this. I'm converting a couple of install scripts I have and will contribute those soon. Windows, macOS, and Linux should work with these scripts.

This is related to #88 and #93.

@tchilcutt I have installers for Windows and macOS/Linux coming in #146. They can be used today with minor tweaks.
Note: to set expecations: I don't work for Okta. I'm an Okta customer. I am not paid to work on this and do it mostly on my personal time.

Hello @AlainODea thank you very much for your replies and for providing your installers.  I appreciate it.  Unfortunately, I have been too busy to give them a try, and I am now leaving on holiday, but I will certainly give them a spin upon my return.
And yes, it is quite unfortunate that I have still not received any response from Okta support.  Hopefully I will one day.

@tchilcutt thank you for your kind words and support. It’s my pleasure. I’m glad when these things work out.
Okta is still finding it’s feet with developer advocacy. The acquisition of StormPath was a good move and once the folks get past the huge hurdle of building the initial sustainable SDKs and frameworks, I think we’ll see a major uptick in developer support engagement.

@tchilcutt any luck with the updated readme and installers?

Hello @AlainODea, I just returned from holiday last night.  I have quite a bit of catching up to do but I will let you know when I get around to testing your fix.  Much appreciated.

@tchilcutt my pleasure. If you run into any issues, let me know and I'll do my best to address them. We make really heavy use of this tool now where I work, so having folks identify all the rough edges really helps me and my colleagues too.

@tchilcutt the STS duration is likely broken by writing ~/.aws/config. I've pushed up #211 to stop altering that file which should mean AWS CLI uses the role assumed from Okta directly rather than trying to assume that role itself and overwriting the OKTA_STS_DURATION you set with a 3600 second (1 hour) default. It's waiting on code review.


[Bug]: Cannot add a key when using keyring-backend=os

What went wrong?
Trying to add a key using cheqd-noded cli on a Linux machine doesn't work if keyring-backend=os, if using keyring-backend=test then it works.
How can we reproduce this bug?
Testing on an Ubuntu (20.04) Linux machine. Using cheqd-noded version 0.6.0.
Trying to add a key on my machine and setting the keyring-backend to os the key is not stored. But if I use test the key is stored.
Command to add key with keyring-backend=os
$ cheqd-noded keys add local-cheqd --keyring-backend=os

- name: local-cheqd-an
  type: local
  address: cheqd1rz9w6y0m2arqh...
  pubkey: '{"@type":"/cosmos.crypto.secp256k1.PubKey","key":"A/3AjLA..."}'
  mnemonic: ""

**Important** write this mnemonic phrase in a safe place.
It is the only way to recover your account if you ever forget your password.

twenty radar ...

I get prompt ("Choose a password for the new keyring") by the operating systems to set a password to store the key and I chose a password and confirm it. But if I query the keys it doesn't show up:
$ cheqd-noded keys list --keyring-backend=os
[]

Command to add key with keyring-backend=test
$ cheqd-noded keys add local-cheqd --keyring-backend=test

- name: local-cheqd
  type: local
  address: cheqd1q9c....
  pubkey: '{"@type":"/cosmos.crypto.secp256k1.PubKey","key":"A9+P8LqPR8cB...."}'
  mnemonic: ""

**Important** write this mnemonic phrase in a safe place.
It is the only way to recover your account if you ever forget your password.

poet glory...

And listing the keys in the test works:
$ cheqd-noded keys list --keyring-backend=test
- name: local-cheqd
  type: local
  address: cheqd1q9c...
  pubkey: '{"@type":"/cosmos.crypto.secp256k1.PubKey","key":"A9+P8LqPR8cB...."}'
  mnemonic: ""

Environment
Not applicable
Bug prevalence
Everytime I run the command
Which browser/client application did you encounter the bug in? (if applicable)
Other
Relevant log output
no logs, I've added the commands and messages above

Code of Conduct

[X] I agree to follow this project's Code of Conduct

That's odd. What happens if you don't explicitly mention the keyring backend? (Our default is "os".)
We've done transactions using the binary, so I'm wondering what the issue could be here. Whether it's just the flag failing, or can't interact with "os" keyring at all.

Also follow-up question: is this an Ubuntu server with a GUI/window environment installed?

Yes Ubuntu desktop (with a GUI)

@andynog Thanks for clarifying that. So I understand that when using GUI environments, the os backend can fail because there might be multiple targets for it to use, e.g., both seahorse as well as kwallet might exist on the same system. Are you actually running this as a node/server/validator or just for local development purposes?
Your point on this working with gaiad is interesting. Will look into why that's the case...

Closing this as likely an OS issue


Add missing dependency on NETStandard.Library for .netstandard target

Also remove unnecessary dependency on EventSource.Register
(Based on a review from @eerhardt)

@pharring,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

we still need EventRegister, do we? This package validates that EventSource is authored correctly. Why removing it?

We need only the Redist piece of EventSource. We do not need EventRegister. It's a tool that is used to create the EventSource class definitions. It applies only to the .Net 4.0 project (it's included in newer frameworks without requiring a separate package). Core.csproj builds fine without it.
Also, it's not in the .nuspec, so customers were not already getting a dependency.

@pharring - EventRegister helps detecting the issues when I do "Write" with the different ID than defined in method attributes. Are you saying it is now integrated into the framework? See this discussion for instance: https://github.com/Microsoft/ApplicationInsights-SDK-Labs/issues/99

Ah, I think you're right. EventRegister also works for System.Diagnostics, so it's not limited to net40.
Looks like we can add it back for all frameworks, but it should not be in the nuspec since it's just used during the build. Stand by, I'll update this PR.

@SergeyKanzhelev fixed.


✨ Add support for data declarations and pattern matching on them.

Even though we don't currently have type checking, it would be useful to support ML-style ADTs.

Declarations should look like:

export enum maybe
  = just _
  | nothing

This can compile to something like:
export function just (a) {
    return { $: 'just', 0: a }
}

export var nothing = { $: 'nothing' }

Imports should look something like:

import 'ren/maybe' exposing { maybe(..) }
import 'ren/maybe' exposing { maybe(just, nothing) }

Pattern matching should look something like:

when val
  is just a  => a
  is nothing => 0

And compile to:
(($) => {
    if (typeof $ == 'object' && $.$ == 'just' && 0 in $) {
        var a = $[0]
        return a
    }

    if (typeof $ == 'object' && $.$ == 'nothing') {
        return 0
    }
})(val)

Some additional thoughts

Because we don't have pattern matching it's possible for one pattern to match against multiple ADTs that share the same tag (aka ADTs are structurally typed the same as objects/records are).
Because variants are exported as functions/variables, we should check to make sure a module doesn't also define a function with the same name as a variant.
We should try and check for exhaustiveness if we know where the variant is coming from.

Closed by #65!


[Windows] naive (incorrect) conversion of byte slice to Go string

First, thanks for this package!
There is a bug on Windows.
This line converts a byte slice to a Go string. This is fine if the byte slice is utf8, however ~Windows likes to store strings as utf16~ it seems like the content of this blob is application defined. Nonetheless when the content is utf16, the result of the cast is an invalid Go string.
The fix would be to validate the byte slice and convert it to utf8 prior to casting it into a Go string.
Edit: I've looked into this somewhat and have realized there's no easy way to do this conversion in general, without knowing the encoding ahead of time.

Thanks for the issue!
Do you want to create a pr to fix it?

@JackMordaunt How do you create those secrets that are wrongly encoded from go-keyring perspective? are they created via go-keyring or in another way?

@JackMordaunt How do you create those secrets that are wrongly encoded from go-keyring perspective? are they created via go-keyring or in another way?

I'm not sure how this came about, as it was a reported by someone else without a reproducer.
I suspect it was a third-party application that they used to write the initial value (perhaps Windows Credential Manager writes utf16 under certain configurations?)


feat: adding ability to dump default configuration

This change is

Codecov Report
Attention: Patch coverage is 0% with 5 lines in your changes are missing coverage. Please review.

Project coverage is 45.81%. Comparing base (eed72fe) to head (b14dad5).

Files
Patch %
Lines

crates/mempool_node/src/bin/dump_config.rs
0.00%
5 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #128      +/-   ##
==========================================
- Coverage   46.20%   45.81%   -0.40%
==========================================
  Files          17       18       +1
  Lines         580      585       +5
  Branches      580      585       +5
==========================================
  Hits          268      268
- Misses        303      308       +5
  Partials        9        9

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Actual Sprite pixel dimensions not being preserved

The sprite dicing algorithm is removing any white space, making fixed-size sprites (32x32, 64x64, etc.) coming out smaller than the really are, which is messing up alignment. For example, if I have a small graphic that animates by moving around the corners of a 32x32 square, the animations just all get cropped to the actual sprite, so it no longer moves around. Does that make sense how I described it? I feel like the dicing algorithm should take into account where it trims the white space and adjusts the pivot point to compensate.

Hey, Sorry for the delay. You can handle such cases by enabling "Keep Original" for pivot when generating the atlas. There is an example for diced sprite animation in the project. When the option is enabled, movement is handled correctly.


Cannot find module './data/lahore.json'

When I tried to run the CLI, I got this error.

Getting the same issue

🤦♂ Dang man.
Should be fine now.
🔗 https://github.com/ahmadawais/ramadan-cli/releases/tag/1.2.0
🔗 https://www.npmjs.com/package/ramadan-cli


Ignore .ionide in VisualStudio gitignore

Reasons for making this change:
Recent version of the Ionide F# extension for VSCode use a .ionide folder similar to how Visual Studio itself uses .vs.
There is work in progress in the extension to suggest to add it to gitignore (https://github.com/ionide/ionide-vscode-fsharp/issues/1159) but it would be nice if the Visual Studio template used by all .Net languages did it by default
Links to documentation supporting these rule changes:
The issue https://github.com/ionide/ionide-vscode-fsharp/issues/1159 contains some details about it, I don't think there is more specific documentation

Duplicate of https://github.com/github/gitignore/pull/3108


ReferenceError: Can't find variable: $

Check full details on Airbrake: #69169722
Backtrace:
/comment?back=%2Fdatasets%2F351%2Fcertificates%2F89&title=The documentation includes machine-readable data for spatial%2Fgeographical coverage &topic=certificates%2F89%2FdocumentationMetadata:42 in "{anonymous}()"

I believe this has been resolved by fixing the Juvia hosting.


Include java version in pom.xml

When I did "mvn clean install" at the root folder, it asked me to run the program with JDK even if it is installed in my machine. Mention JDK version in pom.xml to avoid build issues

@ravikalla You can set your JAVA_HOME to your desired jdk. People can choose to have different jdk minor versions. The pom specifies the maven-compiler-plugin.