Deploy to the Cloud guides (Kubernetes/Docker and Cloud Foundry)

These replace the current docker and cloud deploy guides.

@alexwishart The issue is the guide didn't seem to work for me, I didn't get an external ip for the last step for example.
I'm not sure if it's an issue with the guide directly, but the guide is hard to follow if you're new to Kubernetes, Helm or cloud providers.
There's a lot of places I was expected to navigate away from the guide and have to learn how various other technologies work together before I could return to the guide to attempt to complete it.
I mentioned this briefly before, but if we're suggesting Docker Hub then maybe we should include instructions for that.

Just to clear up my last comment:

Can you follow this guide with a kitura init project and it works? For me there's a few extra steps.

Would be nice to include a section about pushing to the registry. This is such an important part of the process I feel an example should be included. Even if we just pick one, and let users discover the rest for themselves.

A nice to have would be to explain what the commands are actually doing, but not essential at this point.

Hopefully this clears up the points I was making in my previous comment :)

@ddunn2 My idea about the guide was that it was going to have to make some assumptions. The only way we can make sure we don't pidgeonhole people into using IBM Cloud (which is something @ianpartridge was keen on) is to keep the guide a bit vague. I agree, it makes things like pushing to the registry or finding the external ip for the kube service difficult, but the guide is about deploying the kitura init app and I think going too in depth into setting up a registry or your kube cluster could distract from that.
I've been able to run through the guide from a kitura init project.. I've taken the helm init command to come under the assumption that you already have a kube cluster set up, maybe this is something that should be expressed more explicitly at the top of the guide? :)
Overall it was a pretty difficult guide to bring together because of the vagueness so I appreciate any comments. In my opinion the best way forward would be to create/link to additional guides for the instructions you feel are missing?

I feel a guide should literally be guiding a user through doing something step by step, if we're only providing half of the instructions and assuming people should learn the rest they may lose interest pretty quickly, it can be a time consuming process to pull together all the different commands you need to actually complete this guide especially if you're new to it all.
If we're explicit in the guide that you can use whatever cloud service you'd like and then mention that in this guide we'll be using IBM cloud as an example? I understand we don't want to pigeonhole people but if we're going to provide instructions it should be around IBM Cloud :)
Vague guides are difficult to write, and difficult to follow. Personally I'd prefer the guide to be explicit, a one stop place for anyone wanting to deploy an app using Kubernetes, whether they're wanting to use IBM Cloud or not.
Maybe something like:
If you already have a deployment environment set up, you can skip to the deploy section.
If not then the following instructions will take you through how to setup an environment using IBM Cloud...

If you do feel that content specifically around Kitura and IBM Cloud is necessary but not appropriate for Kitura.io, we can put that into the Swift programming guide, and you could link to that.
@jpkessle and christx83 have joined our team and have been hard at work on making programming guides. Just let us know!

Thanks @christiancompton
If there is a nice succinct guide we could link to around setting up an IBM Cloud environment for Kube that'd be useful.
In the meantime we could just include the 'high level' commands here, that is the bare minimum for users to be able to complete this guide starting from nothing.
That is:
Link to CLI installation/account creation (or just provide the commands)
Create a cluster
Create a namespace
Show how to get the public/external ip of cluster
With those four things anyone could complete this guide :)

Just going to merge this for now, we can update this at a later date :)


Support Windows Process Groups

To use more than 64 threads on Windows, you have to use Process Groups:
https://docs.microsoft.com/en-us/windows/win32/procthread/processor-groups?redirectedfrom=MSDN
Each spawned thread must be assigned to a process group. I don't know if it makes sense to handle this in rayon or libstd or what.

Do you know whether this is reflected in num_cpus?
If that will be limited to the 64-thread ceiling already, then I think we're in okay shape by default. We wouldn't want to oversubscribe too many rayon threads into a 64-cpu process group, but I think it's fine if we're limited to that and just don't use additional cpus.
Beyond that, I think it's in the realm of advanced tweaking that the user could deal with in ThreadPoolBuilder::spawn_handler.
See also #319 for general NUMA awareness.

Looking at the source code of num_cpus, I believe it will max out at 64. You'd need to use GetActiveProcessorCount to get a higher number. So it will work fine as is, just underutilize.
Note that this is technically unrelated to NUMA - in particular the just-released 3990WX has 128 threads but only one NUMA node.

the numa-related parts of chapter 4 in https://developer.amd.com/wp-content/resources/56782_1.0.pdf are relevant:

Since all the processors in a single-socket/128-logical-processor NUMA node cannot fit completely within a single Windows Processor Group, Windows creates a (virtual) secondary node to hold the additional processors.

Regardless of NPS settings, applications will need to be multi-group aware to take advantage of all the processors (otherwise their affinity will be to a single processor group).

i.e. the "NUMA node"/"processor group" Windows terminology is becoming blurred, as it is imposing limitations which don't reflect the hardware or the configuration of the hardware...

...this means your code needs to manually move threads to other groups via something like
GROUP_AFFINITY affinity{}, affinity_prev{};
affinity.Group = id / 64;
affinity.Mask = 1ull << (id % 64);
SetThreadGroupAffinity(GetCurrentThread(), &affinity, &affinity_prev);

where the important part is .Group. AFAIK there is no "group mask" field to tell the scheduler to schedule a thread across a set of processor groups(?)
Meaning, just creating 128 threads would not be enough.


[nrf noup] drivers: flash: Allow reading secure mem by nrf_rram.

To read the secure memory we need to use soc_secure_mem_read function instead of memcpy.
This commit allows to use the soc_secure_mem_read function if the CONFIG_TRUSTED_EXECUTION_NONSECURE and PARTITION_MANAGER is enabled.
ref.: NCSDK-29300

There was an issue with Radio Driver, rebasing.

rebased.


Add README that describes the libfuzzer integration.

Problem
No docs.
Change overview
Add some.
Testing
Docs.

Merging this in, need to fix ZAP upstream, but docs don't update it.


I want to add comments to all the files

comments are very necessary
the make us understand the codes easily
there are very less fines in your repo which have comments in them,
comments have too much importance in coding tests of many companies,
thats why I have choosen this issue.

This is under crosswoc
i dont know how to add labels on issues

@kmryashasvi Add comments to existing code in this repository .

i can add comments in any language, please assign this task to me

My comments are also for any of the languages
i am adding comments to all the files irrespective to any language or platform

@kmryashasvi Have you commented all the code ?

Yes,
but if you agree,
I want to let this issue open so that, i can add comments to all future submissions,
even after the announcement of results of crosswoc

yes,
I have added comments to all the present files,
you can close this issue if you want

Yes,
but if you agree,
I want to let this issue open so that, i can add comments to all future submissions,
even after the announcement of results of crosswoc

You will later open this.


Color.withValues(alpha: x) expects values between 0...1 instead of 0...255

Steps to reproduce

Run the provided sample on iOS 18.1
Observe that colors are fully opaque if alpha is bigger than 1 when created with Color.withValues(alpha: 1.01)

My expectation would be that the documentation of Color class is correct -- you should be able to provide alpha with values between 0 and 255 or the docs for withValues should be more explicit about accepted values of alpha.
  /// The alpha channel of this color in an 8 bit value.
  ///
  /// A value of 0 means this color is fully transparent. A value of 255 means
  /// this color is fully opaque.
  @Deprecated('Use .a.')
  int get alpha => (0xff000000 & value) >> 24;

  /// The alpha channel of this color as a double.
  ///
  /// A value of 0.0 means this color is fully transparent. A value of 1.0 means
  /// this color is fully opaque.
  @Deprecated('Use .a.')
  double get opacity => alpha / 0xFF;

Expected results
Alpha values between 0 and 255 should result with semi-opaque colors
Actual results
Alpha of 1 or more is fully opaque
Code sample
Code sample
import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
        useMaterial3: true,
      ),
      home: const MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({super.key, required this.title});

  final String title;

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        backgroundColor: Theme.of(context).colorScheme.inversePrimary,
        title: Text(widget.title),
      ),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            for (var i = 0; i < 10; i++)
              Container(
                color: Color(0xFF000000).withValues(alpha: 255.0 * i / 10),
                width: 200,
                height: 20,
                child: Text(
                  '$i, alpha:${255.0 * i / 10}',
                  style: TextStyle(color: Colors.red),
                ),
              )
          ],
        ),
      ),
    );
  }
}

Screenshots or Video

Screenshots / Video demonstration

Current behavior
Expected behavior

Logs
Logs
Not needed

Flutter Doctor output
Doctor output
[✓] Flutter (Channel stable, 3.27.1, on macOS 15.1.1 24B91 darwin-arm64, locale pl-PL)
    • Flutter version 3.27.1 on channel stable at /Users/dominik/fvm/versions/stable
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 17025dd882 (3 days ago), 2024-12-17 03:23:09 +0900
    • Engine revision cb4b5fff73
    • Dart version 3.6.0
    • DevTools version 2.40.2

[!] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/dominik/Library/Android/sdk
    ✗ cmdline-tools component is missing
      Run `path/to/sdkmanager --install "cmdline-tools;latest"`
      See https://developer.android.com/studio/command-line for more details.
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/to/macos-android-setup for more details.

[✓] Xcode - develop for iOS and macOS (Xcode 16.2)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 16C5032a
    • CocoaPods version 1.16.2

[✓] Android Studio (version 2024.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.11+0-17.0.11b1207.24-11852314)

[✓] VS Code (version 1.96.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.102.0

[✓] Connected device (3 available)
    • iPhone 12 mini (mobile)         • 00008101-0001510C3CF0001E • ios          • iOS 18.1.1 22B91
    • macOS (desktop)                 • macos                     • darwin-arm64 • macOS 15.1.1 24B91 darwin-arm64
    • Mac Designed for iPad (desktop) • mac-designed-for-ipad     • darwin       • macOS 15.1.1 24B91 darwin-arm64

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.```

</details>

Hello @orestesgaolin, Thanks for filing the issue. I was able to repoduce the issue on the latest stable 3.27.1  and the master 3.28.

Pizel 7 (Android)
Iphone 16 pro (IOS)

Web shows some shades but it is still inconsistent and inaccurate
Web

flutter doctor -v
[✓] Flutter (Channel stable, 3.27.1, on macOS 15.1.1 24B2091 darwin-arm64, locale en-US)
    • Flutter version 3.27.1 on channel stable at /Users/mahesh/Development/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 17025dd882 (2 days ago), 2024-12-17 03:23:09 +0900
    • Engine revision cb4b5fff73
    • Dart version 3.6.0
    • DevTools version 2.40.2

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/mahesh/Library/Android/sdk
    • Platform android-35, build-tools 34.0.0
    • ANDROID_HOME = /Users/mahesh/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.11+0-17.0.11b1207.24-11852314)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 16.0)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 16A242d
    • CocoaPods version 1.16.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2024.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.11+0-17.0.11b1207.24-11852314)

[✓] IntelliJ IDEA Community Edition (version 2021.2.1)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 61.2.4
    • Dart plugin version 212.5080.8

[✓] VS Code (version 1.94.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.100.0

[✓] Connected device (3 available)
    • macOS (desktop)                 • macos                 • darwin-arm64   • macOS 15.1.1 24B2091 darwin-arm64
    • Mac Designed for iPad (desktop) • mac-designed-for-ipad • darwin         • macOS 15.1.1 24B2091 darwin-arm64
    • Chrome (web)                    • chrome                • web-javascript • Google Chrome 128.0.6613.85

[✓] Network resources
    • All expected network resources are available.

• No issues found!

[!] Flutter (Channel master, 3.28.0-1.0.pre.123, on macOS 15.1.1 24B2091 darwin-arm64, locale en-US)
    • Flutter version 3.28.0-1.0.pre.123 on channel master at /Users/mahesh/Development/flutter_master
    • Framework revision a547c7f050 (4 days ago), 2024-12-13 22:11:39 -0500
    • Engine revision 4e4a430bcd
    • Dart version 3.7.0 (build 3.7.0-243.0.dev)
    • DevTools version 2.41.0
    • If those were intentional, you can disregard the above warnings; however it is recommended to use
      "git" directly to perform update checks and upgrades.

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/mahesh/Library/Android/sdk
    • Platform android-35, build-tools 34.0.0
    • ANDROID_HOME = /Users/mahesh/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
      This is the JDK bundled with the latest Android Studio installation on this machine.
      To manually set the JDK path, use: `flutter config --jdk-dir="path/to/jdk"`.
    • Java version OpenJDK Runtime Environment (build 17.0.11+0-17.0.11b1207.24-11852314)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 16.0)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 16A242d
    • CocoaPods version 1.16.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2024.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.11+0-17.0.11b1207.24-11852314)

[✓] IntelliJ IDEA Community Edition (version 2021.2.1)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 61.2.4
    • Dart plugin version 212.5080.8

[✓] VS Code (version 1.94.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.100.0

[✓] Connected device (3 available)
    • macOS (desktop)                 • macos                 • darwin-arm64   • macOS 15.1.1 24B2091
      darwin-arm64
    • Mac Designed for iPad (desktop) • mac-designed-for-ipad • darwin         • macOS 15.1.1 24B2091
      darwin-arm64
    • Chrome (web)                    • chrome                • web-javascript • Google Chrome
      128.0.6613.85

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.

I think, the error (resp. misunderstanding) is, that withValues expects doubles values between 0 and 1 and not integer values between 0 and 255. This isn't clearly documented, though.

I agree the documentation could be more clear.


Graphite scaler is returning the earliest datapoint of a query, not the latest.

Report
No response
Expected Behavior
Graphite's scaler acts on the most recent data available from a given query.
Actual Behavior
The scaler will act on the most outdated value returned by a query for a given time range.
Steps to Reproduce the Problem

Define a scaler definition with a QueryTime of 1 hour
Observe scaling behaviours act on hour-old data.

Logs from KEDA operator
example

KEDA Version
2.5.0
Kubernetes Version
No response
Platform
No response
Scaler Details
No response
Anything else?
Here you can see the graphite server response returns values in order of oldest to latest. With the final element of the response array being the most up-to-date timestamp. Graphite's scaler should be acting on the final element of the response array, not the first.

https://graphite-api.readthedocs.io/en/latest/api.html#json
https://github.com/kedacore/keda/blob/main/pkg/scalers/graphite_scaler.go#L197

No response

@bpinske do you think, that there might be some usecase where keeping the current behavior (oldest) makes sense?

It seems very unlikely to me there is any legitimate use in allowing people to intentionally use out of date data.
Prometheus's scaler does not have any equivalent back-in-time functionality either.


Khan academy backend user_id is required to use any further requests

Without saving the user_id we can't do any further requests.

Thanks!


Just added a Question for #4 since it seemed to contain an answer to a question that was not stated.

Description of changes:
Added Question to #4 since there was an answer but no question

Thanks for catching this issue, actually the question was "Does this tool integrate with GuardDuty, Security Hub, CloudWatch, etc.?" so I'll make that update.


Castle.Windsor.Extensions.DependencyInjection No Scope Available

I am using this nuget package to configure Castle Windsor as the DI container. https://www.nuget.org/packages/Castle.Windsor.Extensions.DependencyInjection/
This works fine except for when I try to resolve scoped classes with an IHttpHandler. I get the error: InvalidOperationException: No scope available
What is the correct way to create scope for an HttpHandler?
Here's a sample application reproducing the issue:

namespace WebApplication1
{
    public class Global : System.Web.HttpApplication
    {
        public static IWindsorContainer Container;

        protected void Application_Start(object sender, EventArgs e)
        {
            Console.WriteLine("Hello World!");

            Container = new WindsorContainer();
            var host = new HostBuilder()
                .UseWindsorContainerServiceProvider(Container)
                .ConfigureServices((hostContext, services) =>
                {
                    services.AddScoped<SomeService>();
                })
                .Build();

            //Note: This works great
            var resolved = Container.Resolve<SomeService>();
        }

        protected void Session_Start(object sender, EventArgs e)
        {

        }

        protected void Application_BeginRequest(object sender, EventArgs e)
        {

        }

        protected void Application_AuthenticateRequest(object sender, EventArgs e)
        {

        }

        protected void Application_Error(object sender, EventArgs e)
        {

        }

        protected void Session_End(object sender, EventArgs e)
        {

        }

        protected void Application_End(object sender, EventArgs e)
        {

        }
    }

    public class SomeService
    {

    }
    public class HttpHandler : IHttpHandler
    {
        public void ProcessRequest(HttpContext context)
        {
            using (var scope = Global.Container.BeginScope())
            {
                //this errors
                var someService = Global.Container.Resolve<SomeService>();
            }
        }

        public bool IsReusable => false;
    }

}

Web.config:
<system.webServer>
		<handlers>
			<add verb="*" path="*" type="WebApplication1.HttpHandler, WebApplication1, Version=<IP_ADDRESS>, Culture=neutral" name="default" />
		</handlers>
	</system.webServer>

I was able to come up with a work around, but I think it's a bit of a hack and I'm unsure how it will effect releasing scoped resources.
namespace WebApplication1
{
    public class Global : System.Web.HttpApplication
    {
        public static WindsorServiceProviderFactoryCustom ServiceProviderFactory;

        protected void Application_Start(object sender, EventArgs e)
        {
            Console.WriteLine("Hello World!");

            var container = new WindsorContainer();

            ServiceProviderFactory = new WindsorServiceProviderFactoryCustom(container);

            var host = new HostBuilder()
                .UseWindsorContainerServiceProvider(ServiceProviderFactory)
                .ConfigureServices((hostContext, services) =>
                {
                    services.AddScoped<SomeService>();
                })
                .Build();

            //Note: This works great
            var resolved = ServiceProviderFactory.Container.Resolve<SomeService>();
        }

        protected void Session_Start(object sender, EventArgs e)
        {

        }

        protected void Application_BeginRequest(object sender, EventArgs e)
        {

        }

        protected void Application_AuthenticateRequest(object sender, EventArgs e)
        {

        }

        protected void Application_Error(object sender, EventArgs e)
        {

        }

        protected void Session_End(object sender, EventArgs e)
        {

        }

        protected void Application_End(object sender, EventArgs e)
        {

        }
    }

    public class SomeService
    {
    }

    public class HttpHandler : IHttpHandler
    {
        public void ProcessRequest(HttpContext context)
        {
            //This fixes the error, but I am too unfamiliar with the internals of this library to understand if this is a reasonable workaround. Will the component be scoped to `scope` below?
            Global.ServiceProviderFactory.CreateRootScope();

            using (var scope = Global.ServiceProviderFactory.Container.BeginScope())
            {
                //this no longer errors, but at what cost.
                var someService = Global.ServiceProviderFactory.Container.Resolve<SomeService>();
            }
        }

        public bool IsReusable => false;
    }

    public sealed class WindsorServiceProviderFactoryCustom : WindsorServiceProviderFactoryBase
    {
        public WindsorServiceProviderFactoryCustom(IWindsorContainer container)
        {
            CreateRootScope();
            SetRootContainer(container);
        }

        public new void CreateRootScope()
        {
            base.CreateRootScope();
        }
    }

}

I think this has been fixed as CreateRootScope is called in WindsorServiceProviderFactory constructor.
I think the cleanest way to it though is:

make host static variable
access services via host.Services

using (var scope = host.Services.CreateScope())
{
    scope.GetService<SomeService>();
}

I have this issue with the latest versions of Castle.Windsor and Castle.Windsor.Extensions.Hosting.
I call this code on my ASP.NET server:
var builder = WebApplication.CreateBuilder();
var container = new WindsorContainer();
builder.Host.UseWindsorContainerServiceProvider(container);

And when running my server (with gRPC services, and a Yarp reverse proxy configured), I get this error:
An unhandled exception has occurred while executing the request.
      System.InvalidOperationException: No scope available
         at Castle.Windsor.Extensions.DependencyInjection.Scope.ExtensionContainerScopeCache.get_Current()
         at Castle.Windsor.Extensions.DependencyInjection.Scope.ForcedScope..ctor(ExtensionContainerScopeBase scope)
         at Castle.Windsor.Extensions.DependencyInjection.WindsorScopedServiceProvider.GetRequiredService(Type serviceType)
         at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
         at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
         at Microsoft.AspNetCore.Routing.Matching.DfaMatcherFactory.CreateMatcher(EndpointDataSource dataSource)
         at Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware.InitializeCoreAsync()
      --- End of stack trace from previous location ---

I have checked the method CreateRootScope is indeed called in the underlying WindsorServiceProviderFactory.
Any idea? 😄


Release 100.14.0

Description
This PR merges v.next into main for 100.14.0 release
Linked Issue(s)

common-samples/issues/3485

Release notes since <IP_ADDRESS>, #1147
New samples:

Query related features from non-spatial table

Enhancement:

Updated various samples to utilize applyEdits method on AGSServiceGeodatabase.
Updated all scene samples to use basemapStyle API.

Bug fixes:

Updated ordered anchor points for data used in Dictionary renderer with graphics overlay.
Fixed data tags for Display feature layers.
Fixed missing labels in Show labels on layer in 3D.
Fixed incorrect basemap in List KML contents.
Updated mobile map package workflow in Apply scheduled updates to preplanned map area.
Updated data in Raster layer (service).

This is good to go.


Where is 'processed_annotations' for Million Song Dataset?

Hello, thank you very much for your wonderful GitHub repository. 😊
I tried to train the code in the repository using the Million Song Dataset, but it prompted me with a FileNotFoundError: [Errno 2] No such file or directory: ‘data/million_song_dataset/processed_annotations/output_labels_msd.txt’. 😢
Where can I download the files in the processed_annotations folder such as ‘train_gt_msd.tsv’, ‘output_labels_msd.txt’, ‘index_msd.tsv’, etc.?
I couldn’t find any clues to download these files on the official website of the Million Song Dataset, and the MSD dataset I downloaded did not contain these files.😭
Thank you for your time and assistance. 😊

@Spijkervet

@pvdstel @isaaccorley


Apple Silicon M2 Loss Goes Up After 1-2 (Few) Epochs but not on Google Colab

System information.

Have I written custom code (as opposed to using a stock example script provided in Keras): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Mac OS Ventura 13.5.1
TensorFlow installed from (source or binary): Source
TensorFlow version (use command below): 2.12.0 (Google Colab), 2.12.1 (Mac)
Python version: 3.10.12 (Google Colab), 3.11.4 (Mac)
GPU model and memory: T4 12GB RAM (Google Colab), Apple M2 Max (Mac)

Describe the problem.
I run this code on Mac (Apple Silicon M2 Max)
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Flatten

# Load the data
(x_train, y_train), _ = tf.keras.datasets.mnist.load_data()

# Normalize pixel values
x_train = x_train / 255.0

model = Sequential([
    Flatten(input_shape=(28, 28)),
    Dense(512, activation='relu'),
    Dense(10, activation='softmax')
])

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

print("TensorFlow version: ", tf.__version__)

model.fit(x_train, y_train,
          epochs=10)

Loss immediately starts increasing after 1 epoch.
/Users/felixm/anaconda3/envs/tf230808/bin/python /Users/felixm/PycharmProjects/TFExamStudy/Coursera/2309_scratch.py
TensorFlow version:  2.12.1
2023-09-03 16:53:30.575673: W tensorflow/tsl/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
Epoch 1/10
1875/1875 [==============================] - 7s 3ms/step - loss: 0.3915 - accuracy: 0.8903
Epoch 2/10
1875/1875 [==============================] - 7s 4ms/step - loss: 0.4136 - accuracy: 0.8952
Epoch 3/10
1875/1875 [==============================] - 6s 3ms/step - loss: 0.4905 - accuracy: 0.8897
Epoch 4/10
1875/1875 [==============================] - 6s 3ms/step - loss: 0.5585 - accuracy: 0.8878
Epoch 5/10
1875/1875 [==============================] - 6s 3ms/step - loss: 0.6251 - accuracy: 0.8857
Epoch 6/10
1875/1875 [==============================] - 6s 3ms/step - loss: 0.7148 - accuracy: 0.8848
Epoch 7/10
1875/1875 [==============================] - 6s 3ms/step - loss: 0.7854 - accuracy: 0.8822
Epoch 8/10
1875/1875 [==============================] - 6s 3ms/step - loss: 0.8779 - accuracy: 0.8816
Epoch 9/10
1875/1875 [==============================] - 6s 3ms/step - loss: 0.9782 - accuracy: 0.8821
Epoch 10/10
1875/1875 [==============================] - 6s 3ms/step - loss: 1.0523 - accuracy: 0.8798

Process finished with exit code 0

Running the same code on Google Colab, the loss never increases.

Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
11490434/11490434 [==============================] - 0s 0us/step
TensorFlow version:  2.12.0
Epoch 1/10
1875/1875 [==============================] - 11s 3ms/step - loss: 0.2004 - accuracy: 0.9412
Epoch 2/10
1875/1875 [==============================] - 5s 3ms/step - loss: 0.0810 - accuracy: 0.9750
Epoch 3/10
1875/1875 [==============================] - 6s 3ms/step - loss: 0.0533 - accuracy: 0.9833
Epoch 4/10
1875/1875 [==============================] - 5s 3ms/step - loss: 0.0363 - accuracy: 0.9883
Epoch 5/10
1875/1875 [==============================] - 6s 3ms/step - loss: 0.0274 - accuracy: 0.9911
Epoch 6/10
1875/1875 [==============================] - 5s 3ms/step - loss: 0.0204 - accuracy: 0.9937
Epoch 7/10
1875/1875 [==============================] - 6s 3ms/step - loss: 0.0171 - accuracy: 0.9944
Epoch 8/10
1875/1875 [==============================] - 5s 3ms/step - loss: 0.0131 - accuracy: 0.9955
Epoch 9/10
1875/1875 [==============================] - 5s 3ms/step - loss: 0.0123 - accuracy: 0.9961
Epoch 10/10
1875/1875 [==============================] - 6s 3ms/step - loss: 0.0114 - accuracy: 0.9960
<keras.callbacks.History at 0x780f114f2e60>

I tried with<EMAIL_ADDRESS>problem still occurs.

I have the same problem. Just installed the latest TF version ('2.15.0-rc1') for my M2 Mac and my MNIST accuracy on a generic "Hello World" model architecture is significantly lower than in all the examples with exactly the same architecture.
To test it I just copied the code from a coursera course:
`# Import keras
from tensorflow import keras
Download the dataset and split into train and test sets
(img_train, label_train), (img_test, label_test) = keras.datasets.fashion_mnist.load_data()
Normalize pixel values between 0 and 1
img_train = img_train.astype('float32') / 255.0
img_test = img_test.astype('float32') / 255.0
Build the baseline model using the Sequential API
b_model = keras.Sequential()
b_model.add(keras.layers.Flatten(input_shape=(28, 28)))
b_model.add(keras.layers.Dense(units=512, activation='relu', name='dense_1')) # You will tune this layer later
b_model.add(keras.layers.Dropout(0.2))
b_model.add(keras.layers.Dense(10, activation='softmax'))
Print model summary
b_model.summary()
Setup the training parameters
b_model.compile(optimizer=keras.optimizers.Adam(learning_rate=0.001),
loss=keras.losses.SparseCategoricalCrossentropy(),
metrics=['accuracy'])
Number of training epochs.
NUM_EPOCHS = 10
Train the model
b_model.fit(img_train, label_train, epochs=NUM_EPOCHS, validation_split=0.2)
`
My val_accuracy on the last (10th) epoch is 0.8121
Same code in Google Colab yields 0.8798

I believe I had found a solution: change activation function from relu to tanh does the trick.

I have the similar issue. tldr: seems the root cause is somewhere in tensorflow-metal.
I'm trying to run book examples.

Distilled code version:
import tensorflow as tf

fashion_mnist = tf.keras.datasets.fashion_mnist.load_data()
(X_train_full, y_train_full), (X_test, y_test) = fashion_mnist
X_train, y_train = X_train_full[:-5000], y_train_full[:-5000]
X_valid, y_valid = X_train_full[-5000:], y_train_full[-5000:]

X_train, X_valid, X_test = X_train / 255., X_valid / 255., X_test / 255.

tf.random.set_seed(42)

model = tf.keras.Sequential([
    tf.keras.layers.Flatten(input_shape=[28, 28]),
    tf.keras.layers.Dense(300, activation=tf.keras.activations.relu),
    tf.keras.layers.Dense(100, activation=tf.keras.activations.relu),
    tf.keras.layers.Dense(10, activation=tf.keras.activations.softmax),
])
model.summary()

model.compile(loss='sparse_categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])

history = model.fit(X_train, y_train, epochs=30, validation_data=(X_valid, y_valid))

After some # of epochs I get accuracy dropped and loss exploded (you can check full logs attached):
Epoch 15/30
1719/1719 [==============================] - 8s 5ms/step - loss: 0.9868 - accuracy: 0.7996 - val_loss: 1.0065 - val_accuracy: 0.8128
Epoch 16/30
1719/1719 [==============================] - 8s 5ms/step - loss: 1.1898 - accuracy: 0.7952 - val_loss: 2.3624 - val_accuracy: 0.7086
Epoch 17/30
1719/1719 [==============================] - 8s 5ms/step - loss: 1.6663 - accuracy: 0.7837 - val_loss: 2.4961 - val_accuracy: 0.7822
Epoch 18/30
1719/1719 [==============================] - 8s 5ms/step - loss: 2.4820 - accuracy: 0.7758 - val_loss: 4.9756 - val_accuracy: 0.6454
Epoch 19/30
1719/1719 [==============================] - 9s 5ms/step - loss: 4.7354 - accuracy: 0.7551 - val_loss: 10.3181 - val_accuracy: 0.6346
Epoch 20/30
1719/1719 [==============================] - 8s 5ms/step - loss: 11.1815 - accuracy: 0.7318 - val_loss: 10.7277 - val_accuracy: 0.7696
Epoch 21/30
1719/1719 [==============================] - 8s 5ms/step - loss: 36.0803 - accuracy: 0.7089 - val_loss: 49.7303 - val_accuracy: 0.7086
Epoch 22/30
1719/1719 [==============================] - 8s 5ms/step - loss: 165.2742 - accuracy: 0.6973 - val_loss: 651.5690 - val_accuracy: 0.5884
Epoch 23/30
1719/1719 [==============================] - 8s 5ms/step - loss: nan - accuracy: 0.6632 - val_loss: nan - val_accuracy: 0.1042
Epoch 24/30
1719/1719 [==============================] - 8s 5ms/step - loss: nan - accuracy: 0.0996 - val_loss: nan - val_accuracy: 0.1042

I use MacBook Air with Apple M2 running Sonoma 14.1.1.
$ python --version
Python 3.11.6

Tensorflow packages installed (tried the same with 2.14.0):
$ pip list | grep tensorflow
tensorflow                   2.15.0
tensorflow-estimator         2.15.0
tensorflow-io-gcs-filesystem 0.34.0
tensorflow-macos             2.15.0
tensorflow-metal             1.1.0

I found that above example works fine if I uninstall tensorflow-metal and move all the workloads to cpu. My activity monitor with metal plugin:

and without metal plugin:

Also, in this case running time for cpu-only is even better:
Epoch 8/30
1719/1719 [==============================] - 1s 673us/step - loss: 0.3569 - accuracy: 0.8745 - val_loss: 0.3749 - val_accuracy: 0.8616

But this doesn't work for bigger (different) workloads, tested on Apple's example and get much worse performance without metal support.
Also, I've tired:

Changing relu to tanh - kinda works in my case.
Cast X_train to float32 (to match with model params type) - doesn't help.

tf-issue-with-metal.txt
tf-issue-without-metal.txt

I had a similar problem, and it was also solved by uninstalling tensorflow-metal

I believe I had found a solution: change the hidden layer's activation function from relu to tanh does the trick.

Also softplus activation function does not seem to suffer from this problem.

Why is this issue closed?? It is still present on Tensorflow 2.15.

same issue with apple M4. when using CPU,  when accuracy is correct.
hw = tf.config.get_visible_devices()
tf.config.set_visible_devices(hw[0])

when using gpu, the loss getting increased with acc down.


Add minimal persistence classes

This is a first minimalistic Persistence implementation. It uses a Map internally with a "context" key structure and allows to be stored a JSON and one implementation saves the data using Nodejs FS, but all splitted up so that a Browser implementation could "just" store the JSON differently.
The storage also preserves bigint, Buffer and Uint8Arrays and restores them correctly.

Thanks for writing this!
The APIs to use it seems a little complex though.
I suggest to reorganize the methods to make it simple to use in the code, like this:
const persistentStore = Persistence.get().getStore("my_store");
function myFunction() {
const myPersistentValue = persistentStore.get("myValue");
myPersistentValue++;
persistentStore.set(myPersistentValue);
}
Somewhere this method should be called:
await Persistence.get().init();
Persistence.init(): loads all the persistence in memory (async)
Persistence.getStore(key: string): returns a store to isolate data from one context to another and avoid data name collision
Store.get(): retrieves the data from the memory
Store.set(value: T): sets the data in memory and triggers a background task to persist them
WDYT?

Hi, i agree that to "use" the storage a singleton approach is more easy and I can add this easiely.
Internally I would like to keep such a structure because I want to be flexible with how data is actually really stored - like SQLite, redis, jsonl-db and such and all would be different. Alone in iobroker as example we usually use jsonl-db (https://github.com/AlCalzone/jsonl-db) or use an even higher abstraction level ... I want to allow this.
So with the singleton idea internally Persistence.init(KeyValueStoreInstance) would be the only difference to your approach to set the real storage instance which is then used.
Ok?

@mfucci Ok, I now added the Singleton approach. I also adjusted my local version which uses all of this and all is fine that way ... only issues exists with the signleton approach is in testing because with Singleton we in fact can only have one persistance object (because singleton) for both Controller and Server usage. Normally only one exists in a process and so it is no issue, but for testing we use both in the same process.
The overlapping can happen currently in SessionManager when it is about remembering resumptionData because this is used by both implementations - no idea if we later have even more cases.

@mfucci seems we also missed the approval here too ... changes done as agreed, Singleton removed again.

@mfucci Ok did some further adjustments to your proposal out of "usage requirements"
1.) PersistenceManager.get can be called without default value but then throws an error if value do not exist. Sometimes you can not provide a defautl value (e.g. if it is an Object or such). To support this I added a "has" method to check if a certain key is set ... so you can check before and make sure there is a value
2.) Storage got a "close" method to "store" the data at the end
3.) The storage file not existing is now catched and allowed to get an empty storage
4.) I adjusted the storage of our special types and decided to use the "return as JSON string so it do not get processed in general, but double encoded, so we are sure that no stringify/parse is adjusting our value It is then parsed. If you more like the "normal object way" then we can also change it to return an actual object structure.


Remove *.colorDecorators.enabled language specific settings

The language specific color decorator settings (css.colorDecorators.enable, scss.colorDecorators.enable, less.colorDecorators.enable, and json.colorDecorators.enable) have been marked as deprecated for at least 7 months. "editor.colorDecorators" is now the proper setting
We should remove these deprecated settings

Note to self: Use "deprecationMessage" to mark deprecated, then remove in Dec.

Alright, already deprecated. I'll remove it next month.


compilation error with Clang 5.0.1 on Windows

When compiling with Clang 5.0.1 on Windows I get this error:
In file included from C:/Workspace/ceg/cmake-build-debug/immer-src\immer/flex_vector.hpp:25:
In file included from C:/Workspace/ceg/cmake-build-debug/immer-src\immer/memory_policy.hpp:24:
In file included from C:/Workspace/ceg/cmake-build-debug/immer-src\immer/heap/heap_policy.hpp:26:
C:/Workspace/ceg/cmake-build-debug/immer-src\immer/heap/thread_local_free_list_heap.hpp:44:39: error: cannot compile this non-trivial TLS destruction yet
thread_local_free_list_storage<Heap>::head {nullptr, 0};

I do not see any errors with GCC 7.3

It seems like Clang on Windows does not support thread-local objects with destructors. You can still use immer by disabling the thread_local_Free_list_heap from the memory policy. You can do so by passing a different memory policy as described in https://sinusoid.es/immer/memory.html, or simply by passing defining by passing -DIMMER_NO_FREE_LIST=1 to the compiler (this is, enabling such preprocessor definition at build-system level). I hope this helps!


Upload Path Issue for Base64 Updates

I am implementing an API using Grape mounted on Rails. I followed the instructions in the gem to implement the ability to upload a file using a Base64 string of an image. But the uploaded location seems to be messed up.
Upload location

Normally: public/uploads/customer/profile_picture/:id
Uploads to: public/tmp/1398957623-9523-4518

When I upload using a file it works. But when I use a base 64 string it doesn't.
** If this is a known issue, could you suggest a fix for this? If not could you give me a hint on what I might need to look at?**
Thanks in advance.
The files I think which are important are as follows:
# encoding: utf-8

class ProfilePictureUploader < CarrierWave::Uploader::Base

  include CarrierWave::MiniMagick

  # Choose what kind of storage to use for this uploader:
  storage :file
  # storage :fog

  # Override the directory where uploaded files will be stored.
  # This is a sensible default for uploaders that are meant to be mounted:
  def store_dir
    "uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
  end

  # Provides the default URL when an image is not uploaded
  def default_url
    ActionController::Base.helpers.asset_path('users/no-profile-picture.png')
  end

  version :thumb do
    process resize_to_fit: [58, 58]
  end

  def extension_white_list
    %w(jpg jpeg png)
  end

end

class User < ActiveRecord::Base
  # blah...

  # Profile picture upload
  # mount_uploader :profile_picture, ProfilePictureUploader
  mount_base64_uploader :profile_picture, ProfilePictureUploader

  # blah...
end

class AccountApi < Grape::API

  resource :account do
    # blah....

    desc 'Upload profile picture', { headers: API::HEADER_SETTINGS }
    params do
      requires :user, type: Hash do
        requires :profile_picture, type: Rack::Multipart::UploadedFile
      end
    end
    post :profile_picture do
      profile_picture = params[:user][:profile_picture]

      status = current_user.update(profile_picture: profile_picture)

      {
        uddated:  status,
        url:      current_user.picture,
        size:     profile_picture[:tempfile].size
      }
    end

    desc 'Upload profile picture (base 64)', { headers: API::HEADER_SETTINGS }
    params do
      requires :user, type: Hash do
        requires :profile_picture, type: String
      end
    end
    post :profile_picture_base64 do
      profile_picture = params[:user][:profile_picture]

      status = current_user.update(profile_picture: profile_picture)

      {
        uddated:  status,
        url:      current_user.picture,
        size:     current_user.profile_picture.size
      }
    end

  end

end

@ziyan-junaideen Can you give me an example of a base64 upload? At least the beginning of the string

This is the string I used. https://gist.github.com/ziyan-junaideen/8a3ecb423aa2c4bed323

This is actually very strange. In fact the mount_base64_uploader does not influenses the store_dir in any way, it only transforms the base64-encoded string to a StringIO, and passes it further to carrierwave.

True that, I cloned the app to just play with it locally and was confused what might be causing the issue. Probably needs some close look and debugging.

Judging by '0 issues', I think this is a local issue (particular with the project) is it? I will do some debugging and if I find a reson, will come back to you.

Thanks! I will just close this issue for now, if you think that this is gem specific, just open it again


taskProvider is not a function

Configuration
info Fetching system and libraries information...
System:
    OS: macOS 10.14.5
    CPU: (4) x64 Intel(R) Core(TM) i5-3210M CPU @ 2.50GHz
    Memory: 503.98 MB / 16.00 GB
    Shell: 3.2.57 - /bin/bash
  Binaries:
    Node: 10.15.0 - /usr/local/bin/node
    Yarn: 1.15.2 - /usr/local/bin/yarn
    npm: 6.10.1 - /usr/local/bin/npm
    Watchman: 4.9.0 - /usr/local/bin/watchman
  SDKs:
    iOS SDK:
      Platforms: iOS 12.4, macOS 10.14, tvOS 12.4, watchOS 5.3
  IDEs:
    Android Studio: 3.4 AI-183.6<IP_ADDRESS>92245
    Xcode: 10.3/10G8 - /usr/bin/xcodebuild
  npmPackages:
    react: 16.8.6 => 16.8.6
    react-native: 0.60.4 => 0.60.4
  npmGlobalPackages:
    react-native-cli: 2.0.1
    react-native-create-library: 3.1.2

What react-native-track-player version are you using?
"react": "16.8.6",
"react-native": "0.60.4",
"react-native-track-player": "^1.1.8",

Issue
It was all working until I upgrade to the new RN version. Testing on Android device.
E/ReactNativeJS: taskProvider() is not a function. (In 'taskProvider()(data)', 'taskProvider()' is an instance of Promise)

Code
index.js
import TrackPlayer from 'react-native-track-player';
import events from './src/services/player/events';

TrackPlayer.registerPlaybackService(events);

events
export default async () => {
  TrackPlayer.addEventListener('remote-play', onRemotePlay);
  TrackPlayer.addEventListener('remote-pause', onRemotePause);
  TrackPlayer.addEventListener('remote-stop', onRemoteStop);
  TrackPlayer.addEventListener('playback-state', onPlaybackState);
  TrackPlayer.addEventListener('playback-error', onPlaybackError);
};

component
class Player extends ProgressComponent<IProps, IState> {
  ...
}

export default connect(Ctrl.mapStateToProps, Ctrl.mapDispatchToProps)(Player);

service
export class PlayerService {
  public static async setupPlayer() {
    await TrackPlayer.setupPlayer({ waitForBuffer: true });

    TrackPlayer.updateOptions({
      capabilities: [
        TrackPlayer.CAPABILITY_PLAY,
        TrackPlayer.CAPABILITY_PAUSE,
        TrackPlayer.CAPABILITY_STOP,
      ],
    });

    TrackPlayer.setVolume(1); // 0.1 - 1
  }

  public static async toggle() {
    const state = await TrackPlayer.getState();
    if (state === TrackPlayer.STATE_PLAYING) {
      await TrackPlayer.pause();
    } else if (state === TrackPlayer.STATE_PAUSED) {
      await play();
    } else if (state === TrackPlayer.STATE_STOPPED) {
      await play();
    }
  }
}

[UPDATE]
doing
export default () => {
  return async () => {
    TrackPlayer.addEventListener('remote-play', onRemotePlay);
    ...
  };
};

it stops the error, but the remote controls doesn't work

Sometimes I'm also getting
E/ActivityThread: Service com.guichaguri.trackplayer.service.MusicService has leaked IntentReceiver com.guichaguri.trackplayer.service.MusicManager$1@19ee55c that was originally registered here. Are you missing a call to unregisterReceiver()?
    android.app.IntentReceiverLeaked: Service com.guichaguri.trackplayer.service.MusicService has leaked IntentReceiver com.guichaguri.trackplayer.service.MusicManager$1@19ee55c that was originally registered here. Are you missing a call to unregisterReceiver()?
        at android.app.LoadedApk$ReceiverDispatcher.<init>(LoadedApk.java:1348)
        at android.app.LoadedApk.getReceiverDispatcher(LoadedApk.java:1115)
        at android.app.ContextImpl.registerReceiverInternal(ContextImpl.java:1405)
        at android.app.ContextImpl.registerReceiver(ContextImpl.java:1378)
        at android.app.ContextImpl.registerReceiver(ContextImpl.java:1366)
        at android.content.ContextWrapper.registerReceiver(ContextWrapper.java:614)
        at com.guichaguri.trackplayer.service.MusicManager.onPlay(MusicManager.java:150)
        at com.guichaguri.trackplayer.service.player.ExoPlayback.onPlayerStateChanged(ExoPlayback.java:294)
        at com.guichaguri.trackplayer.service.player.LocalPlayback.onPlayerStateChanged(LocalPlayback.java:192)
        at com.google.android.exoplayer2.ExoPlayerImpl.lambda$setPlayWhenReady$0(ExoPlayerImpl.java:254)
        at com.google.android.exoplayer2.-$$Lambda$ExoPlayerImpl$OKMPvkXpqXeKaJZFBZ8m9YfNXpE.invokeListener(Unknown Source:4)
        at com.google.android.exoplayer2.BasePlayer$ListenerHolder.invoke(BasePlayer.java:165)
        at com.google.android.exoplayer2.ExoPlayerImpl.invokeAll(ExoPlayerImpl.java:823)
        at com.google.android.exoplayer2.ExoPlayerImpl.lambda$notifyListeners$6(ExoPlayerImpl.java:716)
        at com.google.android.exoplayer2.-$$Lambda$ExoPlayerImpl$DrcaME6RvvSdC72wmoYPUB4uP5w.run(Unknown Source:4)
        at com.google.android.exoplayer2.ExoPlayerImpl.notifyListeners(ExoPlayerImpl.java:726)
        at com.google.android.exoplayer2.ExoPlayerImpl.notifyListeners(ExoPlayerImpl.java:716)
        at com.google.android.exoplayer2.ExoPlayerImpl.setPlayWhenReady(ExoPlayerImpl.java:254)
        at com.google.android.exoplayer2.SimpleExoPlayer.updatePlayWhenReady(SimpleExoPlayer.java:1224)
        at com.google.android.exoplayer2.SimpleExoPlayer.setPlayWhenReady(SimpleExoPlayer.java:921)
        at com.guichaguri.trackplayer.service.player.ExoPlayback.play(ExoPlayback.java:134)
        at com.guichaguri.trackplayer.service.player.LocalPlayback.play(LocalPlayback.java:150)
        at com.guichaguri.trackplayer.module.MusicModule.lambda$play$10$MusicModule(MusicModule.java:316)
        at com.guichaguri.trackplayer.module.-$$Lambda$MusicModule$6tPjSgn9zq8na8C_MNqG23hrTLw.run(Unknown Source:4)
        at android.os.Handler.handleCallback(Handler.java:789)
        at android.os.Handler.dispatchMessage(Handler.java:98)
        at android.os.Looper.loop(Looper.java:169)
        at android.app.ActivityThread.main(ActivityThread.java:6595)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:240)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:767)

@rochapablo
Similar configaration is working for me in "react-native-track-player": "^1.2.0".
You can refer the Project https://github.com/YajanaRao/Serenity

You're supposed to have
TrackPlayer.registerPlaybackService(() => events);

and not
TrackPlayer.registerPlaybackService(events);

I am following the installation guide but still receive the "[TypeError: taskProvider() is not a function. (In 'taskProvider()(data)', 'taskProvider()' is undefined)" error. In the index.js I have "TrackPlayer.registerPlaybackService(() => { require("./service"); });" after AppRegistry.


Is '{"a":}' should be considered as a correct JSON?

Hey! Just curious, is it an intended behaviour? Or this module makes pretty loose assumptions?

hey, is not an intended behavior and you are free to improved
Take care


Context menu "Open link in new window" sometimes opens a new tab instead

With this extension installed, the standard Opera context menu "Open link in new window" command sometimes opens a new tab instead.  The new window opens very briefly, then closes, and the tab appears in the original window.  This doesn't happen often or predictably - most of the time, the command opens a new window and the tab stays there.
My options are set to "When opening a tab: Open all tabs at the end of the tab bar", "When closing a tab: Activate the last active tab", and "When opening a link in a new tab: Let the browser decide and Shift+click opens tabs in the same window".

And you're definitely not holding Shift when this happens? Do you know if you were holding Shift at some point before seeing this happen?
Since the extension code which detects whether Ctrl and Shift are pressed cannot run on internal Opera pages (speed dial, bookmarks, settings, etc.), it's also possible Shift could get stuck down if you were holding it while switching to one of those tabs. For instance, I can reproduce this if I hold Shift while clicking the new tab button, then go to any web page and open a link in a new window. Pressing and releasing Shift again while not on an internal page will reset the key state and fix the problem.

That's a good question and a good point.  I'll pay close attention when using the extension and see if I can characterize it better.  Most of the time, I'm opening links from my own not-quite-a-bookmark-list HTML file (file:///C:/blah/blah/blah.html) that have target="_blank" to automatically open them in new tab, but occasionally I right-click etc. to open them in a separate window.  But anyway, I'll work on a better problem report and check back when I have one.

In that case, make sure you've checked "Allow access to file URLs" for Classic Tabs in Opera's extensions page. Otherwise you'll have the same issues I mentioned with both internal pages and your local HTML files.


Addon services unable to access common config

🐞 Bug Report
Affected Services [REQUIRED]
Any addon service that doesn't have the permissions built-in by default, example: device-mqtt
Is this a regression?
No since common config is new
Description and Minimal Reproduction [REQUIRED]
run compose-builder in secure mode with device mqtt:
$ make run mqtt-broker ds-mqtt
🔥 Exception or Error
level=ERROR ts=2023-03-13T16:19:46.251975009Z app=device-mqtt source=bootstrap.go:48 msg="failed to create provider for /all-services: failed to get Configuration Provider (consul) access token: HTTP response with status code 400, message: failed to generate Consul token using [device-mqtt]: {\"errors\":[\"role \\\"device-mqtt\\\" not found\"]}\n"

🌍  Your Environment
Deployment Environment:
EdgeX Version [REQUIRED]:
Anything else relevant?

I am unable to recreate this when pulled latest secure mod images.


fix predict hive

fix predict hive

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


fix: initial voice state cache sets null-guild based cache key

Description
Assume, a bot has connected to gateway or joined a new guild. When the bot receives GUILD_CREATE event, internally nyxx will cache & transmit the event data to their respective cache instances as we expect it to do but there is a catch with how nyxx stores the cache for users who are pre-connected into a voice channel before the bot connects to gateway/joins the guild.
https://github.com/nyxx-discord/nyxx/blob/31ab26d4eb1709bb36284029f29be6f13ec0b83b/lib/src/gateway/gateway.dart#L131
In the above code, nyxx transforms List<VoiceState> as MappedListIterable<VoiceState, Map<Snowflake, VoiceState>> & directly adds them into Cache<VoiceState>.
Now, here is the main issue. In GUILD_CREATE event, Discord doesn't provide guild_id key in voice_states (since we already have event.id provided). So, the value of guild_id here will be null. Therefore, from the above code snippet, the e.cacheKey value will be of Snowflake(Object.hash(null, USER_ID)) return value; which is a bit of incorrect. Instead of using null, it should use event.guild.id value.
Having null will not synchronize with the original pre-connected user's cached voice state when the bot receives VOICE_STATE_UPDATE event. It will create a new cache entry.
This pull request fixes the initial pre-connected user's voice state cache to synchronize with VOICE_STATE_UPDATE event.
Type of change

[x] Breaking change (fix or feature that would cause existing functionality to not work as expected)

A minor breaking change affecting only how nyxx handles voice states caches of pre-connected users.
Checklist:

[x] Ran dart analyze or make analyze and fixed all issues
[x] Ran dart format --set-exit-if-changed -l 160 ./lib or make format and fixed all issues
[x] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works

I think a better solution would be to rework how voice states are cached - currently the implementation suffers from cache collisions, and also this problem you've pointed out.

I think a better solution would be to rework how voice states are cached - currently the implementation suffers from cache collisions, and also this problem you've pointed out.

Could you describe more about the cache collisions?

Whoops, typo - hash collisions, not cache collisions. Two different voice states can end up with the same cache key, which means they'll overwrite each other in the cache.

Whoops, typo - hash collisions, not cache collisions. Two different voice states can end up with the same cache key, which means they'll overwrite each other in the cache.

Hmm, now it makes a bit more sense.
Also, are y'all planning to merge this pull request for at least now to resolve the current cached voice-states issue of pre-connected users until the rework or would prefer to move to the direct rework & fixing this problem as well?

I'd just move to the direct rework - either way the fix will be in 4.0.1.

Thank you! 😄


Add -/- ___ reporting

@JasonEtco @PatMyron Closes #6
Related: https://github.com/JasonEtco/action-badges/issues/6#issuecomment-478677605

@JasonEtco I've also changed mixed to unknown (this is more like what Travis and others report)

@JasonEtco What do you think?

@JasonEtco I've made the changes that you suggested (and updated the tests).

Thanks for those changes - still needs new tests that show the expected output with multiple suites of different values!

@JasonEtco How to do you want me to do this? I'm not exactly sure how your current tests work, because the repo JasonEtco/example doesn't exist. I'll try and take a look soon though

You should be able to just pass more items to the getStatus calls within status.test.js.

@JasonEtco I'll give this a go ... thanks for the help

@JasonEtco Is this all the tests that we need?

@JasonEtco ping ...


Added function to get channel by name.

Hey! (Don't know if you saw my previous pull request but had to correct it)
Basically, I was using your bot and I needed a getChannelByName getter. So I decided to add it to your code. Do you think this should be there? Or was there already a better way to do it?
Thanks! Also, great work so far. Thanks a lot for this.

Looks good! Looks like some useful shorthand code for a lot of people! Thanks


Memory Size

Somewhere alot of memory get's used or it does not get returned fast enough.
Probably need to reduce the BUFFERS_CAP

Should mostyl be fixed


UnhandledPromiseRejectionWarning when building

I'm getting this when building with node 8.7.0:

(node:68828) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): TypeError: c.close is not a function
(node:68828) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

what version of @skpm/builder are you using? I release a new version yesterday that should fix this

La dernière version, 0.1.3. Pourquoi skpm-builder n'est pas dans un repo distinct?

should be fixed in 0.1.4.
Because it's a lot more convenient to have them all in the same repo since I often need to change them all at the same time

Compris. Thanks for all of your work!


Perform additional check for no windows on Amazon Fresh

Amazon Fresh has a different site layout when no delivery windows are available than Whole Foods (at least in San Francisco - screenshot below). This makes the script reach an error state and results in going back to the beginning of the cart each time, losing out on 50% of potential window opportunities.
My solution is to add one last minute check after checking for windows, specifically for Amazon Fresh. If we do this earlier (alongside the first no window check) we would get false negatives when there are no delivery windows today, but there are some the following days. Not the ideal solution imo, but it avoids the unnecessary error + restart.

@mokudekiru Would you happen to have the source (or at least relevant element properties) for that page? I'm working on a platform-independent tool for this and haven't been able to test available slots on Amazon Fresh.


Add conda recipe

Is your feature request related to a problem? Please describe.
Add conda recipe and CI/CD to push to bioconda

#https://github.com/bioconda/bioconda-recipes/pull/45409
Added to bioconda, still need CD workflow

Looks like the channel has its own GitHub bot that handles updates


Use Config File to Drive Components Log Configuration

Summary of changes
Changes introduced in this pull request:
Reference issue to close (if applicable)

Closes #1520
Other information and links

I think there's some confusion here. The --log-file flag is for directing log messages to a file rather than stdout. It shouldn't read a log configuration from that file.
The LogConfig should be part of the regular configuration file. And those log options shouldn't be an alternative to the default options, they should just be applied after the default options have been set.

Second part is fixed, but the logger we are using, pretty_env_logger, doesn't have a feature to save logs to a file. Should we change the logger?

Getting closer. Still need the --log-file flag (or a comment saying why it isn't possible).

the logger we are using, pretty_env_logger, doesn't have a feature to save logs to a file. Should we change the logger?

Getting closer. Still need the --log-file flag (or a comment saying why it isn't possible).

the logger we are using, pretty_env_logger, doesn't have a feature to save logs to a file. Should we change the logger?

pretty_env_logger is layered on top of env_logger. You can get the Builder for env_logger and then change the target.

Getting closer. Still need the --log-file flag (or a comment saying why it isn't possible).

the logger we are using, pretty_env_logger, doesn't have a feature to save logs to a file. Should we change the logger?

pretty_env_logger is layered on top of env_logger. You can get the Builder for env_logger and then change the target.

The only targets that can be used by  env_logger are stdout and stderr

Getting closer. Still need the --log-file flag (or a comment saying why it isn't possible).

the logger we are using, pretty_env_logger, doesn't have a feature to save logs to a file. Should we change the logger?

pretty_env_logger is layered on top of env_logger. You can get the Builder for env_logger and then change the target.

The only targets that can be used by env_logger are stdout and stderr

Ah, the Pipe option was added at a later time. Add an entry for --log-file to the configuration and comment it out. Then write a comment describing why it cannot yet be implemented (and link to https://github.com/seanmonstar/pretty-env-logger/issues/52).

@tyshko5 The dictionary is still wrong. Look at the changes to see why.


I'm not sure that's a bug

hi
i try to do following steps
`
Install (Optional)
If you want to install this header system-wide, you can do so via:
mkdir build
cd build
cmake -Dtest=ON ../
make
make test
make install
`
but failed when doing 'make'
p1code:build p1code$ make Scanning dependencies of target easyloggingpp-unit-tests [ 33%] Building CXX object CMakeFiles/easyloggingpp-unit-tests.dir/src/easylogging++.cc.o [ 66%] Building CXX object CMakeFiles/easyloggingpp-unit-tests.dir/test/main.cc.o In file included from /Users/p1code/Downloads/easyloggingpp-9.96.5/test/main.cc:3: /Users/p1code/Downloads/easyloggingpp-9.96.5/test/test.h:11:10: fatal error:        'gtest/gtest.h' file not found #include <gtest/gtest.h>          ^~~~~~~~~~~~~~~ 1 error generated. make[2]: *** [CMakeFiles/easyloggingpp-unit-tests.dir/test/main.cc.o] Error 1 make[1]: *** [CMakeFiles/easyloggingpp-unit-tests.dir/all] Error 2 make: *** [all] Error 2
then i modify the 'CMakeLists.txt'
include_directories(${gtest_SOURCE_DIR}/include ${gtest_SOURCE_DIR})
to
include_directories(${GTEST_INCLUDE_DIR})
then i remove the 'build' folder and try again
p1code:build p1code$ make Scanning dependencies of target easyloggingpp-unit-tests [ 33%] Building CXX object CMakeFiles/easyloggingpp-unit-tests.dir/src/easylogging++.cc.o [ 66%] Building CXX object CMakeFiles/easyloggingpp-unit-tests.dir/test/main.cc.o [100%] Linking CXX executable easyloggingpp-unit-tests ld: library not found for -lgtest clang: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: *** [easyloggingpp-unit-tests] Error 1 make[1]: *** [CMakeFiles/easyloggingpp-unit-tests.dir/all] Error 2 make: *** [all] Error 2
then i modify the 'CMakeLists.txt'
target_link_libraries(easyloggingpp-unit-tests gtest gtest_main)
to
target_link_libraries(easyloggingpp-unit-tests ${GTEST_LIBRARY} ${GTEST_MAIN_LIBRARY})
then it works.

You need to install gtest suite to run the test. When you remove the linking it's trying to link to gtest lib which it can't find. But when you link it, it's not even finding the gtest headers.
Read more about c++ linking


css imessage image bubble

Your text bubbles are great, have you considered implementing image bubbles?

Thanks @gaohomway! The technique I'm using in the blogpost won't work if you want the image to fill the "tail" as its rendered as another element. You could probably make it work with mask-image and/or some svg magic


Datastore local testing issues and fixes

Following #1822, I have been focusing on datastore testing, and found the following errors.
Test environment
I am using the following script:
from gcloud import datastore

client = datastore.Client()
entity = datastore.Entity(client.key('Test1'))
client.put(entity)

I have checked in python3.4 and python2.7, and these can be reproduced.
Error 1
I suspect the error is in datastore emulator rather than in gcloud python, although I think it triggers an exception that shouldn't happen in datastore.
Setting up environment
To trigger it, you need to have 2 configurations (gcloud config configurations) pointing to two different projects:

Activate configuration 1 (test-suite) gcloud config configurations activate test-suite
Run gcloud beta emulators datastore start
Stop it
Activate configuration 2 (mos-history) gcloud config configurations activate mos-history
Run gcloud beta emulators datastore start
Run the script

Show case
When starting in a changed config environment (activated another configuration), we can find the following error:
(ve2) javier@theoden:~/projects/spinoffs/gae$ python test-script.py
Traceback (most recent call last):
  File "test-script.py", line 5, in <module>
    client.put(entity)
  File "/home/javier/projects/spinoffs/gae/ve2/lib/python2.7/site-packages/gcloud/datastore/client.py", line 319, in put
    self.put_multi(entities=[entity])
  File "/home/javier/projects/spinoffs/gae/ve2/lib/python2.7/site-packages/gcloud/datastore/client.py", line 345, in put_multi
    current.commit()
  File "/home/javier/projects/spinoffs/gae/ve2/lib/python2.7/site-packages/gcloud/datastore/batch.py", line 260, in commit
    self._commit()
  File "/home/javier/projects/spinoffs/gae/ve2/lib/python2.7/site-packages/gcloud/datastore/batch.py", line 243, in _commit
    self.project, self._commit_request, self._id)
  File "/home/javier/projects/spinoffs/gae/ve2/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 331, in commit
    _datastore_pb2.CommitResponse)
  File "/home/javier/projects/spinoffs/gae/ve2/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 121, in _rpc
    data=request_pb.SerializeToString())
  File "/home/javier/projects/spinoffs/gae/ve2/lib/python2.7/site-packages/gcloud/datastore/connection.py", line 97, in _request
    error_status = status_pb2.Status.FromString(content)
  File "/home/javier/projects/spinoffs/gae/ve2/lib/python2.7/site-packages/google/protobuf/internal/python_message.py", line 780, in FromString
    message.MergeFromString(s)
  File "/home/javier/projects/spinoffs/gae/ve2/lib/python2.7/site-packages/google/protobuf/internal/python_message.py", line 1080, in MergeFromString
    if self._InternalParse(serialized, 0, length) != length:
  File "/home/javier/projects/spinoffs/gae/ve2/lib/python2.7/site-packages/google/protobuf/internal/python_message.py", line 1106, in InternalParse
    new_pos = local_SkipField(buffer, new_pos, end, tag_bytes)
  File "/home/javier/projects/spinoffs/gae/ve2/lib/python2.7/site-packages/google/protobuf/internal/decoder.py", line 850, in SkipField
    return WIRETYPE_TO_SKIPPER[wire_type](buffer, pos, end)
  File "/home/javier/projects/spinoffs/gae/ve2/lib/python2.7/site-packages/google/protobuf/internal/decoder.py", line 820, in _RaiseInvalidWireType
    raise _DecodeError('Tag had invalid wire type.')
google.protobuf.message.DecodeError: Tag had invalid wire type.

With the following datastore emulator output:
javier@theoden ~/p/s/gae> gcloud beta emulators datastore start
WARNING: Reusing existing data in [/home/javier/.config/gcloud/emulators/datastore].
Executing: /opt/google-cloud-sdk/platform/gcd/gcd.sh start --host=localhost --port=8572 --store_on_disk=True --consistency=0.9 --allow_remote_shutdown /home/javier/.config/gcloud/emulators/datastore
[datastore] API endpoint: http://localhost:8572/datastore
[datastore] If you are using a library that supports the DATASTORE_LOCAL_HOST environment variable, run:
[datastore]
[datastore]   export DATASTORE_LOCAL_HOST=localhost:8572
[datastore]
[datastore] Invoking dev_appserver with args: [--disable_update_check --jvm_flag=-Doauth.is_admin=true --property=datastore.index_configuration_format=yaml --port=8572 --address=localhost --property=datastore.force_is_high_replication=true --property=datastore.default_high_rep_job_policy_unapplied_job_pct=10.0 --allow_remote_shutdown /home/javier/.config/gcloud/emulators/datastore].
[datastore] Executing [/usr/lib/jvm/java-8-jdk/jre/bin/java, -Doauth.is_admin=true, -javaagent:/opt/google-cloud-sdk/platform/gcd/.appengine/lib/agent/appengine-agent.jar, -Xbootclasspath/p:/opt/google-cloud-sdk/platform/gcd/.appengine/lib/override/appengine-dev-jdk-overrides.jar, -classpath, /opt/google-cloud-sdk/platform/gcd/CloudDatastore.jar:/opt/google-cloud-sdk/platform/gcd/.appengine/lib/impl/appengine-api.jar:/opt/google-cloud-sdk/platform/gcd/.appengine/lib/appengine-tools-api.jar:/opt/google-cloud-sdk/platform/gcd/.appengine/lib/impl/appengine-api-stubs.jar, com.google.appengine.tools.development.DevAppServerMain, --property=kickstart.user.dir=/home/javier/projects/spinoffs/gae, --disable_update_check, --property=datastore.index_configuration_format=yaml, --port=8572, --address=localhost, --property=datastore.force_is_high_replication=true, --property=datastore.default_high_rep_job_policy_unapplied_job_pct=10.0, --allow_remote_shutdown, /home/javier/.config/gcloud/emulators/datastore]
[datastore] May 31, 2016 9:32:02 PM com.google.apphosting.utils.jetty.JettyLogger info
[datastore] INFO: Logging to JettyLogger(null) via com.google.apphosting.utils.jetty.JettyLogger
[datastore] May 31, 2016 9:32:02 PM com.google.appengine.tools.development.DevAppServerImpl setServerTimeZone
[datastore] WARNING: Unable to set the TimeZone to UTC (this is expected if running on JDK 8)
[datastore] May 31, 2016 9:32:02 PM com.google.apphosting.utils.jetty.JettyLogger info
[datastore] INFO: jetty-6.1.x
[datastore] May 31, 2016 9:32:02 PM com.google.apphosting.utils.jetty.JettyLogger info
[datastore] INFO: Started SelectChannelConnector@localhost:8572
[datastore] May 31, 2016 9:32:02 PM com.google.appengine.tools.development.AbstractModule startup
[datastore] INFO: Module instance default is running at http://localhost:8572/
[datastore] May 31, 2016 9:32:02 PM com.google.appengine.tools.development.AbstractModule startup
[datastore] INFO: The admin console is running at http://localhost:8572/_ah/admin
[datastore] May 31, 2016 9:32:02 PM com.google.appengine.tools.development.DevAppServerImpl doStart
[datastore] INFO: Dev App Server is now running
[datastore] May 31, 2016 9:32:22 PM com.google.appengine.tools.development.DevAppEngineWebAppContext disableTransportGuarantee
[datastore] INFO: Ignoring <transport-guarantee> for /datastore/* as the SDK does not support HTTPS.  It will still be used when you upload your application.
[datastore] May 31, 2016 9:32:22 PM com.google.apphosting.client.serviceapp.BaseApiServlet doPost
[datastore] INFO: Hosted project, test-suite, does not match requested project, mos-history.

Workaround
I have discovered that this if fixed by doing
rm -rf ~/.config/gcloud/emulators/datastore/

Running perfect after the datastore purge
javier@theoden ~/p/s/gae> gcloud beta emulators datastore start
Executing: /opt/google-cloud-sdk/platform/gcd/gcd.sh create --project_id=mos-history /home/javier/.config/gcloud/emulators/datastore
[datastore] Created new Cloud Datastore project in '/home/javier/.config/gcloud/emulators/datastore' with project ID 'mos-history'.
Executing: /opt/google-cloud-sdk/platform/gcd/gcd.sh start --host=localhost --port=8725 --store_on_disk=True --consistency=0.9 --allow_remote_shutdown /home/javier/.config/gcloud/emulators/datastore
[datastore] API endpoint: http://localhost:8725/datastore
[datastore] If you are using a library that supports the DATASTORE_LOCAL_HOST environment variable, run:
[datastore]
[datastore]   export DATASTORE_LOCAL_HOST=localhost:8725
[datastore]
[datastore] Invoking dev_appserver with args: [--disable_update_check --jvm_flag=-Doauth.is_admin=true --property=datastore.index_configuration_format=yaml --port=8725 --address=localhost --property=datastore.force_is_high_replication=true --property=datastore.default_high_rep_job_policy_unapplied_job_pct=10.0 --allow_remote_shutdown /home/javier/.config/gcloud/emulators/datastore].
[datastore] Executing [/usr/lib/jvm/java-8-jdk/jre/bin/java, -Doauth.is_admin=true, -javaagent:/opt/google-cloud-sdk/platform/gcd/.appengine/lib/agent/appengine-agent.jar, -Xbootclasspath/p:/opt/google-cloud-sdk/platform/gcd/.appengine/lib/override/appengine-dev-jdk-overrides.jar, -classpath, /opt/google-cloud-sdk/platform/gcd/CloudDatastore.jar:/opt/google-cloud-sdk/platform/gcd/.appengine/lib/impl/appengine-api.jar:/opt/google-cloud-sdk/platform/gcd/.appengine/lib/appengine-tools-api.jar:/opt/google-cloud-sdk/platform/gcd/.appengine/lib/impl/appengine-api-stubs.jar, com.google.appengine.tools.development.DevAppServerMain, --property=kickstart.user.dir=/home/javier/projects/spinoffs/gae, --disable_update_check, --property=datastore.index_configuration_format=yaml, --port=8725, --address=localhost, --property=datastore.force_is_high_replication=true, --property=datastore.default_high_rep_job_policy_unapplied_job_pct=10.0, --allow_remote_shutdown, /home/javier/.config/gcloud/emulators/datastore]
[datastore] May 31, 2016 9:36:32 PM com.google.apphosting.utils.jetty.JettyLogger info
[datastore] INFO: Logging to JettyLogger(null) via com.google.apphosting.utils.jetty.JettyLogger
[datastore] May 31, 2016 9:36:32 PM com.google.appengine.tools.development.DevAppServerImpl setServerTimeZone
[datastore] WARNING: Unable to set the TimeZone to UTC (this is expected if running on JDK 8)
[datastore] May 31, 2016 9:36:32 PM com.google.apphosting.utils.jetty.JettyLogger info
[datastore] INFO: jetty-6.1.x
[datastore] May 31, 2016 9:36:33 PM com.google.apphosting.utils.jetty.JettyLogger info
[datastore] INFO: Started SelectChannelConnector@localhost:8725
[datastore] May 31, 2016 9:36:33 PM com.google.appengine.tools.development.AbstractModule startup
[datastore] INFO: Module instance default is running at http://localhost:8725/
[datastore] May 31, 2016 9:36:33 PM com.google.appengine.tools.development.AbstractModule startup
[datastore] INFO: The admin console is running at http://localhost:8725/_ah/admin
[datastore] May 31, 2016 9:36:33 PM com.google.appengine.tools.development.DevAppServerImpl doStart
[datastore] INFO: Dev App Server is now running
[datastore] May 31, 2016 9:36:50 PM com.google.appengine.tools.development.DevAppEngineWebAppContext disableTransportGuarantee
[datastore] INFO: Ignoring <transport-guarantee> for /datastore/* as the SDK does not support HTTPS.  It will still be used when you upload your application.
[datastore] May 31, 2016 9:36:50 PM com.google.appengine.api.datastore.dev.LocalDatastoreService init
[datastore] INFO: Local Datastore initialized:
[datastore]     Type: High Replication
[datastore]     Storage: /home/javier/.config/gcloud/emulators/datastore/WEB-INF/appengine-generated/local_db.bin
[datastore] May 31, 2016 9:36:50 PM com.google.appengine.api.datastore.dev.LocalDatastoreService load
[datastore] INFO: The backing store, /home/javier/.config/gcloud/emulators/datastore/WEB-INF/appengine-generated/local_db.bin, does not exist. It will be created.
[datastore] May 31, 2016 9:37:20 PM com.google.appengine.api.datastore.dev.LocalDatastoreService$PersistDatastore persist
[datastore] INFO: Time to persist datastore: 17 ms
[datastore] May 31, 2016 9:38:50 PM com.google.appengine.api.datastore.dev.LocalDatastoreService$PersistDatastore persist
[datastore] INFO: Time to persist datastore: 1 ms

The error probably comes from ~/.config/gcloud/emulators/datastore/WEB-INF/appengine-web.xml not getting updated properly.
Why does gcloud-python fail so cryptic is unknown to me.
Error 2
I think this one is about mistakenly patching httplib for local testing.
Setting up environment
This one needs:

Application default credentials set up

Showcase
I think this manifests with a Connection refused error:
(ve) javier@theoden:~/projects/spinoffs/gae$ python test-script.py
Traceback (most recent call last):
  File "test-script.py", line 5, in <module>
    client.put(entity)
  File "/home/javier/projects/spinoffs/gae/ve/lib/python3.4/site-packages/gcloud/datastore/client.py", line 319, in put
    self.put_multi(entities=[entity])
  File "/home/javier/projects/spinoffs/gae/ve/lib/python3.4/site-packages/gcloud/datastore/client.py", line 345, in put_multi
    current.commit()
  File "/home/javier/projects/spinoffs/gae/ve/lib/python3.4/site-packages/gcloud/datastore/batch.py", line 260, in commit
    self._commit()
  File "/home/javier/projects/spinoffs/gae/ve/lib/python3.4/site-packages/gcloud/datastore/batch.py", line 243, in _commit
    self.project, self._commit_request, self._id)
  File "/home/javier/projects/spinoffs/gae/ve/lib/python3.4/site-packages/gcloud/datastore/connection.py", line 331, in commit
    _datastore_pb2.CommitResponse)
  File "/home/javier/projects/spinoffs/gae/ve/lib/python3.4/site-packages/gcloud/datastore/connection.py", line 121, in _rpc
    data=request_pb.SerializeToString())
  File "/home/javier/projects/spinoffs/gae/ve/lib/python3.4/site-packages/gcloud/datastore/connection.py", line 93, in _request
    method='POST', headers=headers, body=data)
  File "/home/javier/projects/spinoffs/gae/ve/lib/python3.4/site-packages/oauth2client/client.py", line 631, in new_request
    redirections, connection_type)
  File "/home/javier/projects/spinoffs/gae/ve/lib/python3.4/site-packages/httplib2/__init__.py", line 1314, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/home/javier/projects/spinoffs/gae/ve/lib/python3.4/site-packages/httplib2/__init__.py", line 1064, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/home/javier/projects/spinoffs/gae/ve/lib/python3.4/site-packages/httplib2/__init__.py", line 987, in _conn_request
    conn.connect()
  File "/usr/lib/python3.4/http/client.py", line 871, in connect
    self.timeout, self.source_address)
  File "/usr/lib/python3.4/socket.py", line 516, in create_connection
    raise err
  File "/usr/lib/python3.4/socket.py", line 507, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

This is caused because it tries to make a auth token refresh against the emulator. There are no relevant logs for the emulator.
Workaround
The problem is the default credential, so it can be fixed by (ordered by personal preference):

Create a configuration with no default credentials, for the same project, or;
Remove the default credentials gcloud beta auth application-default revoke

Hopefully you know how to patch the library not to collide!

@txomon I don't see any mention of using $(gcloud beta emulators datastore start env-init) or export-ing the required environment variables. This seems to be a likely reason that you don't see the right app.
As for the credentials issue, can you file a separate issue so we don't get our streams crossed.

@dhermes that is done, I didn't put it here though. You can actually check that it's connecting in the logs
As for the credentials issue, ok!

/cc @pcostell

So I think there are really two issues here, though fixing either one should solve your issue.

Cloud Datastore emulator enforces one project per data-dir directory
Changing the gcloud config doesn't change the Cloud Datastore emulator directory

For (1) this should be fixed by the new emulator, which you can run with gcloud beta emulators datastore start --no-legacy
For (2) this looks like a bug, I'll dig into it. A workaround is to specify --data-dir, which changes where the data is written. You could then have two different data directories, one for each project.

@pcostell Can we close this?


Showcase teams weather app facelift and refactoring

I rewrote the whole app to react because I couldn't get myself to touch pure html and js. Next to that I have implemented the design provided by Jacek from the creative team.
This facelift is necessary for the launch of the golem-portal to the community

Waiting for cert from DevOps to sign new manifest. Current one is expired.

Updated the manifest. Everything works now


failed to run colab notebook

Downloading ReStyle model for toonify...
ValueError                                Traceback (most recent call last)
 in ()
4     # if google drive receives too many requests, we'll reach the quota limit and be unable to download the model
5     if os.path.getsize(EXPERIMENT_ARGS['model_path']) < 1000000:
----> 6         raise ValueError("Pretrained model was unable to be downloaded correctly!")
7     else:
8         print('Done.')
ValueError: Pretrained model was unable to be downloaded correctly!

It seems like we reached the quota limit for downloading the pretrained model. You should try again in a bit (an hour or so).
You can also try using gdown which may be more stable.

NO, there is some problem in data downloading code , I manually copied the URL and was able to download from URL , so I copied the model into my drive and then added it by connecting my drive.  I believe the downloading command is working asynchronously so before download completes it  triggers the  size condition and throws the error.

@skylum1 , I am not too sure that the downloading command is asynchronous. According to the docs, os.systems() will force python to wait until the process has completed before continuing the execution. In this case, the download must finish before moving on to check that the size is valid.
In any case, I just ran the colab notebook and was able to download the churches model with no issues.
Another easy way to check that the model was downloaded correctly is that if you see that the model is, say, 3kB, you can run cat on it and check if you see the quota warning.
Can you please check again that this is indeed caused by what you mentioned? If the issue persists I will look into it again.

gdown works great for me
pip install gdown
!gdown https://drive.google.com/uc?id=file_id

You can get the file id from download_command

why to use os.system()?
couldn't it just be replace by adding ! wget ?

@Mustafa744 - yea that's definitely possible. I chose to use os.system because that way I could do everything in python. By doing so, I was able to add the check if not os.path.exists(EXPERIMENT_ARGS['model_path']) thereby ensuring that the model was only downloaded once (reduce the number of requests to the drive).
There are probably nicer ways to do so that I am not aware of :)

Prefer using pathlib in that case, handling files is much more simplified

@yuval-alaluf  i hate the os module 🤣
will you update the colab notebook with this edit ?

I can't download the pretrained model

!pip install gdown
!gdown https://drive.google.com/uc?id=1sw6I2lRIB0MpuJkpc8F5BJiSZrc0hjfE

Requirement already satisfied: gdown in /usr/local/lib/python3.7/dist-packages (3.6.4)
Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from gdown) (1.15.0)
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from gdown) (2.23.0)
Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from gdown) (4.41.1)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->gdown) (3.0.4)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->gdown) (2.10)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->gdown) (1.24.3)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->gdown) (2020.12.5)
Permission denied: https://drive.google.com/uc?id=1sw6I2lRIB0MpuJkpc8F5BJiSZrc0hjfE
Maybe you need to change permission over 'Anyone with the link'?

@yuval-alaluf i hate the os module 🤣
will you update the colab notebook with this edit ?

@Mustafa744 - since the current method does work when there are no issues with the quota, I do not plan on making any changes. However, if you feel like there are improvements that can be made you are welcome to open a pull request with these changes.

I can't download the pretrained model

!pip install gdown
!gdown https://drive.google.com/uc?id=1sw6I2lRIB0MpuJkpc8F5BJiSZrc0hjfE

Requirement already satisfied: gdown in /usr/local/lib/python3.7/dist-packages (3.6.4)
Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from gdown) (1.15.0)
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from gdown) (2.23.0)
Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from gdown) (4.41.1)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->gdown) (3.0.4)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->gdown) (2.10)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->gdown) (1.24.3)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->gdown) (2020.12.5)
Permission denied: https://drive.google.com/uc?id=1sw6I2lRIB0MpuJkpc8F5BJiSZrc0hjfE
Maybe you need to change permission over 'Anyone with the link'?
All the other methods to download the pretrained model also fail as described in this issue.
However I can download the file links manually .. too tedious tho.

I have examined this further. All the models I linked here have the correct permissions. Based on this issue, https://github.com/wkentaro/gdown/issues/43, it appears that this is yet again an issue with reaching the quota on Google Drive.
I was able to use gdown --id <file id> to download a select number of files, however,
Unfortunately, there is not much I can do at this time. It just happened that many people have tried downloading the models in the past 24 hours. I kindly ask everyone to be patient to see if this issue resolves itself in the next day or two. Thanks.


JsonB support in postgres and h2

This is kind of related to https://github.com/vladmihalcea/hibernate-types/issues/316
@Type(type = "json")
@Column(columnDefinition = "jsonb")
private MyDataObject data;

Mapping works like a charm in postgres
When using h2 while running tests, I get the following error
Caused by: org.h2.jdbc.JdbcSQLException: Unknown data type: "JSONB"; SQL statement
I see the test cases for H2 use json,  while the ones for  postgres use jsonb
Is the combination of h2 & jsonb  not possible at the moment(maybe because of h2) or is this an oversight?

If you use JsonType, it should work as this Type picks the proper descriptor for each supported DB. Check out the JavaDoc and the project examples for more details.


Add knife tidy notify command

This PR adds the knife tidy notify command and associated documentation.
This command iterates over reports generated by knife tidy server report, queries the configured Chef server for a list of admin users for each org contained in the reports, and generates a summary of stale server data to send to each org's admins via the SMTP server passed in on the command line.

This looks great @jonlives!
Thank you for the excellent contribution.


chore(release): Publish packages<EMAIL_ADDRESS>

<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

storage_client still needs an upgrade #1079

Oh, it wasn't included because it was a chore. Let me manually add it.


fix: move proxyplugin creation and deletion

Description
fixes an issue seen in build-definitions e2e:
[BeforeAll] /github.com/redhat-appstudio/e2e-tests/tests/build/build_templates.go:34
  [It] /github.com/redhat-appstudio/e2e-tests/tests/build/build_templates.go:122
  [FAILED] Unexpected error:
      <*errors.errorString | 0xc000517ea0>:
      unable to create proxy plugin due to no matches for kind "ProxyPlugin" in version "[toolchain.dev.openshift.com/v1alpha1](http://toolchain.dev.openshift.com/v1alpha1)"
      {
          s: "unable to create proxy plugin due to no matches for kind \"ProxyPlugin\" in version \"[toolchain.dev.openshift.com/v1alpha1](http://toolchain.dev.openshift.com/v1alpha1)\"",
      }
  occurred
  In [BeforeAll] at: /github.com/redhat-appstudio/e2e-tests/tests/build/build_templates.go:58 @ 08/22/23 11:11:39.92

Issue ticket number and link
N/A
Type of change

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
Checklist:

[ ] I have performed a self-review of my code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added meaningful description with JIRA/GitHub issue key(if applicable), for example HASSuiteDescribe("STONE-123456789 devfile source")
[ ] I have updated labels (if needed)

/test redhat-appstudio-e2e


vm network list initial upload

knife vsphere vm network list VMNAME
for a vm (VMNAME), list its network cards and for each network card, list its connected network (vlan)
currently not working.
I need some help with listing the networks that a NIC is connected to.

Currently, this only lists the network cards on a VM. I need some help with listing a network cards' connected network. I think the code would be something like this?

nic.network.each do |network|
   puts "#{ui.color('VLAN', :red)}: #{network.name}"
end

Also, when I use
#{a.deviceInfo.summary}, I get an output like this DVSwitch: a8 8f 21 50 2e ac 27 e4-6a c1 84 0f 4d ac 1e b2.
How do you get the summary text

Currently I am a bit closer, I can list the portgroupKey and switchUuid for each nic:

    vm.config.hardware.device.each.grep(RbVmomi::VIM::VirtualEthernetCard).map do |nic|
      puts "#{ui.color('NIC', :cyan)}: #{nic.backing.port.portgroupKey}"
      puts "#{ui.color('NIC', :cyan)}: #{nic.backing.port.switchUuid}"
    end

I need some help to be able to list the switchName or the DVPortgroup name please.

ready

Just used it and it works. Going to merge it, but I think there could be some future improvements to make it a bit faster. Right now it seems to loop through the DC networks once per NIC. So we could either cache that list or do an API call to do a lookup.


Introduce case statements independently of type annotations

It strikes me that we could perhaps eliminate NoneType and have Maybe<T> even without all of #33. (I was inspired by @raiph in thinking in this direction.)
The premise is this:

None and NoneType are gone.
Whenever you want to allow for a None-like value, you give something the type Maybe<Expr> or similar. On the Perl 6 level.
We implement case statements à la #34.
But without any type annotations and type inference switched on, these switch statements are mostly just syntax; they're a way to dig into a Maybe, but they don't come with any strictures. (No totality checking; no checking the cases against the surrounding type.)

I should add that this would be an experiment, to see if we like it better than the current None status quo. There is promise in the idea; it would for example address the concerns raised in #148, too.

I liked pampy, which provides a pretty neat library-only way to pattern match. We could try porting it to 007 and see how it looks; it might be a sensible step on the way to doing pattern matching with syntax.

I'm going to close this one; I find I don't believe strongly in getting rid of none these days. (Also, TypeScript sort of shows that it's not super-necessary either.)


Add content for September 2021 publish

Docs Build status updates of commit 86d960d:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

sysinternals/downloads/autoruns.md
:white_check_mark:Succeeded
View

sysinternals/downloads/index.md
:white_check_mark:Succeeded
View

sysinternals/downloads/procmon.md
:white_check_mark:Succeeded
View

sysinternals/downloads/sysinternals-suite.md
:white_check_mark:Succeeded
View

sysinternals/downloads/tcpview.md
:white_check_mark:Succeeded
View

sysinternals/downloads/winobj.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit ef7d8b8:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

sysinternals/downloads/autoruns.md
:white_check_mark:Succeeded
View

sysinternals/downloads/index.md
:white_check_mark:Succeeded
View

sysinternals/downloads/procmon.md
:white_check_mark:Succeeded
View

sysinternals/downloads/sysinternals-suite.md
:white_check_mark:Succeeded
View

sysinternals/downloads/tcpview.md
:white_check_mark:Succeeded
View

sysinternals/downloads/winobj.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Use Bicep ARM templates

Microsoft is encouraging the use of Bicep ARM templates, it should be possible to use them in the python SDK.
Since it isn't currently possible to deploy a remote Bicep file in Azure (https://docs.microsoft.com/en-us/azure/azure-resource-manager/bicep/deploy-cli#deploy-remote-bicep-file) the SDK should offer an option to source the file locally.

Thanks for the feedback, we’ll investigate asap.

@armleads-azure Do you have any updates on this.

@xiangyan99 do we support deploy with bicep?