Implement one to one mapping between project owner and a project

We want to allow only one PROJECT_ADMIN for a project.

Why? Having multiple PROJECT_ADMINs will avoid password-sharing.

If that is what we need, we could proceed with current implementation.

Decided not to implement this feature for now.


GIVE CUT IN DURATION AND ACCELERATION TO VEHICLE

Hi,  i would like to give a specific maneuver duration to vehicle for lane-change maneuver. Is there any way to assign a maneuver duration to it with scenario_runner.py or in OpenScenario format  ?
On the other hand,  I can give an initial speed to ego_vehicle but at the same time i would like to give an acceleration to it. In openscenario_parser.py, it is written that "Acceleration is not implemented yet. " Is there another way to give an acceleration to ego_vehicle with scenario_runner.py or in OpenScenario format  ?
Ubuntu 16.04, Carla 0.9.8, Python 2.7

Hello @iremonur

specific maneuver duration to vehicle for lane-change maneuver

What do you mean here? You want to the lane change action to take an specific amount of time?

Is there another way to give an acceleration to ego_vehicle

We currently have no atomic to set the acceleration to an specific value so you'd have to create one by yourself. It shouldn't be very hard though, here is the atomic that sets the initial velocity (it is in SR 0.9.9 though). With a quick glance, I'd say you just have to change the set_velocity for add_impulse.

Closing this issue as a TriggerAcceleration atomic has been created, and support to AccelerationCondition


upstream-community-operators node-healthcheck-operator (0.0.2)

release community node-healthcheck-operator 0.0.2

/merge possible


reading of parquet binary type, is very slow:

I find [ https://github.com/jorgecarleitao/arrow2/blob/main/src/io/parquet/read/binary/mod.rs#L34]
capacity cannot be pre allocated here，Cause ‘extend_from_Slice’ is very slow。

Ahahaha I also found that out yesterday! Fully agree, we should pre-reserve this one for sure. Not that in plain, a value is always [4 bytes][values] and each page anounces how many values it contains, so we can pre-allocate exactly.


Add static site

@secwang is github pages set up?

Thank you for your hard work. I set up githubpages, but it seems that it will take some time to take effect. OK, I'll reply to you.

It's ready.’https://secwang.github.io/ngnkcart/‘


[feat] Use default terminal editor for writing SQL

I use NeoVim almost for everything, I'd love to be able to write my SQL in lazysql with highlighting and LSP, which neovim offers for me
So in prefer of the Unix philosophy why don't we make it so you make a temporary file and use the default editor for writing SQL?
Would love to contribute with a PR but I'm kind of busy for now maybe if it wasn't implemented and I got some free time soon I'll implement it and make PR.

If you are in Linux, you can try this


Hyper-V - NATAdapter & Network Adapter

Operating System Version: Windows 10 Professional
Deploying via (VirtualBox/VMWare/AWS/Azure/ESXi): Hyper-V
Vagrant Version (if applicable): 2.2.10

Description of the issue:
When building with the default switch option 1, during in the CMD prompt, all the switches use the  NATswitch still at the end of the build instead of the default switch. I initially tested with option 2 NATSwitch and seemed to hang with WINRM in the build.
There is a second part as well, this is with a full build with option 1 for the default adapter. This error occurs after the shutdown of logger, after the machine is shutdown logger no longer has the IP <IP_ADDRESS> and is unable to communicate with the other machines, everything works prior to this. I tried to remove the NATAdapter and NATSwitch from logger, then in the GUI for hyper-v add the NATSwitch to the machine but doesn't seem to work.
IPs assigned immediately after deployment:
PS C:\WINDOWS\system32> Get-VMNetworkAdapter -VMName *

Name            IsManagementOs VMName                SwitchName     MacAddress   Status IPAddresses
----            -------------- ------                ----------     ----------   ------ -----------
NATAdapter      False          wef.windomain.local   NATSwitch      00155DAB3524 {Ok}   {<IP_ADDRESS>}
Network Adapter False          logger                NATSwitch      00155DAB3528 {Ok}   {<IP_ADDRESS>, fe80::215:5dff:feab:3528}
NATAdapter      False          win10.windomain.local NATSwitch      00155DAB3526 {Ok}   {<IP_ADDRESS>}
NATAdapter      False          dc.windomain.local    NATSwitch      00155DAB3522 {Ok}   {<IP_ADDRESS>}

IP assigned after reboot of logger:
Name            IsManagementOs VMName                SwitchName     MacAddress   Status IPAddresses
----            -------------- ------                ----------     ----------   ------ -----------
NATAdapter      False          wef.windomain.local   NATSwitch      00155DAB3524 {Ok}   {<IP_ADDRESS>}
Network Adapter False          logger                NATSwitch      00155DAB3528 {Ok}   {}
NATAdapter      False          win10.windomain.local NATSwitch      00155DAB3526 {Ok}   {<IP_ADDRESS>}
NATAdapter      False          dc.windomain.local    NATSwitch      00155DAB3522 {Ok}   {<IP_ADDRESS>}

### Link to Gist Containing Build Logs:
I didn't happen to see the build log generated, after build. I can add those generated during the build in CMD here tomorrow.

Hi @APT-0 , unfortunately I don't support the HyperV provider for DetectionLab, but I'm going to tag in @man715 who developed the HyperV code just in case he has any thoughts!
@man715 - hopefully you don't mind me looping you into this issue!

Hey @APT-0 ;

When building with the default switch option 1, during in the CMD prompt, all the switches use the NATswitch still at the end of the build instead of the default switch. I initially tested with option 2 NATSwitch and seemed to hang with WINRM in the build.

You will need to choose the first (default) option for the Network Adapter. That is how the VM is able to get an IP address and the host machine can communicate with the VM. If you try to have the VM use the NATSwitch from the start, it will not get assigned an IP address and Vagrant will not be able to communicate with it. After that, the scripts create a static IP for the box and use the NATSwitch that way the DetectionLab VMs are on their own isolated network.

There is a second part as well, this is with a full build with option 1 for the default adapter. This error occurs after the shutdown of logger, after the machine is shutdown logger no longer has the IP <IP_ADDRESS> and is unable to communicate with the other machines, everything works prior to this. I tried to remove the NATAdapter and NATSwitch from logger, then in the GUI for hyper-v add the NATSwitch to the machine but doesn't seem to work.

I am not sure why it would lose the IP address. I will try to replicate this on my end and get back with you.

I have been able to replicate the issue of the IP address issue. I am working on fixing the cause. A workaround for now would be to edit the /etc/netplan/01-netcfg.yaml and remove anything after renderer: networkd. Then reboot the VM.

Thanks alot @man715 ! That worked.

@APT-0 are there any changes to the documentation I can add to help future users avoid this issue?

Using @man715's suggestion, maybe just a note at end of hyper-v's readme known issues:
A workaround for logger losing network access or no longer having an assigned IP or any access to any of apps including splunk after a your first shutdown, would be to edit the /etc/netplan/01-netcfg.yaml on logger's machine and remove anything after renderer: networkd. Then reboot the VM.
To check if all your virtual machines have an IP assigned use in powershell: Get-VMNetworkAdapter -VMName *

Updated the docs, marking this one as closed. Thanks for jumping in here @man715 !


hi all . i want to use scrcpy -m 1024 input  opencv capture how i doing ?

hi all . i want to use scrcpy -m 1024  input  opencv capture how i doing ?  thanks

See related question: https://github.com/Genymobile/scrcpy/issues/437


Custom Markers not displaying on google maps in flutter app

I made some custom markers for my new google maps flutter project, but my markers are not visible on the map!
I spent a long time searching for solutions but nothing worked. The markers show when I run the app for the first time, but that's about it, once I hot restart, they're gone.
Here's my code:

sample code
import 'dart:typed_data';
import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:google_maps_flutter/google_maps_flutter.dart';
import 'dart:ui' as ui;
import 'package:permission_handler/permission_handler.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: MyHomePage(),
    );
  }
}

class MyHomePage extends StatefulWidget {
  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  Set<Marker> _markers = {};

  late BitmapDescriptor myPosition;
  late BitmapDescriptor safeHouse;
  late BitmapDescriptor customIcon3;
  late BitmapDescriptor customIcon4;
  late BitmapDescriptor customIcon5;
  late BitmapDescriptor customIcon6;
  late BitmapDescriptor customIcon7;

  @override
  void initState() {

    getBytesFromAsset('assets/myPosition.png', 160).then((onValue) {
      myPosition = BitmapDescriptor.fromBytes(onValue);
    });
    getBytesFromAsset('assets/safePeople.png', 160).then((onValue) {
      safeHouse = BitmapDescriptor.fromBytes(onValue);
    });
    requestPermission();
    super.initState();
    //setPermissions();

  }

  Future<void> requestPermission() async {
    await Permission.location.request();
  }
  //void setPermissions() async {
  //Map<PermissionGroup, PermissionStatus> permissions =
  //await PermissionHandler()
  //.requestPermissions([PermissionGroup.location]);
  // }

  static Future<Uint8List> getBytesFromAsset(String path, int width) async {
    ByteData data = await rootBundle.load(path);
    ui.Codec codec = await ui.instantiateImageCodec(data.buffer.asUint8List(),
        targetWidth: width);
    ui.FrameInfo fi = await codec.getNextFrame();

    return (await fi.image.toByteData(format: ui.ImageByteFormat.png))!
        .buffer
        .asUint8List();
  }

  void _onMapcreated(GoogleMapController controller) {
    setState(() {
      _markers.add(Marker(
          icon: myPosition,
          markerId: MarkerId("Your position"),
          infoWindow: InfoWindow(title: "Your Position"),
          position: LatLng(36.737846, 4.043071)));

      _markers.add(Marker(
          markerId: MarkerId("safeHouse1"),
          infoWindow: InfoWindow(title: "Safe house 1"),
          icon: safeHouse,
          position: LatLng(36.736147, 4.045501)));

      _markers.add(Marker(
          icon: safeHouse,
          markerId: MarkerId("safeHouse2"),
          infoWindow: InfoWindow(title: "Safe house 2"),
          position: LatLng(36.737090, 4.043984)));

      _markers.add(Marker(
          icon: safeHouse,
          markerId: MarkerId("safeHouse3"),
          infoWindow: InfoWindow(title: "Safe house 3"),
          position: LatLng(36.736847, 4.045026)));

      _markers.add(Marker(
          icon: safeHouse,
          markerId: MarkerId("safeHouse4"),
          infoWindow: InfoWindow(title: "Safe house 4"),
          position: LatLng(36.736178, 4.044390)));
    });
  }

  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        backgroundColor: Colors.red,
      ),
      body: Stack(
        children: [
          GoogleMap(
              onMapCreated: _onMapcreated,
              mapToolbarEnabled: true,
              buildingsEnabled: false,
              myLocationButtonEnabled: true,
              mapType: MapType.normal,
              markers: _markers,
              initialCameraPosition: CameraPosition(
                target: LatLng(36.736147, 4.045501),
                zoom: 17.5,
              )),

        ],
      ),
    );
  }
}

flutter doctor:

flutter doctor -v
Doctor summary (to see all details, run flutter doctor -v):
[√] Flutter (Channel dev, 2.6.0-11.0.pre, on Microsoft Windows [Version 10.0.19042.1165], locale en-US)
[!] Android toolchain - develop for Android devices (Android SDK version 30.0.1)
    X cmdline-tools component is missing
      Run path/to/sdkmanager --install "cmdline-tools;latest"
      See https://developer.android.com/studio/command-line for more details.
    X Android license status unknown.
      Run flutter doctor --android-licenses to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/windows#android-setup for more details.
[√] Chrome - develop for the web
[√] Android Studio (version 4.0)
[√] VS Code (version 1.62.1)
[√] Connected device (3 available)

Any thoughts ?

Hi @R12-cookies, thanks for filing the issue.
Can you share the output of flutter run -v? The output will be huge, so please put it into a txt file and attach it here.
Thank you

I think I figured out what's the issue, It's an asynchronous suspension basically google map widget is rendering the markers before my getBytesFromAsset method converts the assets to bitmaps. So I need a way to halt the markers' rendering until the getBytesFromAsset method converts all the assets to bitmaps.
Not sure if I'm correct but any suggestions?

Hi @R12-cookies, I believe that is what is happening.
You could create your markers before the MyHomePage widget is created, however, this is working with the assumption that the markers won't be changing over time. If that is the case, this scenario might not work out unless you apply some sort of state management to the markers.
That way you can listen for any changes to the markers and update them accordingly.
If the markers are dependent on some API call, then the state management approach will be the way to go.
That way after you've retrieved the locations for the markers, you can then update the UI with markers.
You could also look into doing the computation for getBytesFromAsset only once and storing the values somewhere so that you can use them for the rest of the BitmapDescriptors where applicable.

@danagbemava-nc I ended up converted all the assets to bmp, since it's a fast solution and it works.
Thanks for your help


Need to fix slicing for Brain_Data() instances

Can only index, but not slice (e.g., dat[2:4]).

Fixed


Does not work anymore.

I already signed up a few months ago, it worked perfectly. Since a few weeks I can't receive any messages and I always get:
To use WHIN, you must first sign up to get your service token by sending the word signup.
When I send "signup" I get the same message.

Hi, yes I can, it is 49......48914

This is related to #41 https://github.com/inUtil-info/node-red-contrib-whin/issues/41
You activated the ephemeralSetting on Whatsapp, and this is why you're not getting the signup msg processed.
You can wait till we solve this known issue, or turn off the setting.
I will leave this issue open till we complete the changes needed on the backend.
regards

@peterossege Can you please check if it works now?

It works! Thank you!

good to know.


Importing Aiociscospark:  invalid syntax

Hello Friend,
Thanks for this great library, while I'm importing the library using import aiociscospark i'm getting SyntaxError: invalid syntax i'm using Python 3.7 on a virtual env created by Pycharm
Detailed Error:
Traceback (most recent call last):
  File "C:/Users/pc/PycharmProjects/TeamsExt/TeamsExt.py", line 8, in <module>
    import aiociscospark
  File "C:\Users\pc\PycharmProjects\TeamsExt\venv\lib\site-packages\aiociscospark\__init__.py", line 3, in <module>
    from . import http_client  # noqa
  File "C:\Users\pc\PycharmProjects\TeamsExt\venv\lib\site-packages\aiociscospark\http_client.py", line 1, in <module>
    import aiohttp
  File "C:\Users\pc\PycharmProjects\TeamsExt\venv\lib\site-packages\aiohttp\__init__.py", line 6, in <module>
    from .client import *  # noqa
  File "C:\Users\pc\PycharmProjects\TeamsExt\venv\lib\site-packages\aiohttp\client.py", line 15, in <module>
    from . import connector as connector_mod
  File "C:\Users\pc\PycharmProjects\TeamsExt\venv\lib\site-packages\aiohttp\connector.py", line 12, in <module>
    from . import hdrs, helpers
  File "C:\Users\pc\PycharmProjects\TeamsExt\venv\lib\site-packages\aiohttp\helpers.py", line 33
    ensure_future = asyncio.async
                                ^
SyntaxError: invalid syntax

Hello i arrived to fix it by just updating the Aiohttp library to the latest version!


macOS ARM64 / Apple Silicon

Operating System: macOS
Revery Version: trunk
OCaml version: 4.10.1
Native, Bytecode, or JS build: Native

This is mostly to track the requirements from the Revery side to the new macOS using ARM64
Known Issues

[ ] esy-sdl2 needs to be updated

Now that we're on 4.12, the compiler is already there. Will be checking which deps are needed

seems like esy-sdl2 is still the missing piece

Yeah @EduardoRFS I made a PR in esy-sdl2 a while back to upgrade us to 2.0.14 (which worked for me on my M1 Mac Mini) -- apparently there are some issues with how I rebased with Wayland which I haven't gotten around to fixing. It seems pretty simple if you want to look at it! revery-ui/esy-sdl2#19

I also am worried that we need to rebase esy-skia as well, which might be a nightmare 😨

As a M1 owner and proud user of Onivim2, I was just wondering what is the current status for this issue? 🙏

yes, I tried to build revery in Apple M1, it failed because SKIA compiled with x86, I made a PR in esy-skia to fix that https://github.com/revery-ui/esy-skia/pull/38 with that PR I am able to build Revery natively in M1.


Fix external dependency import

Fix
Import one level deep to speed-up bundling

Wouldn’t core imports be treeshaked even when mixed?

Code style issues found in the above file(s). Forgot to run Prettier?

How can I run Prettier (my IDE doesn't support it)?

Something broken with latest husky release. For some reason precommit hooks don’t work even for me 🤷♂

On the core repository, we have yarn prettier, but I'm not sure what to use here 😁. Plus, there is no visual output I can use to patch it by hand.

For some reason precommit hooks don’t work even for me

It does run on my side, I wish it didn't run actually, it's frustrating me to have to wait a couple of seconds to be able to push.


Bug in the Cell Segmentation Example ValueError: None values not supported..

Describe the bug
I ran the cell sample from the github page and till the compile step everything is fine.
When running the compile step I get:
Error in py_call_impl(callable, dots$args, dots$keywords) :
RuntimeError: Evaluation error: ValueError: None values not supported..
A clear and concise description of what the bug is as
Detailed traceback:
Error in py_call_impl(callable, dots$args, dots$keywords) :
RuntimeError: Evaluation error: ValueError: None values not supported..
File "C:\Users\THOMAS~1\DOCUME~1\CONDA~1\envs\R-RETI~1\lib\site-packages\tensorflow_core\python\ops\math_ops.py", line 1422, in range
limit = ops.convert_to_tensor(limit, name="limit")
File "C:\Users\THOMAS~1\DOCUME~1\CONDA~1\envs\R-RETI~1\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1184, in convert_to_tensor
return convert_to_tensor_v2(value, dtype, preferred_dtype, name)
File "C:\Users\THOMAS~1\DOCUME~1\CONDA~1\envs\R-RETI~1\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1242, in convert_to_tensor_v2
as_ref=False)
File "C:\Users\THOMAS~1\DOCUME~1\CONDA~1\envs\R-RETI~1\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1296, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "C:\Users\THOMAS~1\DOCUME~1\CONDA~1\envs\R-RETI~1\lib\site-packages\tensorflow_c
The output after running yolo3
2020-10-17 19:37:03.572520: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_100.dll
2020-10-17 19:37:06.629388: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library nvcuda.dll
2020-10-17 19:37:06.667244: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: GeForce GTX 1070 major: 6 minor: 1 memoryClockRate(GHz): 1.645
pciBusID: 0000:01:00.0
2020-10-17 19:37:06.667676: I tensorflow/stream_executor/platform/default/dlopen_checker_stub.cc:25] GPU libraries are statically linked, skip dlopen check.
2020-10-17 19:37:06.668985: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2020-10-17 19:37:06.669443: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2020-10-17 19:37:06.672431: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: GeForce GTX 1070 major: 6 minor: 1 memoryClockRate(GHz): 1.645
pciBusID: 0000:01:00.0
2020-10-17 19:37:06.672801: I tensorflow/stream_executor/platform/default/dlopen_checker_stub.cc:25] GPU libraries are statically linked, skip dlopen check.
2020-10-17 19:37:06.674102: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2020-10-17 19:37:07.358505: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-10-17 19:37:07.358893: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0
2020-10-17 19:37:07.359069: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N
2020-10-17 19:37:07.360054: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6794 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
The code i ran
library(tidyverse)
library(platypus)
library(abind)
BCCD_path <- "BCCD/"
annot_path <- file.path(BCCD_path, "Annotations/")
blood_labels <- c("Platelets", "RBC", "WBC")
n_class <- length(blood_labels)
net_h <- 416 # Must be divisible by 32
net_w <- 416 # Must be divisible by 32
anchors_per_grid <- 3
blood_anchors <- generate_anchors(
anchors_per_grid = anchors_per_grid, # Number of anchors (per one grid) to generate
annot_path = annot_path, # Annotations directory
labels = blood_labels, # Class labels
n_iter = 10, # Number of k-means++ iterations
annot_format = "pascal_voc", # Annotations format
seed = 55, # Random seed
centroid_fun = mean # Centroid function
)
blood_yolo <- yolo3(
net_h = net_h, # Input image height
net_w = net_w, # Input image width
grayscale = FALSE, # Should images be loaded as grayscale or RGB
n_class = n_class, # Number of object classes (80 for COCO dataset)
anchors = blood_anchors # Anchor boxes
)
blood_yolo %>% load_darknet_weights("yolov3.weights") # Optional
YOUR CODE HERE

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Session information (please complete the following information):

OS: [e.g. iOS]:
R version:
Python version:
TensorFlow (Python) version (tensorflow::tf_version()):
R session information (sessionInfo()):

PASTE RESULTS FROM sessionInfo()

Additional context
Add any other context about the problem here.

Was a general problem with my keras installation


Proposed 7 very important Algorithms.

A small step in hactoberfest.
Pls consider my PR.
What type of PR is this? (check all applicable)

[ ] 🚀 Added Name
[ ] ✨ Feature
[ ] 🌟 stared the repo
[ ] 🐛 Grammatical Error
[ ] 📝 Documentation Update
[✓ ] 🚩 Other

Description
This is exclusively for hactoberfest.
Add Link of GitHub Profile
https://github.com/Darkcoder-paditya

@keshavsingh4522


Embed functionality

All 3 embed views now have functionality.
Still need to fix up style and export button.

@devoncarew @lukechurch

All 3 embed views now have functionality.

woot! Can you have the export button display an 'under construction' dialog?
lgtm

@devoncarew
Added!
We still have some issues with styling.
I have fixed an issues with the location of the error toasts - though we need to decide where to place the errors.

Merging


Ipf v4.0.2

End to end path had a lot of changes.
I separated the Unicast, Multicast and Gateway searches since they all take different options.
Added support to do a second query to pull SVG data along with the JSON.  SVG can be written directly to a file and saved or most likely displayed in a browser.
I tried PNG but was getting 500 errors.
Added some IP address/subnet checking and error handling.

Need to figure out version changing since this is not backwards compatible with IPF < v4


A Memory Leak in PetClinic

Hey,
I am running PetCinic in VM, and I am having a memory leak. Is there any thought as to why it happens and how to solve it?
Thanks,
Hammam

Could you please help us to identify the memory leak? Wich objects remains in memory? And do you have used a profiler?

Without any feeback, I close this issue.


DON-667: Declare compatibility with Angular 15 as well as 14 to allow…

… frontend to upgrade

@thebiggive/devs the syntax is || I think? https://semver.npmjs.com/
It's also normally a better idea IMO to use the package manager's CLI commands to make any changes if you're not sure. Then you know if the syntax works before committing.


You can create an organization user without password

You can create an organization user without password. Is this somehow useful (bug or feature :D), @iriberri ?

Wow... I'd say bug too :-)
If we're not generating a random password and sending it to the user via email or something, and then making the user change it in the first log in (very complex solution at the moment, I guess), creating a password should be mandatory.
This way we avoid subusers to go through the reset password process, which is kind of ugly for the onboarding of a new user.

Closed, you can't do this anymore.


Moment.js is now in legacy/maintenance mode

Relevant read: https://momentjs.com/docs/#/-project-status/
In some areas of the SDK, moment and moment-timezone are not being used in integral-and-non-replaceable manners and could potentially be easily replaced (the SDK should not be shipping any public dependency on moment). However, this may not be the case in some of the newer Adaptive-related libraries. @boydc2014, @chon219, or @Danieladu are more familiar with these libraries and probably have invaluable insight here.
In addition to Moment.js' legacy status, size of the package is a nudge to consider moving from moment if possible.
The Moment.js team has recommended a few alternatives to moment that should be explored (including moment/luxon and Day.js).

Attach the guideline and practice https://github.com/you-dont-need/You-Dont-Need-Momentjs/blob/master/README.md


set_altitude, set_azimuth were disabled by new spec code

The new spec code in surf and mesh disables set_azimuth and set_altitude because it causes the azimuth and altitude fields in page to be ignored.
These fields and functions are recent additions.  The functions set azimuth and altitude fields in page that had been given default values by _create_page.   Now the fields in the page are set from within surf and mesh by
  p.altitude <- _get_altitude spec 33.;
  p.azimuth <- _get_azimuth spec 115.;

so if these values were changed by the set_ functions, the values are ignored.  Maybe something like this would be better?
  p.altitude <- _get_altitude spec p.altitude;
  p.azimuth <- _get_azimuth spec p.azimuth;

_create_page is setting the defaults to 33. and 115. respectively, so the behavior shouldn't change in other respects.
I thought about removing the set_ functions, but I think it's useful to be able to set the altitude and azimuth and have the new values apply by default to a series of surf or mesh calls.
CORRECTION: That doesn't really work because each subplot or plot page has a new page structure, so the azimuth and altitude don't persist between calls to surf and mesh, unless one is writing multiple 3D plot calls on one subplot, which would be rare enough that it wouldn't matter.
So maybe set_altitude and set_azimuth should be removed, and the default values left in hardcoded surf and mesh?

p.altitude <- _get_altitude spec p.altitude;
p.azimuth <- _get_azimuth spec p.azimuth;

Yes, this is better than my current code so I have changed it, many thanks.

> So maybe `set_altitude` and `set_azimuth` should be removed, and the default values left in hardcoded `surf` and `mesh`?

I agree that it is not very common to call 3d plot multiple times in one figure, so I have removed the `set_altitude` and `set_azimuth` for the time being. But it can be added back in future whenever the usecase becomes clear.


espcoredump.py & xtensa-esp32-elf-gdb not working in docker (IDFGH-3297)

Environment

Environment : WSL2 (ubuntu18.04)
Docker image : release/v4.0 & latest

Problem Description
My goal is to use the docker container to process coredump.
Running espcoredump.py lead to
================== CURRENT THREAD REGISTERS ===================
Traceback (most recent call last):
  File "/opt/esp/idf/components/espcoredump/espcoredump.py", line 1190, in <module>
    main()
  File "/opt/esp/idf/components/espcoredump/espcoredump.py", line 1185, in main
    operation_func(args)
  File "/opt/esp/idf/components/espcoredump/espcoredump.py", line 1065, in info_corefile
    p = gdbmi_getinfo(p, handlers, "info registers")
  File "/opt/esp/idf/components/espcoredump/espcoredump.py", line 979, in gdbmi_getinfo
    p.stdin.write(bytearray("-interpreter-exec console \"%s\"\n" % gdb_cmd, encoding='utf-8'))
BrokenPipeError: [Errno 32] Broken pipe

Trying to run xtensa-esp32-elf-gdb by itself is not working inside the container :
root@40e761ae8a4a:/project# xtensa-esp32-elf-gdb
xtensa-esp32-elf-gdb: error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No such file or directory

which is probably the cause of the espcoredump.py error.
Steps to reproduce
Run :
> docker run --rm -v $PWD:/project -w /project -t espressif/idf:latest xtensa-esp32-elf-gdb
> docker run --rm -v $PWD:/project -w /project -t espressif/idf:release-v4.0 xtensa-esp32-elf-gdb
> docker run --rm -v $PWD:/project -w /project -t espressif/idf:release-v4.0 python /opt/esp/idf/components/espcoredump/espcoredump.py info_corefile  /project/device.elf -c /project/coredump.bin -t raw

@vtunr Thanks for reporting, we will look into.

I have just hit the same error from xtensa-esp32-elf-gdb running on a clean Kubunutu 20.04 with stable esp-idf.  It appears to be trying to load python 2.7. This system has 3.8.2 installed by default.  Really don't want to have to install 2.7 to be able to run GDB?

@vtunr @PaulMichell We will switch to building GDB with Python 3 libraries in the next toolchain release. For now there are two options:

Build the toolchain (including GDB) from source, either disabling Python scripting support, or configuring it to use Python 3: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/linux-setup-scratch.html
Install Python libraries: apt-get install libpython2.7-dev (do apt-get update first, if running inside IDF Docker)

Sorry for the trouble.

We will switch to building GDB with Python 3 libraries in the next toolchain release.

ETA?

We will switch to building GDB with Python 3 libraries in the next toolchain release.

This turned out more complex than expected, as libraries for different Python 3.x versions are not binary compatible. So if we build GDB, say, with Python 3.7 libraries, it won't work for users who have 3.8 installed.
It is possible to link Python statically, but that increases the toolchain distributable size.
For now, we have merged #5817 to add libpython2.7 to the docker image. This is also in the process of being backported to other release branches.

It is possible to link Python statically, but that increases the toolchain distributable size.

Bigger toolchain is less worse than a broken one, no?

It is possible to link Python statically, but that increases the toolchain distributable size.

Bigger toolchain is less worse than a broken one, no?


feat: not return error when getting HTTP 204 no content responses

Hello @FredrikOseberg @sighphyre, hope you are doing well.
We have got PollerError.data when starting the poller intermittently due to getting HTTP 204 no content responses from the feature flag proxy.
The mechanism on our web server is basically the first time a client makes a request to the server it returns an etag header which is a hash of the resource content. In order to save bandwidth on following requests it will compare this hash to see if any content has changed and if not the web server does not resend a full response as the client already has a copy cached from the first request. The SDK throws data error when the response is 204 no content in func start(completionHandler: ((PollerError?) -> Void)? = nil) -> Void.
This PR treats 204s response as successful requests by adding a checker in the session response. I hope this makes sense for you and please feel free to let me know any feedback or recommended approaches for this case.
Thanks a lot.
Mickey

Hi Mickey,
I can definitely see how that would be a huge pain to deal with!
I'm sure there's an excellent reason or it can't be changed, but would a 304 not be a more appropriate response from the server? That's already a handled case here
Aside from that, I agree in principle that 204s should not raise an error but I think that rather than treating them as a special case, we should make the poller tolerant to an unhanded response code rather than raising an error. There aren't a lot of reasonable things the poller can do with a junk response aside from try again later and crashing the poller (if I'm reading your issue correctly) is not a friendly thing to do to consumers. I want to say that if we get an HTTP response that we don't recognise and can't handle we should log and swallow that, which is pretty much what we do in the JS version of this. I think that should make your issue go away? WDYT?

Hi Simon,
Thanks for having a look. I have updated the PR with your suggestion. Now the SDK swallows the empty response situation and logs it. Let me know if any problems 🙂

@sighphyre Thanks for the approval. Would you mind tagging a new release 0.0.5?

Hello @sighphyre, that's pretty fair enough. We could wait for one to two weeks for the proper release. Thank you very much.

Hello @sighphyre, may I know is there any updates about the release? Thanks

Hey @mickeyleel, apologies, we've all been on a company offsite. Testing is scheduled for this afternoon so it that goes smooth it should be out before end of day

Hey @mickeyleel, tested and released, thanks so much for the contribution!


Istio metrics showing the cause for 403s (app or authorizationpolicy)

Describe the feature request
403 responses can come from AuthorizationPolicy, ext_authz and the application.
To detect authorization policy issues (and attacks), it would be nice to have a prometheus metric that shows that a 403 comes from the AuthorizationPolicy and not the destination service pod itself.
For ext_authz this is visible in the response flags  as UAEX (UnauthorizedExternalService)
For application 403s and generic AuthorizationPolicy denies (match none) the response flag is - so it is not possible to know what kind of response this is.
Describe alternatives you've considered
The logs output contains rbac_access_denied_matched_policy[none], but it does have the container source workload namespace, and metrics would be nice to have anyway.
Affected product area (please put an X in all that apply)
[ ] Ambient
[ ] Docs
[ ] Dual Stack
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[x] Extensions and Telemetry
[x] Security
[ ] Test and Release
[x] User Experience
[ ] Developer Infrastructure
Affected features (please put an X in all that apply)
[ ] Multi Cluster
[ ] Virtual Machine
[ ] Multi Control Plane
Additional context

@yangminzhu do you think this is possible? Small or large change?
I tried to track down the code for UAEX and got to
https://github.com/envoyproxy/envoy/blob/28e383ec59a824887bb66718e7386d6023740f00/source/extensions/filters/http/ext_authz/ext_authz.cc#L424-L429
So maybe this requires a change in Envoy, not Istio?


Add tags to widgets and textboxes

What type of PR is this?

[x] Feature

Description
This change adds the possibility to filter the widgets to show according to their tags ,still with the help of an allowed_widgets query results that has for each parameter value , the list of allowed tags.

This was introduced by doing changes both in backend and in frontend.
In backend :

We updated the database by introducing the metrWidget table which is related to the widget with a 1 to 1 relationship , this allows us to avoid conflicts in the future, the added metrWidget has the list of tags of the widget.

We added an endpoint that allows to save the tags, while saving we check if the widget does not have yet a metrWidget we create on the fly, this way old existing widget won't have an issue.

Even if the used url sends the widget id (widget_id), we are having it as api/metrwidgets ... to avoid conflicts

In frontend :

we  keep the usage of allowed_widgets_{dashboard_id}
we update the widget component to have the tags (widgets with visualization or textboxes )
we update the widget service by adding the call that updates the tags related to a widget
we show the tags control in the edit mode only (while editing a dashboard)

How is this tested?

[x] Unit tests (pytest, jest)
[x] Manually

Related Tickets & Documents
This PR closes https://github.com/metr-systems/backlog/issues/3436
Mobile & Desktop Screenshots/Recordings (if there are UI changes)
This has been deployed to staging and accessible here https://dashboard.staging.metr.systems/staging/

Awesome work! Thanks for looking into my suggestions even though I wasn't so sure they were possible :)
I'm already approving, but would be cool to add the tests I mentioned before

Thanks, i added the tests here in the latest commit

Just as a note here:
I was digging deeper into making the permissions tests better , my plan was to have something similar to the following (or more detailed) in order to have track exactly of the permissions checked
  def test_post_allowed_if_user_has_permission_to_edit_dashboard(self):
      # in other words if admin user
      widget = self.factory.create_widget()
      user = self.factory.create_admin(org=self.factory.org,group_ids=[self.factory.admin_group.id, self.factory.default_group.id]) # admin user

      self.assertTrue(user.has_permission("edit_dashboard"))
      self.assertTrue(can_modify(widget.dashboard, user))

      response = self.make_request("post",f"/api/metrwidgets/{widget.id}/tags",data= {"tags":["tag1", "tag2"]}, user=user)
      self.assertEqual(response.status_code, 200)

but since the authentication happens inside the self.make_request method, can_modify fails if i try to check on it earlier


Look into #= operator

Might be a way to get rid of the calculation block, and make math formula relational. Might also be a way to reimplement the date library in a relational way.

The problem is that a constraint equation is a relationship between three values, and I don't have any way to conclude a value between three entities, so there is no way to reverse through it. I'm also getting some sort of existence errors. I could keep playing with it, but it doesn't seem like something that is going to have a quick conclusion without maybe violating the unary/binary structure of the code. I feel like maybe this would make more sense inside the date library first, so be able to reverse certain kinds of date equations. I think I'm going to push this back to do eventually, because I don't think it's going to help a lot immediately.

Also, it uses rational numbers, which means that we need to figure out a way to translate from rational number formats in the explanations.


omegaconf  library doesn't work

Dear Sir/Madam,
I got the error message as the following when following your tutorial steps:
ImportError: cannot import name 'Config' from 'omegaconf' (/Users/youyin/.local/lib/python3.7/site-packages/omegaconf/init.py)
Can you send me your requirements.txt or any other library to help me resolve it, many thanks.

Hi @1993ryan, could you let me know in what step you encountered this error? It works with no problem on my machine when following the setup.


OPENAI_API_BASE_URL with port - semi colon issue

Describe the bug
in the keys.cfg file, the line below is not supported:
OPENAI_API_BASE_URL:http://<IP_ADDRESS>:4000
Steps/commands/code to Reproduce
OPENAI_API_BASE_URL:http://<IP_ADDRESS>:4000
Error message/results
app-1     | During handling of the above exception, another exception occurred:
app-1     |
app-1     | Traceback (most recent call last):
app-1     |   File "/app/sweagent/utils/config.py", line 45, in __init__
app-1     |     self._keys_cfg = config_file.Config(str(keys_cfg_path))
app-1     |   File "/usr/local/lib/python3.9/site-packages/config/__init__.py", line 707, in __init__
app-1     |     self.load_file(stream_or_path, kwargs.get('encoding'))
app-1     |   File "/usr/local/lib/python3.9/site-packages/config/__init__.py", line 816, in load_file
app-1     |     self.load(f)
app-1     |   File "/usr/local/lib/python3.9/site-packages/config/__init__.py", line 807, in load
app-1     |     raise cfe
app-1     | config.ConfigFormatError: Unexpected ':' at (2, 25)
app-1     |
app-1     | The above exception was the direct cause of the following exception:
app-1     |
app-1     | Traceback (most recent call last):
app-1     |   File "/app/run.py", line 44, in <module>
app-1     |     from sweagent.agent.agents import Agent, AgentArguments
app-1     |   File "/app/sweagent/agent/agents.py", line 16, in <module>
app-1     |     from sweagent.agent.models import (
app-1     |   File "/app/sweagent/agent/models.py", line 22, in <module>
app-1     |     from sweagent.utils.config import keys_config
app-1     |   File "/app/sweagent/utils/config.py", line 85, in <module>
app-1     |     keys_config = Config()
app-1     |   File "/app/sweagent/utils/config.py", line 48, in __init__
app-1     |     raise RuntimeError(msg) from e
app-1     | RuntimeError: Error loading keys.cfg from /app/keys.cfg. Please check the file.

System Information
ubuntu with dockers
Checklist

[X] I'm running with the latest docker container/on the latest development version
[X] I've searched the other issues for a duplicate
[X] I have copied the full command/code that I ran (as text, not as screenshot!)
[X] If applicable: I have copied the full log file/error message that was the result (as text, not as screenshot!)
[X] I have enclosed code/log messages in triple backticks (docs) and clicked "Preview" to make sure it's displayed correctly.

I believe you're missing the quotation marks.


Replace TockValue's auto-derived Debug implementation with a hand-written implementation optimized for size.

For Google's h1b_tests application, this saves 9440 bytes in .text. The caveat is that invoking Debug on a TockValue will not format the value contained in the TockValue::Expected().

This improves the situation described in #84

The serde version and Debug changes should be independent -- I'll separate the serde changes into separate PRs shortly.

The serde version fix has been separated into #86 .


Dilato: Core: There is no file name in the working files panel when remote a URL with / at the end for the first time

Steps:

Launch Brackets 1.13
Press ctrl+shift+O for win and cmd+shift+O for mac
Input a URL with / at the end, such as "https://www.adobe.com/"
Press ctrl+shift+O for win and cmd+shift+O for mac
Input another URL with / at the end, such as "https://stock.adobe.com/"

Result:
After step 3: there is no file name in the Working files panel
After step 5: the file name appears, but there is no color change when select one
Expected: There is always a file name for the URL, and it could change the color when select them
After step 3:

After step 5

Normal:

Build: 1.13.17668-1023

@RuchiAgrawalAdobe, Please verify and close this issue.

Verified this issue on build 1.13.0-17688 Fixed


catch all filter

Hi, is there a way to add threadfin custom filter that adds all channels in playlist?

ah it was easy as single space (hit on space bar one time)


Usage in a Vite/Vue App

Hey, I am currently trying to implement the magic link login in a Vite/Vue app.
As soon as I install the npm package I get the following error trying to run it in dev mode (before adding any other code):

The thing is, all of those react packages are already in the node_modules folder before adding it.
I have tried to add react, react-dom, etc to the "external" build option and ssr option. But no improvements.
Any idea what I am doing wrong? Is it simply not possible to use this in a Vite/Vue app?

I cant share much code from the project as its not public, but I dont even have to add any code for the error to arise. Its a vue3 app running on vite with a pinia(ts) store for auth.
As soon as I npm -i amazon-cognito-passwordless-auth it stops working with the error I posted above.
Im on node 16.16, I dont know if that matters?
Here is a fuller error:

I have tried adding anything react into build.rollupoptions.external and ssr.external, but no change.

Mmmm. This is little to work with for us but it looks like it's an interplay issue with something else. Did you expect to see react-redux in your dependency tree? Since you mentioned Vue and pinia.
The lib here is not installing react-redux.
The error you see is from esbuild not rollup, so you may need to tweak esbuild config via vite options.
When do you see the error. From your screen shots it is not even clear this is an issue with the lib here.

Hey there! Feel free to close it (Ill leave it up to you). The temporary solution of adding react-dom and excluding it from all builds is working fine atm.
Tho the error only appears after adding this library. So I dont really know whats going on to be fair. Haha.

Ok sure. Will close for now as I don't see leads to pursue currently. Should they arise we'll re-open


Update network.go for broken docs link

The old URL https://docs.starknet.io/documentation/useful_info/ is returning 404 now.

It should be changed to https://docs.starknet.io/documentation/tools/important_addresses/, this PR does this changes.

By the way, should we remove all Goerli related content too, as starknet states:

Goerli testnet support is now removed.

Closing the PR since corresponding changes already in main


Preinstall pip

pip: command not found

pip is popular for installing Python apps (such as the AWS CLI)
sudo apt-get install -y python-dev
wget https://bootstrap.pypa.io/get-pip.py
python get-pip.py

Wouldn't mind it, if it conforms to https://github.com/bitrise-io/osx-box-bootstrap/blob/master/PULL_REQUEST_TEMPLATE.md feel free to send a PR! ;)

Just a note, python-pip is available in apt. In one of our projects we do:
sudo apt-get install -y python-pip python-dev build-essential

that should be enough for PIP

That said, I'd rather have this in the base docker image (https://github.com/bitrise-docker/bitrise-base), as this is not android related

Ok, I'll move it over there.


Update Wiki Getting Started

Create a set of instructions for Front End Devs to help them get started with development.

First draft is done.


Vendor initializers

This should fix the build on GCC 4.9 - apparently my previous syntax was technically legal (maybe) but the new syntax is more legal and more simple. Also more commented.
cc @JeroMiya

Compiles and links on Windows, Mac and Linux.  Works to open at least the DK2 raw VRPN device on Windows in DirectMode.


SyntaxError: Could not find export 'webcrypto' in module 'crypto'

Is this related to https://github.com/awslabs/llrt/issues/184? If so, this can be closed. Otherwise, I'm a little confused :)
I get this when running locally on Fedora Kinoite x86_64 with llrt-linux-x86 binary. I also tried the lambda-arm64 binary within arm64 lambda but there I get the following:
2024-08-11T19:10:15.339Z	n/a	FATAL	TypeError: String contained internal null bytes: nul byte found in provided data at position: 7

INIT_REPORT Init Duration: 24.23 ms	Phase: init	Status: error	Error Type: Runtime.ExitError
2024-08-11T19:10:52.147Z	n/a	FATAL	TypeError: String contained internal null bytes: nul byte found in provided data at position: 7

INIT_REPORT Init Duration: 25.43 ms	Phase: invoke	Status: error	Error Type: Runtime.ExitError
START RequestId: XXX Version: $LATEST
RequestId: XXX Error: Runtime exited with error: exit status 1
Runtime.ExitError
END RequestId: XXX
REPORT RequestId: XXX	Duration: 26.40 ms	Billed Duration: 27 ms	Memory Size: 3008 MB	Max Memory Used: 23 MB

To note, I'm trying to run SvelteKit using LLRT for improved cold starts, I bundled the JS as per instruction: ES2023

Hi, yes it's related to https://github.com/awslabs/llrt/issues/537.
ES2023 is in upcoming 0.2-beta. Will release next week :)


About upgrading mediainfo and its related lambda layer

Have you ever planned to upgrade https://github.com/awslabs/media2cloud/tree/master/source/layers/mediainfo (linux binary / windows DLL) and the lambda layer sources leveraging on it ?
The version of https://github.com/awslabs/media2cloud/tree/master/source/layers/mediainfo/amazon/linux2/mediainfo is currently 0.7.99
Whereas the latest official version of https://mediaarea.net/download/binary/mediainfo/20.08/MediaInfo_CLI_20.08_Lambda.zip is 20.08
Then the (new) XML format produced from running source/layers/mediainfo/index.js would not be compliant anymore. Javascript execution of such a lambda layer will get broken as a consequence because of unexpected XML output produced by https://mediaarea.net/download/binary/mediainfo/20.08 executable.
The reason why I wonder if updating mediainfo and especially its backed-up lambda master/source/layers/mediainfo/index.js was in your roadmap or in AWSLABS plans ?

@hmachefe,
We are qualifying the latest mediainfo v20.08 and updating the nodeJS codes. Will publish an update when it passes the test.

Great, good news, your prompt and smart feedback is greatly appreciated. Any idea about when it will be published approximately ? Next month ? maybe earlier ? Just to have an estimation

Hi @hmachefe,
v20.08 unit test is looking good. I am testing the M2C state machines and web app where it uses the mediainfo. Will keep you updated when the code is ready.
Just curious if you have any AWS contact (Account Manager or Solution Architects) who works with you on this?

Excellent, sounds great. (We are impressed). OTH I am afraid we don't have any AWS contact (Account Manager or Solution Architects) who works with us on this. I first contacted AWS support team who gently redirected our original requests to you.

Hi @hmachefe,
Updated the code to use v20.08 mediainfo.


Add SHARP_DIST_AUTH_HEADER to pass basic authorization for libvips download

Fixes #938
Setting SHARP_DIST_BASE_URL="https://username:password@hostname/path/" doesn't work as the got library throws the error: Error: Basic authentication must be done with auth option. This pull request allows the user to specify SHARP_DIST_AUTH_HEADER="username:password" which will set the auth header on the got request for libvips.

Coverage remained the same at 99.319% when pulling 9f42e4ae4c7f92c1ef06e93aff718d175bc36152 on pbomb:dist_auth_header into 5f29d1ba9c2ca89b3e1e166e2747b99e7c6622b9 on lovell:master.

I think I'd prefer to have this parsed from the existing single URL. The switch to simple-get discussed in #938 should achieve this.

Closing in favor of #945


ugrep: 3.9.2 -> 3.9.3

Description of changes
https://github.com/Genivia/ugrep/releases/tag/v3.9.3
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 208455 run on x86_64-linux 1

1 package built:

ugrep


Documentation to get access token for Service principal missing

The above documentation says that you can generate access token for PowerBI using service principal and the example shows C# code to do it. But I could not find documentation on how to generate the token if I'm not using C# library. Could you please help me find it?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5a1179b0-7f3d-77de-239b-a33c9d30ff61
Version Independent ID: ab708f2a-3cb5-fcf3-9733-a2bc26b09e60
Content: Authenticate users and get an Azure AD access token for your application - Power BI
Content Source: powerbi-docs/developer/get-azuread-access-token.md
Service: powerbi
Sub-service: powerbi-developer
GitHub Login: @rkarlin
Microsoft Alias: rkarlin

Hello, I had the same problem, but now I have an example using PHP (based on this link and this other link). This is the code:
$clientId = 'YOUR_APP_ID';
$clientSecret = 'YOUR_APP_SECRET';
$urlAuthorize = 'https://login.microsoftonline.com/common/oauth2/v2.0/authorize';
$tenantId = 'YOUR_TENANT_ID';
$urlAccessToken = "https://login.windows.net/$tenantId/oauth2/token";
$scope = 'openid';
$resource = 'https://analysis.windows.net/powerbi/api';

$oauthClient = new GenericProvider([
    'clientId'                        => $clientId,
    'clientSecret'                    => $clientSecret,
    'urlAuthorize'                    => $urlAuthorize,
    'urlAccessToken'                  => $urlAccessToken,
    'urlResourceOwnerDetails'         => '',
    'scope'                           => 'openid'
]);

$accessToken= $oauthClient->getAccessToken('client_credentials', [
	'resource' => $resource
]);

$token = $accessToken->getToken();

I hope this helps.

@jesusvpct Thanks a lot. This does help. It looks like I'm supposed to use OAuth2 Client Credential flow to generate the token.

Thank you for your question (and the helpful reply). I've assigned this to the writer to do a doc enhancement per your request. We are in between writers right now so the one who's covering might not get to it right away.

@kfile Thanks Kelly. I guess if the documentation mentioned that you have to use OAuth2 Client Credential flow, it would have given me some direction on how to use the service principal.

Understood. I think that will be clear to the writer.

same issue here. We are trying to implement in PHP and can't rely on REST sequence diagram or something like that to fetch our first data (client app auth - SP > fetch first report or data)

OK thanks @ka2er

Thanks for your comment, everyone This looks like a support request rather than documentation feedback – you can get support by going to https://support.powerbi.com. You might find a solution in the Power BI Community, found at https://community.powerbi.com.
I would add your suggestion to the ideas page.

We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion, and we will reopen the issue.


Thoughts on support for zarr storage format? And other Awkward-storage interaction.

I'm researching what is the best storage format for AwkwardArray (audio data) and Pandas DataFrames. My requirements are sliced access to arrays, which HDF5 seems best for.
However, in a multi-process / multi-machine setting HDF5 is very limited in terms of reading, and especially writing: https://github.com/h5py/h5py/issues/1459
I looked at ROOT, but since I'm not in the HEP field, it is quite a huge install for just I/O access.
It also seems not be too fast for reading? Since people are recommending temporary storage in HDF5 / .npy: https://stackoverflow.com/questions/58817554/what-the-fastest-most-memory-efficient-way-of-opening-a-root-ntuple-for-machine
You also mention Zarr as possible solution.
Is there any intention to support Zarr?

I met with the Zarr authors in September to talk about setting up a two-way connection. Their primary focus is genetics, but just means they need fast access to large arrays like the rest of us. There is a peculiarly to that focus, however: their datasets of interest are mostly flat arrays with a possibly jagged inner dimension. The datasets I'm interested in (from a particle physics perspective) make heavy use of record structures with named fields.
If you have truly flat arrays (no records and no jaggedness), you can get away with a lightweight option. NumPy's npy/npz files are about as fast as you can get because there's no translation between disk and memory. If your data are highly compressible (such that decompression +reading fewer bytes is faster than reading more bytes), you can use npz's built-in zlib compression. HDF5 is for big, flat arrays, but it's a heavyweight option with a lot of knobs to tune before it gets efficient. But once it is properly tuned—it's what supercomputers use in HPC.
Another lightweight, flat array option is blosc/bcolz, which does the decompression on demand—it has such a fast decompressor that it stores the arrays in memory in compressed form and decompresses just before each calculation. The idea is that you transfer fewer bytes from main memory to your CPU cache (same idea as the above, but for the memory-to-cache transition, rather than the disk-to-memory transition).
The handling of jaggedness in Zarr isn't very efficient yet (and the users in genetics are avoiding it), but that will hopefully change in the upcoming year. ROOT pioneered jaggedness-handling, with records, but now Parquet does the same job. The Arrow library for Python, pyarrow, has the best Parquet reader/writer.
At larger scales, you might consider object stores, each with a column of data in it, though that means more manual work doing conversions if you need to turn the flat arrays into structures. Awkward 0.x has a protocol for saving full awkward arrays in flat-array (or raw-blob) backends, and thus you can save awkward arrays in npy/npz files and HDF5. Upgrading that to Awkward 1 isn't a high priority, though.
Another thing that you should know: ROOT is revamping its storage format to take advantage of the new ecosystem—a class names RNTuple will be replacing TTree. The new RNTuple code is being developed in such a way that it can be used separately from the rest of the ROOT framework. You might be interested in that, though it's not ready for general users yet.

Thank you for the elaborate reply and various suggestions.
In my case I deal with audio of varying length (most of the time), hence numpy is too limiting for a one-in-all solution for my purpose. Also, I need lazy loading (sliced access), and it seems that loading a .npy file would load all data into memory, right?

Awkward 0.x has a protocol for saving full awkward arrays in flat-array (or raw-blob) backends, and thus you can save awkward arrays in npy/npz files and HDF5. Upgrading that to Awkward 1 isn't a high priority, though.

Actually I already started to build my database with HDF5, since it can store AwkwardArrays and I assumed this was still possible from 1.x forward (would be good to clarify this in the Awkward 0.x docs?). I'm glad I started looking for other solutions, due to the parallel writing/reading problems of HDF5. Otherwise I would have been stuck with Awkward 0.x :')

ROOT pioneered jaggedness-handling, with records, but now Parquet does the same job. The Arrow library for Python, pyarrow, has the best Parquet reader/writer.
I've started experimenting with Parquet now. Although in their documentation it is all about DataFrames, with AwkwardArray It seems to do the job ^-^ It's unfortunate that Parquet cannot include metadata like HDF5, but the integration with Arrow is a nice thing.

ROOT is revamping its storage format to take advantage of the new ecosystem
This sounds promising! Once this new format is ready, what would be the advantages / disadvantages compared to using Parquet instead?

The difference between NumPy's npy and npz is that npz is a collection of npy binary blobs in a ZIP file. Each of those binary blobs can be separately loaded. If you have 1-dimensional audio arrays, each can be a different length and separately loaded by putting them all in an npz file. The granularity of lazy-loading is one binary blob (array), though there can be many in the file.
Also, if you put uncompressed data into npy files, they can be lazily loaded with finer granularity by loading that npz with numpy.memmap. This uses the operating system's own virtual memory, which lazily loads in chunks of (probably) 4 kB. This happens whenever you read a file anyway (it's the operating system's cache); memory-mapping lets you access that cache directly, which can be good for sequential access.
Zarr gives you more control over the lazy-loading, with some advantages over HDF5. (Maybe parallel reading? I'm hazy on the details.)
You can definitely store flat arrays in Parquet. The pyarrow library has a to_numpy (or similar) method to turn its internal array format into NumPy. You can definitely get at the data from a pyarrow.Array by passing its buffers into numpy.frombuffer (with the appropriate dtype), but that may be too low-level.
I don't see the need for jagged arrays in your case because even though every audio sample is a different length, a single audio sample is simply one-dimensional, right? If your audio samples are reasonably large (kB to MB or more) and you don't have too many of them (thousands, not millions), then you can efficiently work with them using a separate NumPy array for each audio sample. Unless this isn't your case, you might be overthinking it.
The persistence layer for awkward will be reproduced in version 1.0, but not right away. (It's not one of the priority items. We should have old-to-new and new-to-old conversion functions to help with the transition, though.)

Thank you for explaining the difference between npy and npz, and helping me brainstorm.
I think I do need JaggedArrows / lazy loading. My audio data is 2D (2 channels), can be 5 min ~ 30 min, and I want to access them database-like. Sometimes I need only channel 0, other times both channels. Sometimes I need the full length, sometimes only the first 10 seconds. Sometimes I need to iterate over all audio samples, sometimes only a selection (e.g. indexed selection). This is all possible with JaggedArrows (I thought while I was typing this...):
# only channel 0
jarr_2d[:, 0]
# <JaggedArray [[0 1 2] [6 7] [10 11 12 13]] at 0x7f0943cb1710>

# iterate over selection (array 0 and 2)
selection_idx = np.array([0, 2])
for jarr in jarr_2d[selection_idx, :]:
    print(jarr)
    print(type(jarr[:]))
    # how do I get a 2D numpy array back though?
# [[0 1 2] [3 4 5]]
# <class 'awkward.array.jagged.JaggedArray'>
# [[10 11 12 13] [14 15 16 17]]
# <class 'awkward.array.jagged.JaggedArray'>

# first 2 values only
jarr_2d[:, 0, :2]
# <JaggedArray [[0 1] [6 7] [10 11]] at 0x7f0943cb76d0>

# PROBLEM: 3D slice not possible
jarr_2d[:, :, :2]
# NotImplementedError: this implementation cannot slice a JaggedArray in more than two dimensions

When using .npy, I would need to create many files, and with .npz I would be forced to use a for-loop over selection_idx for returning the audio samples.

Seems I misunderstood something while trying it out.
Apparently I can not do a selection like "take first x elements of all 2D arrays" jarr_2d[:, :, :2] -> Not implemented error
Are there any plans to implement this?
Is it possible to store a 2D numpy array (of the same length)? As in not:
jarr_2d[0, :]
# <JaggedArray [[0 1 2] [3 4 5]] at 0x7f0943d11750>

but a return like this (not actual working code):
jarr_2d[0, :]  # not correct
# array([[0, 1, 2],
#        [3, 4, 5]])

See the page labeled "9/16" (page 18 in the PDF) here: https://indico.cern.ch/event/773049/contributions/3473258/ The inability to generalize was exactly why I need to rewrite it. Each depth of nested slicing was a separate implementation using different NumPy tricks, and so I only went up to 2. Now that I can write for loops in C++, I've solved the problem of slicing arbitrary depths with a recursive function.
Although I like promoting the use of Awkward, I also recommend the simplest solution to a problem. If you have a lot of 2-channel audio, why not save each audio clip in a separate two-dimensional array file (in which the second dimension has length 2)? These "files" may be separately loadable binary blobs in a NumPy npz file. If you want large audio clips to be only partially loaded, then make your database a collection of real files, possibly npy, in a directory and memory-map them. Or use Zarr or HDF5. Or Dask. I don't see here any intrinsic need for jaggedness. (There are other applications that really need it, and I obviously like finding other such cases, but I honestly don't see it here.) There's no reason, for instance, to make all of your samples a single array.
Assuming high-fidelity audio samples for numeric processing, not just listening, such as 2 channels of 64-bit floating point numbers at 48 kHz, 5 minutes is 220 MB and 30 minutes is 1320 MB (48000 × 60 × #minutes × 2 × 8 / 1024 / 1024). On a computer with multiple GB of RAM, you can load several of these in their entirety at any given time and do computations across them (adding or correlating or whatever). Also, the smallest ones are big enough that if you do a Python for loop over audio samples, you will not be dominated by Python overhead. Your data's already in a sweet spot where each unit is big enough to put into Python variables and small enough to fully load into memory (although if this is a problem, memory-mapping, Zarr, HDF5, or Dask).

Thank you for linking that PDF and the other approaches. It was very informative.
I still think AwkwardArray is beneficial in my situation, even for things not directly related to performance (code simplicity/flexibility, selecting relevant arrays with numpy arrays (instead of strings), improvements with Awkward 1.0 (3D+ array slicing), etc).
I didn't fully sketch my current situation, because that is out of the scope of this GitHub issue. I would like to tell you into more detail if you're up to it? You can send me an email to address shown in my profile (https://github.com/NumesSanguis). Then we can arrange a audio/video chat, e.g. with https://zoom.us/ ?

I'm willing to accept that there's something about your situation that you can't use many flat arrays. (Actually, it's a pet peeve of mine when people ignore arguments that situation X is different from situation Y and therefore requires different software. I won't do the same!)
That said, persistence is low on the priority list for Awkward 1. Using a storage format like npy/npz, Zarr, HDF5, Arrow, Parquet would be a matter of unpacking the Awkward array into its constituents and then re-packing it on the other side. That's pretty simple if the data structure is simple and known. It gets more complicated when you want to do it in general, for any structure, and that's the low-priority item for me.
We can talk by Zoom if you set up the meeting. (Zoom it's one that I've never set up, but I've successfully joined meetings.) I'm generally available 8am to 4pm U.S. Central time. We should probably continue by email: pivarski at Princeton (edu).

Currently I'm attempting the (un)packing of the AwkwardArray into Parquet, as you can see here: https://stackoverflow.com/questions/59264202/awkward-array-how-to-get-numpy-array-after-storing-as-parquet-not-bitmasked
I've send you an email, thank you.

Thank you again for helping me out. Since this issue drifted far from the original question (Zarr support), I'll be closing it. If Zarr support is still wanted, I think it's better to create a new issue for it.


Add DSA to SHA1 mapping in dnssec AlgorithmToHash

Discussion in issue https://github.com/miekg/dns/issues/669

Codecov Report

Merging #670 into master will increase coverage by 0.03%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #670      +/-   ##
==========================================
+ Coverage   58.05%   58.09%   +0.04%
==========================================
  Files          37       37
  Lines        9999     9999
==========================================
+ Hits         5805     5809       +4
+ Misses       3144     3141       -3
+ Partials     1050     1049       -1

Impacted Files
Coverage Δ

dnssec.go
59.1% <ø> (ø)
:arrow_up:

msg.go
78.17% <0%> (+0.64%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 01d5935...be96dcd. Read the comment docs.

I've seen the discussion, but does this still need to be merged?

If you feel that DSA should be in the map, then yes.


Endless loop on brute force

Hi,
When I tried a brute force with this script with a dictionary, like rockyou, the brute force never ended. When the script finishes reading the dictionary, it starts reading it again.
Thx.

Closed due to inactivity


Can't use OAuth2 with just the outgoing server

Checklist

[X] I have used the search function to see if someone else has already submitted the same bug report.
[X] I will describe the problem with as much detail as possible.

App version
6.400
Where did you get the app from?
No response
Android version
doesn't matter
Device model
No response
Steps to reproduce

Manually set up an IMAP account.
For the incoming server use an account with a provider where the app doesn't support OAuth 2.0.
For the outgoing server use an account with a provider where the app supports OAuth 2.0.

Expected behavior
Setting up the account should work. The OAuth flow should be triggered for the outgoing server.
Actual behavior
The error message "OAuth 2.0 is currently not supported with this provider" is displayed.
Right now we use the name of the incoming server to find the OAuth configuration for a provider. We're also not using the user name configured for a incoming/outgoing server for the OAuth flow. Instead we're using the account's default email address.
Logs
No response

Just found this - +1 for this being a problem, in both parts: I use outlook only for smtp not for imap, and my default email address is not the same as the outlook username.
Never hacked on k9mail, but will look at it if not fixed by mid Feb.

+1 on this with a slight twist. I'm using OAuth2 for incoming mail from outlook365, but I want to send with gmail, which also requires OAuth2. It's no problem configuring it, but it fails on the second configuration on the same account. It seems that the K-9 Mail shares the OAuth2 token between incoming and outgoing settings. This is suitable if the server is the same for "fetching" and "sending", but it doesn't work if these email services are different for fetching and sending.


use process.stdout.write rather than console.log to prevent printf parsing of output (and speed things up)

solves https://github.com/taskrabbit/elasticsearch-dump/issues/119

> npm publish
+<EMAIL_ADDRESS>


Authorize 0 as month value

Hello,
Thanks for your work! Your package is very convenient.  :-)
Here is a small contribution: the possibility to accept a zero-valued month.
As a matter of fact, I have to deal with a JSON payload containing a 4-minute long duration, which is formatted like this: "P0Y0M0DT0H4M0S" (it is generated by a PHP project).
I have also fixed some small errors in the tests.
Regars,
Matt

I'm glad you liked it.  :-)


Jenkins CI should collect logs

When Jenkins CI fails a PR test, it puts together a tar. Currently this only appears to include the console output and some kind of test debug output. It should also include general cilium information.
At minimum:

Cilium daemon logs
Proxy logs (if relevant)

Also useful:

Output of cilium commands, eg "cilium endpoint get"; "cilium policy get"; etc.
Copy of /var/run/cilium/* (Perhaps minus the binaries)
Dump of maps from /sys/fs/bpf/tc/globals/cilium*

Most of this currently exists - is it not appearing in a specific build ?

The k8s guestbook test in this run is an example:
https://jenkins.cilium.io/job/cilium/job/cilium/job/PR-1599/15/

There is an issue with this test, in that if the cleanup steps fail during the test, the logs are not gathered appropriately. The proper fix would be to do something like:
function finish_test {
  log "finishing test: ${TEST_NAME}"
  cleanup || true
  gather_files ${TEST_NAME} k8s-tests
  gather_k8s_logs "2" ${LOGS_DIR}
  log "finished running test: $TEST_NAME"
}

as a quick fix. The drawback to doing this is that if the cleanup fails, then it might cause failures in the next test since the environment for the next test won't be clean. We could do something like the following to avoid that:
set +e
cleanup
CLEANUP_RETURN_CODE=$?
set -e
...
# End of finish_test function
if [[ "$CLEANUP_RETURN_CODE" != "0" ]];
  log "cleanup failed, exiting"
  exit 1
fi

Issue no longer present.


Added support for go modules without slash in the name

Some go modules don't have a slash (/) in their names in the go.mod file. One example is go4.org module that is being used by golang.org/x/build. You can see the go.mod file here.
The cli is failing to push this dependencies to Artifactory with the following error message:
[Error] Module name missing in go.mod file

You can use the following steps to reproduce the issue:
#!/bin/bash
export GOPATH=<REPLACE_GO_PATH>
export PATH=$PATH:$GOPATH/bin
export GO111MODULE=off
cd $GOPATH
go get go4.org
cd $GOPATH/src/go4.org
git checkout 417644f6feb5 .
export GO111MODULE=on
go mod init go4.org
go mod tidy
jfrog rt gp <REPLACE_GO_REPO> v0.0.0-20180809161055-417644f6feb5 --server-id=<REPLACE_SERVER_ID> --self=true

I'm proposing a change in the regex pattern that gets the module name to make the slash optional.

Thanks @elioengcomp. Looks good. Can you please also create a unit test for this? You can add this test to the project_test.go.

Thanks @elioengcomp. Looks good. Can you please also create a unit test for this? You can add this test to the project_test.go.

Done!


User Guide: explain what the build command does

Running this code:
# Build and push the app-operator image to a public registry such as quay.io
$ operator-sdk build quay.io/example/app-operator

Will output some docker related code && also print Create deploy/operator.yaml.
There are a few questions:

/deploy/operator.yaml is generated.  Should this be manually edited at this point?  The assumption is that it should not.  If so, ideally print something at the top of the file, like # GENERATED FILE DO NOT EDIT
Other files under /deploy already exist from the initial operator-sdk new  Should these be edited directly, or should we assume everything under /deploy is generated?
does operator-sdk build do any other interesting things that should be noted?

Thanks!

I'm also wondering if the generated yaml should not be edited manually, should the build process not support the passing of some flags to allow the output to be customised yet consistent.
The first use case that comes to mind is the the ability to pass it a service account name. To me, defaulting to using the default service account and then expecting to grant it a set of permissions does  not seem correct.


Element type is invalid (When adding ApplicationTabs)

I am getting the error that was initially shown in your tutorial video here :
https://youtu.be/6zVGVHWklg8?t=6m1s
I have named the file correctly "/app/containers/ApplicationTabs/index.ios.js" and the file contents are exactly as shown in the tutorial. However where the app worked in your video once you connected ApplicationTabs I am still getting that error after doing so myself.
I am using a newer version of a lot of the react/react-native/redux modules however.
I am not sure if the convention of exporting with files named with "index.ios.js" has changed or anything? Any help you could provide would be greatly appreciated.
Below is my package.json dependencies :
  "dependencies": {     "react": "15.4.2",     "react-native": "0.40.0",     "react-redux": "^5.0.2",     "redux": "^3.6.0",     "redux-logger": "^2.7.4",     "redux-thunk": "^2.2.0"   }
And this is my /app/containers/ApplicationTabs/index.ios.js file contents :
import { View, TabBarIOS, TabBarItemIOS } from 'react-native'; import React, { Component } from 'react'; import { connect } from 'react-redux'; class ApplicationTabs extends Component {     render() {         return (             <View />         );     } } function mapStateToProps(state) {     return {     }; } export default connect(mapStateToProps)(ApplicationTabs);
And the contents of AppContainer :
import React, {Component} from 'react'; import {connect} from 'react-redux'; import {ActionCreators} from '../actions'; import {bindActionCreators} from 'redux'; import ApplicationTabs from './ApplicationTabs'; class AppContainer extends Component {     render() {         return (             <ApplicationTabs { ...this.props } />         )     } } function mapDispatchToProps(dispatch) {     return bindActionCreators(ActionCreators, dispatch); } export default connect( () => { return {} }, mapDispatchToProps)(AppContainer);

Hi @AkabaneK I'm afraid that React Native has changed their navigation quite a bit since I published this. I think it's been pulled out here: https://reactnavigation.org/
Hope that helps!


Refactor Pre-2FA

Previous PR:
https://github.com/near/near-contract-helper/pull/165
This PR triggers render. It should be tested, rebased and then merged into master prior to adding 2FA features for v0 of wallet.

@mattlockyer looks like tests are failing on CI

@vgrichina I don't know why they fail on Jenkins, looks like timeouts, but I can run them locally.
There is 1 test that I expect to fail, because we no longer require that the signature comes from a key that's a full access key (if the user has enabled 2fa).
I changed the filter condition on verifySignature (now in middleware/near.js) so we accept signatures from limited access keys that still recover to the accountId. What do you think about this?
Should I remove the test?

There is 1 test that I expect to fail, because we no longer require that the signature comes from a key that's a full access key (if the user has enabled 2fa).

@mattlockyer

let's make sure tests actually pass on CI for every PR. If test fails it is necessary to either fix code or test (if something is wrong about it), otherwise PR won't be ready to merge.
If this is just "refactor" then by definition none of the tests should be failing. If there are intended logic changes – they need to be described in the description of the PR and tests need to be updated accordingly.

There is 1 test that I expect to fail, because we no longer require that the signature comes from full access key (if the user has enabled 2fa).
I changed the filter condition on verifySignature (now in middleware/near.js) so we accept signatures from limited access keys that still recover to the accountId. What do you think about this?
Should I remove the test?

Do you see any security issue if verifySignature, which is used by checkAccountOwnership is not checking if the key is full access?
https://github.com/near/near-contract-helper/blob/fab413eb6be71cd9c0468a107565c9172f744f3b/middleware/near.js#L11-L22

Thanks for the help! I will check that it's a multisig key by checking the contract methods, but I'll push that in the 2fa PR.

@vgrichina
check the test change here please, I think it's correct, was not passing before (locally, but not in CI)
https://github.com/near/near-contract-helper/pull/188/commits/6391f47bd75d7cdd24c32246cab96697ed4e7c8a

Now I just seem to be getting the downstream timeouts causing errors to fail.
When errors fail, they fail with 400, but none of our other codes.
And all tests pass locally.


Document how the Environment works as easy proxy

$tenancy = app(Hyn\Tenancy\Environment::class);

// get website
$tenancy->website();
// get hostname
$tenancy->hostname();
// get customer (if any)
$tenancy->customer();
// set hostname
$hostname = Hostname::firstOrFail();
$tenancy->hostname($hostname);

Include the Tenancy facade.


JS SDK?

Is there a plan to provide a JS SDK, or expose the API over HTTP so we can use this API from frontend apps without backend?
This can greatly reduce time to deploy, will allow many frontend developers to built UIs and applications, will allow for "self serve" applications, that users could paste their own API keys to, chrome extensions and more.

Same question here

We've uploaded the protobuf definitions for the API here: https://github.com/Stability-AI/stability-sdk/blob/main/src/proto/generation.proto
This should enable users to build a client in any language that supports gRPC. We'll roll out code to demonstrate connecting in other languages soon. I'm hoping the protobuf spec will be enough to get the ball rolling for the time being though.

We've uploaded the protobuf definitions for the API here: https://github.com/Stability-AI/stability-sdk/blob/main/src/proto/generation.proto
This should enable users to build a client in any language that supports gRPC. We'll roll out code to demonstrate connecting in other languages soon. I'm hoping the protobuf spec will be enough to get the ball rolling for the time being though.

Thank you! I'm currently working on a client-side Swift implementation for iOS.

I was unable to use the proto file. Hope you make a javascript sdk.

@svumme I've added precompiled stubs for javascript and typescript here: https://github.com/Stability-AI/stability-sdk/tree/main/src/js
I'll close this issue once we've got demo usage documented

I've actually created a typescript client/CLI that uses the stability-sdk typescript that you've precompiled, @dmarx. (CC @altryne @Pau-Lozano @kaishin @svumm)
https://github.com/jakiestfu/stability-ts

In case folks are interested, I've got another example of using the gRPC service from TypeScript at https://github.com/jeffposnick/magic-realism-dream-bot/tree/main/proto
That example uses the https://www.npmjs.com/package/@grpc/grpc-js client, which I believe is the most "modern" of the various gRPC libraries for JavaScript/TypeScript.

I've also since updated the JS SDK to use the new stability-sdk repo https://github.com/jakiestfu/stability-ts

@jakiestfu I'm getting this error when trying to use it
Error: Must provide image parameters.

Anyone know what's going on here?
"UNKNOWN:Error received from peer ipv4:<IP_ADDRESS>:443 {created_time:"2022-09-22T16:12:19.839528-06:00", grpc_status:3, grpc_message:"Must provide image parameters."}"

Anyone know what's going on here?
"UNKNOWN:Error received from peer ipv4:<IP_ADDRESS>:443 {created_time:"2022-09-22T16:12:19.839528-06:00", grpc_status:3, grpc_message:"Must provide image parameters."}"

@CedYF @dholbrook86 @jakiestfu You should no longer pass ClassifierParameters in the request (for Jake's TS client, remove line 145 and 146 of index.ts)
Apologies for the breaking change. The python client should work if you update the pip package.

So is it possible to connect to beta.dreamstudio.ai or any other URL from Stability AI to use the Stable Diffusion? Should I send protocol buffers as binary data over the HTTP POST request? What URL should I communicate with?

Anyone know what's going on here?
"UNKNOWN:Error received from peer ipv4:<IP_ADDRESS>:443 {created_time:"2022-09-22T16:12:19.839528-06:00", grpc_status:3, grpc_message:"Must provide image parameters."}"

@CedYF @dholbrook86 @jakiestfu You should no longer pass ClassifierParameters in the request (for Jake's TS client, remove line 145 and 146 of index.ts)
Apologies for the breaking change. The python client should work if you update the pip package.

but when at the linux, it still appear the same error message, even if I use the command pip install --upgrade stability-sdk, at the windows, it can work now, any other steps should I do at the linux?

So is it possible to connect to beta.dreamstudio.ai or any other URL from Stability AI to use the Stable Diffusion? Should I send protocol buffers as binary data over the HTTP POST request? What URL should I communicate with?

You should use https://grpc.stability.ai:443 (which is the default) for the URL. There are no other production URLs at present. The API keys can be managed at https://beta.dreamstudio.ai/membership?tab=apiKeys
Currently, the best way to send the request in js/ts is to make a submodule of  https://github.com/Stability-AI/api-interfaces. Then import Generation from "api-interfaces/gooseai/generation/generation_pb"; and import { grpc } from "@improbable-eng/grpc-web";
You can then construct a request const request = new Generation.Request(); and send it over grpc. We hope to provide an easier official way of doing this soon, but in the mean time you can have a look at https://github.com/vpzomtrrfrt/stability-client/blob/main/src/index.ts for reference of what needs to go into the request (or use @vpzomtrrfrt 's package https://github.com/vpzomtrrfrt/stability-client )

@TandK8600

Anyone know what's going on here?
"UNKNOWN:Error received from peer ipv4:<IP_ADDRESS>:443 {created_time:"2022-09-22T16:12:19.839528-06:00", grpc_status:3, grpc_message:"Must provide image parameters."}"

@CedYF @dholbrook86 @jakiestfu You should no longer pass ClassifierParameters in the request (for Jake's TS client, remove line 145 and 146 of index.ts)
Apologies for the breaking change. The python client should work if you update the pip package.

but when at the linux, it still appear the same error message, even if I use the command pip install --upgrade stability-sdk, at the windows, it can work now, any other steps should I do at the linux?

On your linux box, are you using the pip package, or a cloned version of this repository? (if the command you run to invoke is python3 -m stability_sdk.client then you are probably using the pip version, if the command you run to invoke is python3 src/stability_sdk/client.py then you are using the cloned repository.
If the pip package, when you run pip show stability-sdk do you see version 0.2.5?
If the cloned repo, make sure that you have the latest (git pull and git submodule update --init --recursive)

@chigozienri Thanks, but I am getting "404 page not found" as a HTTP response. I tried to send empty data both through GET and POST method.
Is there any clear description of the API, which does not require using any of your libraries?

(I'm not affiliated with this project.)
@photopea, I think your best bet would be to read through https://grpc.io/about/, as from what I can tell, the Stable Diffusion API is exposed as a "standard" gRPC endpoint. It's not trivial to interact with an gRPC endpoint without using a helper library, as you'd need to serialize and deserialize protobuffs yourself.
https://grpc.io/docs/platforms/web/basics/ is the library that most folks use for accessing gRPC services from the browser, and it will handle both the transport and serialization/deserialization for you.

@photopea Thanks for sending your code snippet, that makes it clearer. As @jeffposnick says, you can't simply send an HTTP GET or POST request, it needs to be a gRPC call. We do not have a REST API available at present (there is a small discussion on this here) - nor, unfortunately, do we have a description for using the API other than to look at the protos, or to emulate how they are used in the python client in this repo/ other people's implementations such as https://github.com/vpzomtrrfrt/stability-client

I am familiar with protocol buffers and I am planning to communicate through them. But I am curious, why your server does not return any error message, but returns "404" instead.

@TandK8600

Anyone know what's going on here?
"UNKNOWN:Error received from peer ipv4:<IP_ADDRESS>:443 {created_time:"2022-09-22T16:12:19.839528-06:00", grpc_status:3, grpc_message:"Must provide image parameters."}"

@CedYF @dholbrook86 @jakiestfu You should no longer pass ClassifierParameters in the request (for Jake's TS client, remove line 145 and 146 of index.ts)
Apologies for the breaking change. The python client should work if you update the pip package.

but when at the linux, it still appear the same error message, even if I use the command pip install --upgrade stability-sdk, at the windows, it can work now, any other steps should I do at the linux?

On your linux box, are you using the pip package, or a cloned version of this repository? (if the command you run to invoke is python3 -m stability_sdk.client then you are probably using the pip version, if the command you run to invoke is python3 src/stability_sdk/client.py then you are using the cloned repository.
If the pip package, when you run pip show stability-sdk do you see version 0.2.5? If the cloned repo, make sure that you have the latest (git pull and git submodule update --init --recursive)

@TandK8600

Anyone know what's going on here?
"UNKNOWN:Error received from peer ipv4:<IP_ADDRESS>:443 {created_time:"2022-09-22T16:12:19.839528-06:00", grpc_status:3, grpc_message:"Must provide image parameters."}"

@CedYF @dholbrook86 @jakiestfu You should no longer pass ClassifierParameters in the request (for Jake's TS client, remove line 145 and 146 of index.ts)
Apologies for the breaking change. The python client should work if you update the pip package.

but when at the linux, it still appear the same error message, even if I use the command pip install --upgrade stability-sdk, at the windows, it can work now, any other steps should I do at the linux?

On your linux box, are you using the pip package, or a cloned version of this repository? (if the command you run to invoke is python3 -m stability_sdk.client then you are probably using the pip version, if the command you run to invoke is python3 src/stability_sdk/client.py then you are using the cloned repository.
If the pip package, when you run pip show stability-sdk do you see version 0.2.5? If the cloned repo, make sure that you have the latest (git pull and git submodule update --init --recursive)

Hi, with regards to this issue ""UNKNOWN:Error received from peer ipv4:<IP_ADDRESS>:443 {created_time:"2022-09-22T16:12:19.839528-06:00", grpc_status:3, grpc_message:"Must provide image parameters."}"" The error still persist on linux despite updating/upgrading to the newest version 0.2.6. Does anyone have any idea how to recitfy this ?

Same issue on Google Cloud Functions:
line 48, in stabilityai for resp in answers: File "/layers/google.python.pip/pip/lib/python3.7/site-packages/stability_sdk/client.py", line 300, in generate for answer in self.stub.Generate(rq, **self.grpc_args): File "/layers/google.python.pip/pip/lib/python3.7/site-packages/grpc/_channel.py", line 426, in __next__ return self._next() File "/layers/google.python.pip/pip/lib/python3.7/site-packages/grpc/_channel.py", line 826, in _next raise self grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with: status = StatusCode.INVALID_ARGUMENT details = "Must provide image parameters." debug_error_string = "UNKNOWN:Error received from peer ipv4:<IP_ADDRESS>:443 {created_time:"2022-10-19T15:46:11.888314867+00:00", grpc_status:3, grpc_message:"Must provide image parameters."}"

Find the solution; The code is working on Google colab so freeze the requirements on colab then compare the package versions between colab and your environment and update them. it works now for me on Google cloud functions.

@mohsendelsouz Hi, it seems that this issue was solved at #55 and subsequently merged with #64, where all we needed to do was remove the classifiers. Credit to @sayanarijit.

Could you make a simple HTTP API, which does not require implementing Protocol Buffers, or using huge JS libraries?
Look at the API of DALL-E: https://beta.openai.com/docs/api-reference/images/create

I would definitely echo @photopea's comment. A REST API would open up a lot of possibilities.
To give you some additional tangibles, here are other environments/projects where it's very difficult to implement gRPC:

Krita plugins (@imperator-maximus created his own python grpc implementation, because Krita doesn't allow 3rd party libs)
Blender add-ons (I jumped through a lot of hoops to create my own wrapper API for Stability SDK, because Blender doesn't have reliable/easy support for outside libs)
Any website, like others have discussed above

PS. I LOVE Stable Diffusion, and just want more developers to be able to integrate it!

We've uploaded the protobuf definitions for the API here: https://github.com/Stability-AI/stability-sdk/blob/main/src/proto/generation.proto
This should enable users to build a client in any language that supports gRPC. We'll roll out code to demonstrate connecting in other languages soon. I'm hoping the protobuf spec will be enough to get the ball rolling for the time being though.

Thank you! I'm currently working on a client-side Swift implementation for iOS.
Update: Working like a charm!

May I ask how you achieved it? I want to inherit it into Macapp, which is also developed in Swift.

I would definitely echo @photopea's comment. A REST API would open up a lot of possibilities.
To give you some additional tangibles, here are other environments/projects where it's very difficult to implement gRPC:

Krita plugins (@imperator-maximus created his own python grpc implementation, because Krita doesn't allow 3rd party libs)
Blender add-ons (I jumped through a lot of hoops to create my own wrapper API for Stability SDK, because Blender doesn't have reliable/easy support for outside libs)
Any website, like others have discussed above

PS. I LOVE Stable Diffusion, and just want more developers to be able to integrate it!

see my detailed comment on this:
https://github.com/Stability-AI/stability-sdk/discussions/51#discussioncomment-4171959
In short: grpc was a nice idea but it failed. so better use REST only in future.

@all see this comment: https://github.com/Stability-AI/stability-sdk/discussions/51#discussioncomment-4218518
It sounds like JS will be much more easily supported soon!

@svumme I've added precompiled stubs for javascript and typescript here: https://github.com/Stability-AI/stability-sdk/tree/main/src/js
I'll close this issue once we've got demo usage documented

broken link

REST API developer preview was just released! https://twitter.com/jcsabath/status/1597629332184145920

Hey I'm also interested in working on something for iOS , would you be able to share your work ?

Hey @kaishin I'm also interested in working on something on iOS based on this AI. Would you be willing to share your work ?
Thanks a lot !

@charvoa @imperator-maximus I can clean it up and do a short write up about it in the coming weeks! I'll update this thread once done.

Closing this issue, for additional context please check out the following comment regarding JS/TS: https://github.com/Stability-AI/stability-sdk/issues/26#issuecomment-1487248450


Compilation failes due to downloader package update.

Compilation of the risc0-rust-starter fails. The problem is the latest downloader 0.2.7.  cargo update -p downloader --precise 0.2.6 is a rescue here, but it's better to drop the dependency on this package, and get something more mature. Another option is updating the toolchain date.
Logs:
[2023/01/04 17:49:09] ziel ~/workspace/risc0
o_0 > git clone https://github.com/risc0/risc0-rust-starter
Cloning into 'risc0-rust-starter'...
remote: Enumerating objects: 296, done.
remote: Counting objects: 100% (154/154), done.
remote: Compressing objects: 100% (86/86), done.
remote: Total 296 (delta 101), reused 80 (delta 60), pack-reused 142
Receiving objects: 100% (296/296), 74.76 KiB | 1.70 MiB/s, done.
Resolving deltas: 100% (121/121), done.
[2023/01/04 17:49:54] ziel ~/workspace/risc0
o_0 > cd risc0-rust-starter/
[2023/01/04 17:50:01] ziel ~/workspace/risc0/risc0-rust-starter main
o_0 > cargo run
    Updating git repository `https://github.com/risc0/risc0.git`
    Updating crates.io index
   Compiling libc v0.2.139
   ...
   Compiling downloader v0.2.7
error[E0658]: `let...else` statements are unstable
  --> /home/ziel/.cargo/registry/src/github.com-1ecc6299db9ec823/downloader-0.2.7/src/download.rs:28:5
   |
28 |     let Ok(url) = reqwest::Url::parse(url) else { return std::path::PathBuf::new() };
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: see issue #87335 <https://github.com/rust-lang/rust/issues/87335> for more information
   = help: add `#![feature(let_else)]` to the crate attributes to enable

For more information about this error, try `rustc --explain E0658`.
error: could not compile `downloader` due to previous error
warning: build failed, waiting for other jobs to finish...

This should get fixed once this lands: https://github.com/risc0/rust/pull/8

This should now be fixed


Fix uninitialized variable build warns -> errors

Not sure if this is just a toolchain quirk, but without this I can't get skiboot to build without this.

Can one of the admins verify this patch?

Looks good - but can you add a Signed-off-by line? We need this to be able to merge.

Merged (although we usually go via the skiboot mailing list, I merged this one)


[feature] search results as MRT file

MRT file is much more compressed and suitable for long-term storage.

Example code for reference: https://github.com/bgpkit/bgpkit-parser/blob/main/examples/mrt_filter_archiver.rs

Resolved in eec513d and #64