Can't add Netscaler VPX

ISSUE TYPE

Bug Report

COMPONENT NAME

API
UI
NETWORK PROVIDER

CLOUDSTACK VERSION

4.16
4.15.x

CONFIGURATION

Advanced zone, physical network with Netscaler service provider in section Network Service Providers

OS / ENVIRONMENT

Citrix VPX 13
VPX 12
VPX 11.0

SUMMARY

Error when adding Netscaler VPX to an advanced zone

STEPS TO REPRODUCE

Installing a VPX Netscaler VM
Configuring NSIP, SNIP a vlan on 1/2 and a SNIP on this vlan binding on 1/2
On an advanced zone, Network Service Providers, Netscaler menu
Add a Netscaler device form with the following entries :

ip : NSIP
user : nsroot
mdp : pass
type : Netscaler VPX Loadbalancer
public interface : 1/2
private interface 1/1
dedicated : true
EXPECTED RESULTS

This should add the VPX device

ACTUAL RESULTS

This send me an error on all these versions (Error on the GUI) :
Add Netscaler device
(Netscaler) Failed to verify device type specified when matching with actuall device type due to Netscalar device type specified does not match with the actuall device type.

In plugins/network-elements/netscaler/src/main/java/com/cloud/network/resource/NetscalerResource.java
`if (_deviceName.equalsIgnoreCase("NetscalerMPXLoadBalancer") && nsHw.get_hwdescription().contains("MPX") ||
_deviceName.equalsIgnoreCase("NetscalerVPXLoadBalancer") && nsHw.get_hwdescription().contains("NetScaler Virtual Appliance")) {
return;
}
throw new ExecutionException("Netscalar device type specified does not match with the actuall device type.");
}
Netscaler 13 per example shows : "Netscaler Remote Licensed Virtual Appliance 450000"
This could explain why adding VPX is
bugged.
Perhaps are there other lines causing a problem

Thanks @kurushi9000 for reporting, the community doesn't have access to netscaler device and the old netscaler plugin isn't maintained for years - you should contact the vendor (Citrix) to help maintain the network plugin or get them to sponsor test infrastructure for the same.

Hello @rohityadavcloud , i tested creating a new account on citrix.com and saw Vpx is free and available for downloading
Perhaps this way, the community could access the appliance ?

I have tested on VPX 13 freemium
Now i have tested on VPX 11.0 licensed and it returns : "Failed to log in to Netscaler device at a.b.c.d due to Certificates do not conform to algorithm constraints"

@sureshanaparti , this is well working with vpx12 and vpx13
The only issue with VPX is when testing type of Netscaler
I successed to add vpx12 and vpx13 to ACS 4.16 👍
First add add a MIP VPX11.0 (with a developpement licence : https://www.citrix.com/downloads/netscaler-adc/virtual-appliances/netscaler-vpx-developer-edition.html)
After what i deleted the MIP and recreated it on VPX12 freemium or VPX13 freemium
All other features are still working on it with ACS
The only bug is about the check with "Netscaler Remote Licensed Virtual Appliance 450000"
On vpx 11.0 ns hardware description give : "NetScaler Virtual Appliance 450000"
On later vpx ns hardware description gives : Netscaler Remote Licensed Virtual Appliance 450000"
This bug could be easily been corrected, no ?

If a Netscaler is being added with vpx11, once upgrade is done towards vpx13, default password is forced to change
So when vpx13 password is changed, it need to be changed in ACS Database as there are no Gui for this on ACS for an existing Netscaler device
Then new password has to be encrypted with Jasypt library with two way handshake using the private "key" given on ACS cloudstack installation (cloudstack-setup-databases cloud:@ --deploy-as=<Db_User:<Db_Pass> -e file -m  -k  -i <Acs_ip_node>)
There is an online encryption tool here : https://www.devglan.com/online-tools/jasypt-online-encryption-decryption
The encrypted password to be changed is on host_details, entry named 'password' where "host_details"."host_id" is "external_load_balancer_devices"."host_id"
Created an VPC Redundant Offering with Netscaler public LB capability .
Lb rules are well working !
Tested on vpx 11.0, vpx 12.1 and vpx 13.0 build 84.11

@BenoitLair is there any work on this going on?

It wouldn't be possible to do this as we don't have access to the said component/hardware/appliance. Need more information.

Hello Rohit,
Is there no more anybody from citrix in ACS Dev Team ?
I can see with Citrix asking for a Development account with freemium VPX versions
It is a feature kill if we cant add vpx into ACS Mgmt server
Adding Netscaler is not possible due to "Platform" label value checking with value "NetScaler Virtual Appliance 450000" which is now is now "Netscaler Remote Licensed Virtual Appliance 450000"
The test should implement checking LIKE "Netscaler%%Virtual Appliance 450000"
Also we should have choice of editing password for a Netscaler device
Also there are been some minor changes when working with a VPC with External Load balancer of type Netscaler
When adding some LB rules, the API calls had some little changes
For working with it i installed a Nginx mounting NS ip device declared in ACS and forwarding request to a NS MIP ip
I used lua rewrite file and subfilters in order to adapt NS API changes
`
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
upstream tunnel_ns-vpx13-ssl {
#use MIP created on Netscaler vpx13, does work with vpx12
server aa.bb.cc.dd:80;
keepalive 32;
}
proxy_cache_path /var/cache/nginx-vpx13-ssl levels=1:2 keys_zone=ns-vpx13-ssl_cache:10m max_size=3g inactive=120m use_temp_path=off;
server{
listen 443 ssl;
server_name ee.ff.gg.hh; # use NS ip declared on ACS Mgmt server, has been created with vpx11 device due to constraint 'NetScaler Virtual Appliance'
ssl_certificate /etc/ssl/certs/cert-selfsigned.crt;
ssl_certificate_key /etc/ssl/private/cert-selfsigned.key;
ssl_dhparam /etc/ssl/certs/cert-dhparam.pem;
access_log      /var/log/nginx/ns-vpx13-ssl-access.log;
error_log              /var/log/nginx/ns-vpx13-ssl-error.log;
location / {
	access_by_lua_file /etc/nginx/vpx13-prod-ee.ff.gg.hh.lua;

	sub_filter_types text/html text/css text/xml application/json;
	sub_filter 'NITRO' 'NITRO2';
	sub_filter 'Login Failure' 'Login Failure22';
	sub_filter 'Netscaler Remote Licensed Virtual Appliance' 'NetScaler Virtual Appliance';

	client_max_body_size 50M;
	proxy_set_header Connection "";
	proxy_set_header Host $http_host;
	proxy_set_header X-Real-IP $remote_addr;
	proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
	proxy_set_header X-Forwarded-Proto $scheme;
	proxy_set_header X-Frame-Options SAMEORIGIN;
	proxy_buffers 256 16k;
	proxy_buffer_size 16k;
	proxy_read_timeout 600s;
	proxy_cache ns-vpx13-ssl_cache;
	proxy_cache_revalidate on;
	proxy_cache_min_uses 2;
	proxy_cache_use_stale timeout;
	proxy_cache_lock on;
	proxy_http_version 1.1;
	proxy_pass http://tunnel_ns-vpx13-ssl;
}

}
Content of /etc/nginx/vpx13-prod-ee.ff.gg.hh.lua
function remove_user_key()
ngx.req.read_body()
-- log the original body so we can compare to the new one later
local oldbody = ngx.req.get_body_data()
--log(oldbody)
-- grab the POST parameters as a table
local params = ngx.req.get_post_args()
-- build up the new JSON string
local newbody = "{"
for k,v in pairs(params) do
-- add all the params we want to keep
if k ~= "serviceType" then
-- log('adding"..k.." as "..v.." :')
newbody = newbody..'"'..k..'":"'..v..'",'
else
-- log("adding modified serviceType")
newbody = newbody..'"'..k..'":"'HTTP'",'
end
end
--remove the last trailing comma before closing this off
newbody = string.sub(newbody, 0, #newbody-1)
newbody = newbody.."}"
ngx.req.set_body_data(newbody)
-- log(newbody)
end
function format_http_vservers_protocol()
ngx.req.read_body()
local body = ngx.req.get_body_data()
-- ngx.log("not adding user_key")
ngx.log(ngx.NOTICE, "hello world")
-- if string.find(body,"Cloud-Service-") then
body = string.gsub(body, "TCP", "HTTP")
ngx.req.set_body_data(body)
-- end
end
if ngx.req.get_method() == "POST" then
-- remove_user_key()
-- ngx.req.read_body()
-- local oldbody = ngx.req.get_body_data()
-- log(oldbody)
format_http_vservers_protocol()
-- ngx.req.read_body()
-- local body = ngx.req.get_body_data()
-- if body then
--    body = string.gsub(body, "TCP", "HTTP")
-- end
-- ngx.req.set_body_data(body)
end
`

This is a blocker for me in order to upgrade from CS 4.16 to next versions of CS

Hi
we are trying to add NS 14.1 to cloud stack 4.19
and getting the same error.
This send me an error on all these versions (Error on the GUI) :
Add Netscaler device
(Netscaler) Failed to verify device type specified when matching with actuall device type due to Netscalar device type specified does not match with the actuall device type.

@sourabh1515-inf , no it wasn't . it was closed as 'stale', as there was no one working on it and the user didn't respond any more. If you have new input, please re-open or create a new issue


Click and dblclick action mapping

I wanted to implement a use case, that allows users to click or dblclick a tree node. If user clicks an ACTION1 is made, and if user dbclocks an ACTION2 is made. The problem now that if user makes double click, the click (ACTION1) is also initiated. Is there any debouncer or something similar in the core that i could use out of the box?

I am also facing the same issue saulide did you found any solution or hack for this. because on SO people are saying that you should not bind handlers to same element for single and double click. And what is that debouncer idea can you tell me what is the approach.?

Action mapping can be defined like this:
this.options = {
  actionMapping: {
      mouse: {
          drop: (tree, node, $event, {from, to}) => {
              console.log('Action drop!');
          },
          dblClick: (tree, node, $event) => {
              console.log('Action double click!');
          }
      }
  }
};

Closing this issue due to inactivity. Please open a new issue if the problem still exists.


Downloadable portable

I managed to create a moveable working zip. I ran it on Windows Server 2012, Windows 7 and Windows 8.1. With config in appdata.
I need to figure out gtk warnings or use MS file chooser istead of gnomes.
Footprint is about 60mb.
Edit: Better language.

Nice:)

Got footprint down to 47 mb, but zip is 16 mb. Now I will try to use ms explorer instead of gnome to open and save files.

This is great, is the MSYS2 install a prerequisite for your executable?

Yes, and the gnome menus work if I copy alot of conf files and link them, but it looks wrong and makes the zip much bigger. Windows open file dialog is already in C:\windows\system32 so no conf should be needed.

Btw do the latest https://github.com/cppit/jucipp compile on Ubuntu 15?

Nvm, I think I got the SIGC_FUNCTORS_DEDUCE_RESULT_TYPE_WITH_DECLTYPE stuff working finally on all platforms. Commit coming soon.

I have a working implementation now, however it needs some more work. I need to make it a bit more generic.

@eidheim, I can't figure out why this happens:
[ 80%] Building CXX object src/CMakeFiles/juci.dir/singletons.cc.o
In file included from /home/zalox/projects/juci/src/window.cc:8:0:
/home/zalox/projects/juci/src/dialogs.h:6:22: warning: ‘std::string Dialog::select_folder()’ used but never defined
   static std::string select_folder();
                      ^
/home/zalox/projects/juci/src/dialogs.h:7:22: warning: ‘std::string Dialog::select_file()’ declared ‘static’ but never defined [-Wunused-function]
   static std::string select_file();
                      ^
[ 86%] Building CXX object src/CMakeFiles/juci.dir/cmake.cc.o
[ 93%] Building CXX object src/CMakeFiles/juci.dir/terminal.cc.o
[100%] Building CXX object src/CMakeFiles/juci.dir/dialogs.cc.o
/home/zalox/projects/juci/src/dialogs.cc:5:13: warning: ‘std::string Dialog::select_folder()’ defined but not used [-Wunused-function]
 std::string Dialog::select_folder() {
             ^
In file included from /home/zalox/projects/juci/src/dialogs.cc:1:0:
/home/zalox/projects/juci/src/dialogs.h:7:22: warning: ‘std::string Dialog::select_file()’ declared ‘static’ but never defined [-Wunused-function]
   static std::string select_file();

Seems like posting this helped me. For some reason I had to use "class" instead of "namespace" to namespace the Dialogs.

Great, since I couldnt figure it out:)

Ran into some problems last night.

could not compile with mingw64
tried to extract the code into a precompiled dll with msvc
turns out mingw64 only can communicate with c-code compiled with msvc

Todo:
make c bindings for my windows dialog implementation. I will try to finish this tonight. Have a bit more time this week :)

We could close this, and rather help out and test the various distributed juCi++ packages. I also see no reason to use juCi++ on Windows without MSYS2 with its easily installable open source libraries.


Mapillary instance annotation?

Hi,
Thanks for your wonderful repo. I follow the steps in preparing datasets, but it seems that datasets/prepare_mapillary_vistas_ins_seg.py is not provided. Could you pls check it out?

I also meet  the problems. It seems datasets/prepare_mapillary_vistas_ins_seg.py is not provided.

The current code does not support instance segmentation on mapillary vistas. Because converting mapillary vistas to COCO-style annotation is very inefficient due to high resolution.

OK, thanks.


how to modify the default min and max years for date inputs

I'm using a date input type and would like to know how to customize the min and max date range. Currently, all my employees have birthdates after 2012.  I would prefer not to use the Datepicker as I like the ability to select from a dropdown of years and not scroll through a calendar.
Thank you.

Found the solution in a Rails cast: http://railscasts.com/episodes/184-formtastic-part-1?view=asciicast
It would be useful to have the start_year option in the docs.  Thank you.

@lauraeci contributions always welcome!

Hi @mikz, happy to help add documentation.  Let me know how and where to add it and I can take a stab at a doc PR.

@lauraeci I think that could be good if you could think of a way where you'd look for this documentation. Because if we would put it on a place, where you didn't look, it would not help.
Not sure if you checked RDoc, Wiki or just the Readme.


Issue querying with .In method

Summary:
How to get single result using scan or query from a list of param values?
Code sample:
Schema
var dynamoose = require('dynamoose');
const Joi = require('joi');

var FeatureSchema = new dynamoose.Schema({
  id: {
    type: String,
    index: {
      global:true,
      name: "FeatureIdIndex",
      rangeKey: "type"
    }
  },
  serviceId : {
    hashKey:true,
    type: String
  },
  name: {
    rangeKey: true,
    type: String,
  },
  description: {
    type: String
  },
  cost: {
    type: Number
  },
  numOfUsers: {
    type: Number
  },
  trial: {
    type: Number
  },
  type: {
    type: String,
    enum:["base", "additional"],
    index: {
      global: true,
      name: "TypeIndex",
      rangeKey: "id"
    }
  },
  frequency: {
    type: String,
    enum: ["bi-weekly", "monthly", "3 months", "6 months", "1 year", null]
  },
  parentId: {
    type: String,
    index:{
      global: true,
      rangeKey: 'id',
      name: 'FeatureParentIndex',
      project: ["service", "name"]
    }
  },
  parentName: {
    type: String
  }
});

Model
var FeatureSchema = require("./feature");
dynamoose.AWS.config.update({region: 'eu-west-1'});

module.exports =  {
    Feature: dynamoose.model("Feature", FeatureSchema),
}

General
const {Feature} = require("../models");
var featureIds = ["blah", "test"];
var feature = await Feature.query("type").eq("base").and().in({id:values}).limit(1).exec();

Current output and behavior:

UnhandledPromiseRejectionWarning: QueryError: Invalid Query state: %s must follow query(), where() or filter()

Expected output and behavior:
Should return a single record.
Environment:
Operating System:
Operating System Version:
Node.js version (node -v):
NPM version: (npm -v):
Dynamoose version:
Dynamoose Plugins:

Dynamoose Plugins:

[ ] Yes, I believe that one or multiple 3rd party Dynamoose plugins are effecting this issue
[x] No, I believe this issue is independent of any 3rd party Dynamoose plugins I'm using
[x] Unknown, I'm unsure if Dynamoose plugins are effecting this issue
[x] I am not using any Dynamoose plugins

Other information (if applicable):
Type (select 1):

[ ] Bug report
[ ] Feature suggestion
[x] Question
[ ] Other suggestion
[ ] Something not listed here

Other:

[x] I have read through the Dynamoose documentation before posting this issue
[x] I have searched through the GitHub issues (including closed issues) and pull requests to ensure this issue has not already been raised before
[x] I have searched the internet and Stack Overflow to ensure this issue hasn't been raised or answered before
[x] I have tested the code provided and am confident it doesn't work as intended
[x] I have ensured that all of my plugins that I'm using with Dynamoose are listed above
[x] I have filled out all fields above
[x] I am running the latest version of Dynamoose

@fishcharlie tried that and getting this error
(node:19517) UnhandledPromiseRejectionWarning: ValidationException: Query condition missed key schema element: serviceId
Using this query:
 var feature = await Feature.query("type").eq("base").filter("id").in(featureIds).exec();

As you can see, I've setup a global index on type with rangeKey id (see my schema above in my original post) but, can't seem to query it.

Closing this due to no recent activity. If this is still a problem or you have questions please feel free to comment again and we can try to help further.


File change detection

The file change detection system is not fast, it should be made faster.

Hi, I really liked this project but I have no clue where to see/start for this issue, any suggestion?
I m looking forward to contribute.
Thanks

The outcome starts "from" this line https://github.com/BuckshotPlusPlus/BuckshotPlusPlus/blob/main/BuckshotPlusPlus/Program.cs#L18

Your idea is perfect, but you should also limit the detection to "bpp" type files

@MoskalykA Hey, can you add hacktoberfest accepted label here as well, would help a lot.
Thanks!


Upgrading to new versions link totally broken

The link to upgrade create-react-app has a link to https://github.com/facebook/create-react-app/blob/master/packages/react-scripts/template/README.md#updating-to-new-releases but that doesn't really go anywhere.
What's the right link? 🙏

Where are you viewing the README? That link has been updated. It points here: https://facebook.github.io/create-react-app/docs/updating-to-new-releases


Quiet option

How can I use "quiet" in the gulp module?
This solution does not work
return gulpif('*.scss', sass.sync({ 					outputStyle: 'compressed', 					quietDeps: true, 					quiet: true, 				}).on('error', sass.logError));

See #811

@xzyfer I'd merge this with #811 and perhaps close that too.


Wanted to Add program using  recursion

Is your feature request related to a problem? Please describe.
I wanted to add program like decimal to octal conversion , decimal to binary conversion , tower of hanoi etc.
In algorithm->recursion
Describe the solution you'd like
I will create also the program using recursion in c language..
Do you want to work on it
Yes i wanted  to get assign for this . Please Assign me

Heyy, a issue deals with only 1 problem statement at a time. Plz keep this in mind and also add a little detail about yr problem statement in the issue next time onwards :)

ok


Dev

:tada: This PR is included in version 0.3.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 0.3.0-dev.5 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


stylish wav file content?

eg:
The first byte can be the current time,
The second byte can be the location,
Then it can be the Audio content.

I agree i havent provided any wav header customization option.  I may provide wav Header option to be appended on Pcm audio,
if some one really really need it cause i dont think its very much needed and i think i suspect it may create issues with reading file by Wav Player .


Tapping on settings cells always open the "Transnational Exposure Logging" information

Avoid duplicates

[x] Bug is not mentioned in the FAQ
[x] Bug is not already reported in another issue

Describe the bug
When tapping on settings cells, the app always opens the "Translational Exposure Logging" info view, which irritates user when they did not request that information. See the attached video for details!
Expected behaviour
The "Translational Exposure Logging" info view should only be shown when tapping on the "Translational Exposure Logging" cell.
Steps to reproduce the issue

Go to settings
Click on any of the settings cells

Video of the issue: https://www.dropbox.com/s/orllcgddcfo51qb/CWA_iOS_Settings_Bug_Repro.MP4?dl=0
Technical details

iOS Version: 121
Device: iOS Simulator / iPhone 11 Pro

Possible Fix
Guard the cell item-type before opening the view. A PR is ready for this: https://github.com/corona-warn-app/cwa-app-ios/pull/1459
Thank you for allowing OSS contributions on this project!

Hey @hansemannn ,
Thank you for the contribution to this project. The devs have included your fix in the upcoming 1.7 release.
I have also created an internal Jira ticket so that the devs can track your the issue.
Regards,
CH

Corona-Warn-App Open Source Team

@heinezen I think this can be closed, since the PR is already merged. Thx!


failed to import yaml module

File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 696, in load_module
    exec(compile(source, source_path, 'exec'), mod.__dict__)
  File "common.util.file in /Users/xxx/Library/Application Support/Sublime Text 3/Installed Packages/GitSavvy.sublime-package", line 2, in <module>
ImportError: No module named 'yaml'

OS: OS X EI Capitan
Sublime: Sublime Text 3 Build 3109 (Dev Build)

Traceback (most recent call last):
  File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 78, in reload_plugin
    m = importlib.import_module(modulename)
  File "./importlib/__init__.py", line 90, in import_module
  File "<frozen importlib._bootstrap>", line 1584, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1565, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1532, in _find_and_load_unlocked
  File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 696, in load_module
    exec(compile(source, source_path, 'exec'), mod.__dict__)
  File "git_savvy in /Users/micke/Library/Application Support/Sublime Text 3/Installed Packages/GitSavvy.sublime-package", line 35, in <module>
  File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 696, in load_module
    exec(compile(source, source_path, 'exec'), mod.__dict__)
  File "common.commands in /Users/micke/Library/Application Support/Sublime Text 3/Installed Packages/GitSavvy.sublime-package", line 1, in <module>
  File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 696, in load_module
    exec(compile(source, source_path, 'exec'), mod.__dict__)
  File "common.commands.debug in /Users/micke/Library/Application Support/Sublime Text 3/Installed Packages/GitSavvy.sublime-package", line 12, in <module>
  File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 696, in load_module
    exec(compile(source, source_path, 'exec'), mod.__dict__)
  File "common.util in /Users/micke/Library/Application Support/Sublime Text 3/Installed Packages/GitSavvy.sublime-package", line 6, in <module>
  File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 696, in load_module
    exec(compile(source, source_path, 'exec'), mod.__dict__)
  File "common.util.file in /Users/micke/Library/Application Support/Sublime Text 3/Installed Packages/GitSavvy.sublime-package", line 2, in <module>
ImportError: No module named 'yaml'

OS: OS X EI Capitan
Sublime: Sublime Text 3 Build 3103

@demon386 @majkiee, I just pushed out an update that will disable the offending code as a temporary work-around.  Syntax highlighting will disabled in inline-diff views, but we'll address that regression separately.  If you see this in the next few minutes, it would be super helpful if you could pull down master and confirm that things are working for you.  Sorry for the bad update!

It's fixed now. Thanks for the quick response.


@conda-forge-admin, please restart ci

@conda-forge-admin, please restart ci

Seems like this does not work...


open file only return False if file not found

Summary
Fixes #50 and makes sure the properties for recent files and last directory are updated despite various errors other than file not found.
Details and comments
Now in BaseController when  self.model.load_file(filename) encounters an exception,  False is only returned if no file name was provided or if a FileNotFoundError was encountered. Return is still true on validation errors or other file errors such as a file permissions error.

Hi @jwoehr,
Thanks again for your contribution.
I changed the line where you check for a certain exception msg to:
Only return false if no file or if file not found
if self.model.get_filename() is None or isinstance(ex, FileNotFoundError):
ret = False
The exception with error message "Missing input file" only happens when the filename is None.
Also, I added msgbox error in case you try to edit the invalid property since the file is now loading and creating a json data with the invalid property in it.
Please let me know if you see any problems.

Looks much better your way! Thanks!


fix(compactor):  Compactor potential oom risk of builder

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
The following PR addresses the issue of out-of-memory (OOM) errors caused by inaccurate memory estimation in the compactor component. With the implementation of StreamingUploader and SstableBuilder, we will now pass the buffer to StreamingUploader and trigger an asynchronous upload when the buffer size is met. We will then harvest these join handles when compaction is completed. However, if the object store service capability is poor, a large amount of memory may be consumed by in-flight requests, leading to OOM errors in the compactor component. Asynchronous operations may bring better pipeline effects and higher throughput, but it may worsen things in the above situation. Therefore, some changes were made in this PR
change

Change the memory estimation of the compact task (reserved whole sst capacity)
Wait for uploader finish synchronously when switching builder.

refactor

use batch cancel for  heartbeat cancel_task
introduce the new compact_task HeartbeatProgressCacnel to distinguish lack of visible progress and  no heartbeat

related to https://github.com/risingwavelabs/risingwave/issues/15946

Checklist

[ ] I have written necessary rustdoc comments
[ ] I have added necessary unit tests and integration tests
[ ] I have added test labels as necessary. See details.
[ ] I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features #7934).
[ ] My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
[x] All checks passed in ./risedev check (or alias, ./risedev c)
[ ] My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

[ ] My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

[ ] My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note
If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes.

Benchmarking bench_multi_builder/bench_streaming_upload_4mb: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 60.0s. You may wish to increase target time to 107.7s or enable flat sampling.
bench_multi_builder/bench_streaming_upload_4mb
                        time:   [1.9721 s 2.0226 s 2.0809 s]
                        change: [+117.88% +134.37% +151.32%] (p = 0.00 < 0.05)
                        Performance has regressed.

bench_multi_builder/bench_batch_upload_32mb
                        time:   [3.9310 s 3.9432 s 3.9578 s]
                        change: [+41.753% +47.433% +52.305%] (p = 0.00 < 0.05)
                        Performance has regressed.

Benchmarking bench_multi_builder/bench_streaming_upload_32mb: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 60.0s. You may wish to increase target time to 69.6s or enable flat sampling.
bench_multi_builder/bench_streaming_upload_32mb
                        time:   [1.3364 s 1.3574 s 1.3694 s]
                        change: [+22.254% +29.773% +38.817%] (p = 0.00 < 0.05)
                        Performance has regressed.

bench_multi_builder/bench_batch_upload_64mb
                        time:   [4.1482 s 4.1621 s 4.1787 s]
                        change: [+54.469% +56.908% +59.020%] (p = 0.00 < 0.05)
                        Performance has regressed.

bench_multi_builder/bench_streaming_upload_64mb
                        time:   [1.0002 s 1.0386 s 1.1139 s]
                        change: [-39.964% -27.946% -8.3196%] (p = 0.02 < 0.05)
                        Performance has improved.

bench_multi_builder/bench_batch_upload_128mb
                        time:   [3.9774 s 3.9868 s 3.9959 s]
                        change: [+41.640% +44.627% +47.235%] (p = 0.00 < 0.05)
                        Performance has regressed.

bench_multi_builder/bench_streaming_upload_128mb
                        time:   [892.23 ms 910.19 ms 925.58 ms]
                        change: [-22.538% -5.1263% +13.777%] (p = 0.69 > 0.05)
                        No change in performance detected.

bench_multi_builder/bench_batch_upload_256mb
                        time:   [3.7851 s 3.7950 s 3.8038 s]
                        change: [+28.301% +28.879% +29.389%] (p = 0.00 < 0.05)
                        Performance has regressed.

bench_multi_builder/bench_streaming_upload_256mb
                        time:   [810.20 ms 861.72 ms 924.04 ms]
                        change: [-1.9790% +5.2835% +12.629%] (p = 0.19 > 0.05)
                        No change in performance detected.

I think the minio benchmark does not make difference.
Because we have use streaming upload and multipart upload to write data asynchronously.
If this PR make some regression it means that the streaming upload did not work well in this case and the performance of the object storage may not be good. Even if we can get a better performance with handle all pending upload sst, it is at the cost of consuming more memory.

Even if we can get a better performance with handle all pending upload sst, it is at the cost of consuming more memory.

I agree. The issue here is that how we can avoid OOM even if the object store is slow while still achieve best performance on fast object store. Right now we can only achieve the latter, not the former. Ideally we should have some "backpressure" here.

Introduce a new config compactor_concurrent_uploading_sst_count to limit the counts of join_handle. This compromise ensures behavior like before the modification when we don't need to limit memory strictly. @hzxa21 @zwang28 @Little-Wallace


Add compatibility with Nova 2.1.0

Nova 2.1.0 added showOnIndex($callback) on Laravel\Nova\Fields\FieldElement
This PR add the compatibility with it to avoid the following error
Declaration of Froala\NovaFroalaField\Froala::showOnIndex() should be compatible with Laravel\Nova\Fields\FieldElement::showOnIndex($callback = true)

Why not remove the function here? you don't really need the overwrite and its more future proof. Don't overwrite what doesn't need overwriting.

@niekdemelker, I’m totally agree with you, but as the original code was forcing showOnIndex to true, I kept the same behavior.
But we are agree that this method should be removed

Method removed - https://github.com/froala/nova-froala-field/pull/32


Faker 1.8 release

I was wondering if we could have a faker 1.8 release. The mainline has some nice features in it now like seeding the random generation that would be really nice to release to the public. The last release 1.7.3 came out just about 2 months ago. Any chance for a 1.8 release?

@stympy If you are looking for help with maintainers I would gladly chip in.

Done -- enjoy. :)


Different Tx type representation as bytes array

I've found that when we need represent **PaymentTransaction ** as bytes array we use tx type as 1-byte array proof
override lazy val bytes: Array[Byte] = {
    val typeBytes = Array(TypeId.toByte)
......

But when we calculate singnature data we represent Tx type as 4-byte array proof:
  private def signatureData(sender: PublicKeyAccount, recipient: Account,
                            amount: Long, fee: Long, timestamp: Long): Array[Byte] = {
    val typeBytes = Ints.toByteArray(TransactionType.PaymentTransaction.id)
......

Shall we use similar solution in both cases ?

Not a bug actually


Peek doesnt seem to preview MP3 or AudioFIles in General.

Description of the new feature / enhancement
Peek does a great job of previewing videos and images, it seems to only suggest opening audio in the media player instead. Would love for that to be added.
Scenario when this would be used?
The same reasons we would preview video in a workflow.
Supporting information
No response

/dup #25989


Make linting error indicator configurable

Hi there,
it would be super cool if I could change how the error is displayed.
Currently I'm using the theme "One Monokai Theme" and the error for me is almost invisible.

Feature request:
Let the user choose a background and a foreground color for the character
Example from IntelliJ:

I would like to second this request. No lint issues is a pre-commit hook on the project I'm working on. It would be ideal for me to highlight whole line in bright red or something equally as obvious.

@nerdbeere @alexgorbatchev please open an issue against VSCode (https://github.com/Microsoft/vscode). The ESLint simply creates diagnostic object with a severity. The extension doesn't have any logic in terms of colors.


feat: add entity changetracker create command

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.2 out of 3 committers have signed the CLA.:white_check_mark: Julien4218:white_check_mark: mbazhlekova:x: lchenayYou have signed the CLA already but the status is still pending? Let us recheck it.


Modify the DefaultCompactionPlanner to set priorities at a table name level

We currently create three compaction services for root, meta, and a default compaction service for a running instance of accumulo.
If the DefaultCompactionPlanner is modified to use table names in addition to file sizes, then a single compaction service could be used instead of creating three different compaction services.
Priority levels would be something like root:1, meta:2, accumulo*:3, user:4 where 1 is the highest priority.
This would allow a single compaction service to be used when starting up a instance of accumulo.

Is there a reason to limit the size to 16 bits?  Thinking that we might want more room than 2-bits for table type to allow for expansion - one example would be to include accumulo.fate (or whatever it gets called) and we could find in the future that an accumulo.hosting or something outside of the metadata that provides explicit elasticity information.  Things that may want a finer level of control than being lumped into accumulo.*.
Something like byte for table type, byte for compaction type, and 2 bytes for count would fit into 32 bits and allow for growth. Limits for file counts could be limited to use less that the full 2^16 if necessary.

Is there a reason to limit the size to 16 bits?

I think it was mainly driven by this map.  We used to have a long for the priority, but @dlmarion and I realized that if the priority as a long was extremely high cardinality that could cause an OOME.  We could not think of a good reason to have an extremely high cardinality priority so switched the long to a short.

we could find in the future that an accumulo.hosting or something outside of the metadata that provides explicit elasticity information. Things that may want a finer level of control than being lumped into accumulo.*.

The computation of the priority is done by a pluggable component, the CompactionPlanner.  So it other situations could be handled, they could compute the priority in anyway they like.

Something like byte for table type, byte for compaction type, and 2 bytes for count would fit into 32 bits and allow for growth. Limits for file counts could be limited to use less that the full 2^16 if necessary.

I think many situations like this can be handled with the short.  May need to do things like take the log or sqrt of a value you are putting in the short to compress the information while still maintaining sort order.  The priority does not need to be super precise, just getting things in the neighborhood is usually good enough.

Repointed this to elasticity because 3.1 still has the concept of internal compactions.


Omi AI app submission with the marketplace

WE need to provide a demo test email for apple in submissions like<EMAIL_ADDRESS>and in this email the  app marketplace SHOULD be disabled (or at least paid apps should be hidden)
we can NEVER mention apps payments functionality to apple/google in "what's New" notes - if they see it they will trigger
@thinh you need to remove $30/per month and replace it with "subscribe"
when the person clicks on the button, you need to redirect them to a webview/site to actually subscribe
We need to add over-the-air updates as soon as possible so that Apple doesn't trigger
ref: https://discord.com/channels/1192313062041067520/1295877183587221646/1316171505293332490

@mdmohsin7

WE need to provide a demo test email for apple in submissions like<EMAIL_ADDRESS>and in this email the app marketplace SHOULD be disabled (or at least paid apps should be hidden)

for this one.

Accounts created @beastoin

@mdmohsin7 for the submit app logic, pls 1/ rquest the api /v1/app/plans with auth then 2/ check if having at least 1 plan then show the app pricing logic, if empty -> hide it.
ref: https://github.com/BasedHardware/omi/pull/1542/files#diff-2909fc3590a2e91de52c71f77e268a2da5db3a9bc7d936ca349398096a4595eaR292

backend dev deployment: https://github.com/BasedHardware/omi/actions/runs/12335907335


Error: ENOTEMPTY: directory not empty, rmdir

When there are files in the output directory the plugin fails, even though I have clear set to true in config.
Full error:
{ Error: ENOTEMPTY: directory not empty, rmdir '/Users/solar/dev/eth/tug-of-war/solidity/../dapp/src/abi'
    at Object.rmdirSync (fs.js:684:3)
    at OverriddenTaskDefinition._action (/Users/solar/dev/eth/tug-of-war/solidity/node_modules/hardhat-abi-exporter/index.js:31:8)
  errno: -66,
  syscall: 'rmdir',
  code: 'ENOTEMPTY',
  path: '/Users/solar/dev/eth/tug-of-war/solidity/../dapp/src/abi' }

I'm not sure how to reproduce this.  The recursive option is set on the rmdir call, so a non-empty directory should not be an issue.

What version of node are you using?  Looks like the recursive option was introduced in 12.10.0.

Ahh could very well be I'm on node 10. I'll check it when I get home later tonight!

I was on node 10.22. Perhaps you could add node >= 12 as dependency in the readme. Closing!


[FEATURE REQ] add the option to backup lnd with gitea instead of github, or on nextcloud.

What & why
Creating an account on github for the backup does not seem a good idea in terms of privacy, so why not on gitea and/or nextcloud that people can easily self-host?

Creating an account on github for the backup does not seem a good idea in terms of privacy

It's similar with any other centralized service. But recommendation is to use it using Tor only and to create special account for this purpose:

Go to GitHub, sign up for a new user account, or log in with an existing one. If you don’t want for GitHub to know your identity and IP address in relation to your Lightning node, it is recommended to create new account even if you have existing one, and use Tor Browser for this and following steps.

Then part of guide is to configure git for channel backup user to always do anything only via Tor when accessing any git repositories.

git config --global core.sshCommand "torsocks ssh"

But Gitea or anything else using git should be simple, big parts doesn't change, only creating repo and granting access rights.

About self hosting - you should always try to have backups in different physical locations. So it should not be self hosting at the same physical building where your node is located.

About self hosting - you should always try to have backups in different physical locations. So it should not be self hosting at the same physical building where your node is located.

Yes I agree, the idea if to offer more options, but in the end it's the user responsability to mitigate those risks.


I tried pip install ...

and got https://pypi.python.org/pypi/boom2/1.2.0 one of the garbage packages on Pypi ...
would be fine to add in README.rst
 pip install git+https://github.com/tarekziade/boom2

oh it's not ready yet, I have not released the first version. I'll add a warning in the README
Cheers


Add type roots on tsconfig.json

PR #69 did the same thing, but missing on current repo.
Just added typeRoots to tsconfig.json.

Yep looks good, good catch 👍
@all-contributors please add @challenger71498 for code


test: remove optimize deps hack

This PR also adds JetBrains stuff to gitignore.

Do we still need dedup vue? Previously it wasn't there.


Miss cert-dir and creds option for buildah bud

It looks that if the command is operating with a registry, it have options authfile, cert-dir, creds and tls-verify. So I think cert-dir and creds options need to add to buildah bud as well.

@gouyang fixed with #419.  Please reopen if you find otherwise and thanks for the problem report.


Existing deployment breaking with latest deployment with asp.net core 5.0.2

Describe the bug
If I deploy the diacom-server to an existing instance using the latest ARM Template , that upgrades the asp.net core runtime to 5.0.2 and processor to 64bit, I am getting error "HTTP Error 503. The service is unavailable.".
If I manually remove the exiting asp.net core runtime 3.1.X, it works.
How to make it touchless deployment?
To Reproduce
Steps to reproduce the behavior:

Use the  latest ARM Template
Access the site :  https://dicom-websitename.azurewebsites.net.

Expected behavior
Default Page Or HTTP ERROR 404 - If Authentication/Authorization configured.
When I access the health API /api/dicom/health - should get response as {"Status" : "Healthy"}
Actual behavior
When I access the health API /api/dicom/health - getting the following error

We are working to provide better guidance, but currently the templates are for development purposes and do not work for upgrade. Please either redeploy the application or remove the old version of ASP.NET Core extensions from the webapp. We are also following up with the webapp team on best practices in regard to this issue.

closing, we have updated the readme to provide better guidance on the support


Swagger spec doesn't account for paging.

In the swagger spec, for networking for example, there is an implication that the List api returns all results, rather than just the first page.

List the List network security rule operation retrieves all the routes in a route table.

But it doesn't retrieve all the routes. It returns the first page and then one can use ListNextResults to get the subsequent page(s). Also, the generated sentence doesn't really parse in English correctly.
The confusion worsens when one sees the ListAll and reads the description:

ListAll the list NetworkSecurityGroups returns all network security groups in a subscription

But it also doesn't "return all", but rather the first page and then one is expected to use ListAllNextResults to page through the results.
Go-autorest uses the swagger description to form documentation for the Go sdk. This documentation can be a bit misleading due to these descriptions.

@DeepakRajendranMsft - Can you please update the documentation to be in sync with the behavior of the method.

@DeepakRajendranMsft can you please provide some updates here?

@amarzavery and @salameer, It makes no sense to update every instance of List or ListAll in Network. Shouldn't this be documented into overall API docs? Lots of services use List or ListAll and this behavior happens in many places. @liumichelle

Hi @markjbrown,
The Overall microsoft or Azure APIs documentation would definitely make sense if the name of the operations actually described the behavior. the problem is that the use of the naming "List" & "List all" and the behavior of these operations is not consistent so even if we document that at an a higher level it will still be miss leading to the customers.
To sum up, what's being asked here is that the documentation of the method regardless of the name actually describes the behavior of the operation

I see this pattern across RP's so this isn't unique to Networking. Is this something that is standardized across other RP's as well?

To be honest these are generic API and programming good practices and they provide customer pain. regarding other APIs we are pointing out similar issues to the Documentation team and the north star is for continuous improvement whether captured in issues or direct feedback.
I think this effort is low hanging fruit if the fact that it doesn't cause breaking changes and greatly improves the documentation on docs.microsoft.com and the generated SDKs usage experience
Let's Make it AWESOME mark LOL :)

Alright you know what i think I'll negate myself here. I think the issue here is that both names have the same behavior but follow two different naming conventions. Sooo fixing these will introduce breaking changes and not sure about the benefit VS impact here specially for Networking API.
Sooo Mark and @DeepakRajendranMsft  my recommendation is to follow the naming guidlines in the swagger checklist for future new APIs or resource types to have standardized naming.
and @colemickens I think to ease your problem, Go SDK should handle lazy  enumeration or loading regardless of the name and avoid the need to use ListAllNextResults. @marstr and @mcardosos let's follow up on this.
Closing this one for now. please reopen if anyone doesn't agree

Howdy @salameer,
As we discussed offline. We updated our support for the Pageable extension to grab each page lazily as it is requested. This should dramatically reduce the complexity involved with Listing, and should wipe away the concern about this particular type of misleading documentation.

Awesome!
LONG LIVE @marstr


Request: Get tests running or delete them

It seems that development has broken tests some time ago. Should we delete them for now?

yeah the tests are just the default generated stubs, so would make sense to delete the failing ones


Make docs.sciml.ai the default for all of SciML

Now that this is starting to deploy, we need to update everything to be pointing here. For every package, we need:

Update the docs badges to just have a badge pointing to the docs.sciml.ai version
Update the build canonical links to point there https://github.com/SciML/DiffEqFlux.jl/blob/master/docs/make.jl#L25
Add redirects to the new site pages (I'm not quite sure how)
Update the within-package and between-package links to these new pages (to try and avoid the redirects)
Update the Github repo settings so the canonical website link is to the global package docs.

@ArnoStrouwen can you help with this?

Yes, I'm going to a conference this week so these menial tasks like updating badges are a good fit.

Should DifferentialEquations also just get fully merged into SciMLDocs, or do you want to keep a standalone version there?

We should keep a separate version for that for both formatting reasons (so it shows up in the top bar), and because we intend to make all of its tutorials into @examples so it'll be a big build process of its own.

Not yet done:

DiffEqDocs
SciMLBase
SciMLTutorials
SciMLBenchmarks

And the redirects, though the redirects could be their own issue.

I can't do much about the redirects without access to sciml.ai
My guess is it could be done using
https://docs.aws.amazon.com/AmazonS3/latest/userguide/how-to-page-redirect.html.
removing any CNAME records already there and also removing the github pages custom domains in each repo's settings?

I did a test redirect with noise.sciml.ai awhile ago and that worked. So now I setup the rest and in a day or so the DNS update should resolve and this should be done.

Done


Fix for FIPS startup

Addresses #358

here I cannot judge if this is a regression for others / older installations
@torsten-sap  @friegger  we need someone with BOSH expierence to verify that this change does not break existing landscapes

SHA1 and 3DES raises hackles here. Aren't there other ciphers/hashes that aren't so tied to the 20th century?
SHA1 was deprecated nearly a decade ago.


Update app-insights-anomaly.md

Updated content to reflect recent changes.
Will send link to Proactive detection blade and Mockups for images.

Hi @antonfrMSFT, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no contribution license agreement is required at this point. Real humans will now evaluate your PR.

TTYL, AZPRBOT;

@alancameronwills can you work with @antonfrMSFT to fix the merge conflicts in the PR, or if you don't need it anymore please close it. Thanks.


Detach wallet

This contains a HUGE refactor of the daemon with the main purpose of detaching the embedded wallet and make it connect to an external ocean wallet instead. Secondary, this refactors the scaffolding and file naming for app and domain layers in order to increase the readability and maintainability of the codebase.
Changes

All stuff related to Vault and Unspent domain/repo has been dropped.
A new portable interface OceanWallet has been introduced.
Strong refactor of how the grpc services are started to decrease overall code complexity
Refactor of the env vars to configure the daemon at startup
Updates to external pkgs in order to user latest go-elements and psetv2.
New tdex-daemon/v2 proto that requires updates to clients such as the feeder or the dashboard that interact with operator interface.
Updated e2e test in order to use dockerized services instead of compiled binaries.

Note: I'll provide more detailed clues about the macro changes listed above in later comments.
Closes #260.
Closes #309.
Closes #413.
Closes #415.
Closes #482.
Closes #539.
Closes #552.
Closes #654. (Removed env vars to fix asset)
Please @tiero review.

Also maybe we should have docker_compose in root and make dev in Makefile which would start tdexd and ocean

@sekulicd do you have any suggestion for the CodeQL failing action (InsecureSkipVerify)? I tried several things but none of them seemed to work. Not sure why now it's not working now. I didn't actually change that and before this PR it was not giving problems...

CodeQL

Shouldn't //nosec enough?

I solved by manually dismissing those alerts with "Won't fix" reason.

Changes to env vars

Removed:

TRADE_EXPIRY_TIME - the expiration time is given by the ocean wallet when selecting the utxos. At the moment ocean protos don't expect this value to be set within the request message. Until we eventually commit this change in ocean, this env var won't be restored.
EXPLORER_* - The daemon doesn't make use of an explorer service anymore.
BASE_ASSET, QUOTE_ASSET - The daemon doesn't make use of fixed base/quote asset anymore. Any kind of market can be created.
NETWORK, NATIVE_ASSET - The daemon retrieves info about the network from the underlying ocean wallet.
CRAWL_INTERVAL - The daemon doesn't have any underlying component taking care of watching the blockchain anymore.
TRADE_SATS_PER_BYTE - Always defaults to 0.1 sats/VByte. This env var was removed for easiness, but it can (should?) be restored.
MAX_FAILING_REQUESTS, FAILING_RATIO - The daemon doesn't make use of any ratelimiter/circuitbreaker anymore.
RESCAN_* - Wallet restoration isn't yet available since it's a feature under development, but in general maybe better to provide these info in the request message.

Renamed (this refactor was the opportunity to give some env vars a more expressive name):

DATA_DIR_PATH --> DATADIR
DEFAULT_FEE --> PERCENTAGE_FEE
SSL_KEY, SSL_CERT --> TRADE_TLS_KEY, TRADE_TLS_CERT

Added:

WALLET_ADDR - address of the ocean wallet service to connect to
DB_TYPE - type of db to use (for now only badger, soon we will support also postgres)

The daemon doesn't make use of fixed base/quote asset anymore. Any kind of market can be created.

how do we account for it when restoring? how do we know which is base and which is quote?

This env var was removed for easiness, but it can (should?) be restored.

Maybe this should be calculated dynamically looking at meempool? user may want to turn off this as "priority" or not

how do we account for it when restoring? how do we know which is base and which is quote?

The user could pass a list of base assets when restoring a wallet. In general, my intention was to move this kind of config from env var to request message so the user can pass this info at runtime.

Maybe this should be calculated dynamically looking at meempool? user may want to turn off this as "priority" or not

Let's put this on pause for now. We'll create a dedicated ticket as soon as this PR gets merged so we can discuss about it further

Changes to tdex-daemon/v2 proto compared to v1:

Removed

WalletService - this service was meant to be used as operator's personal wallet. He can now use directly ocean for this purpose.
OperatorService/Claim* rpc - not needed anymore because the daemon gets notified about deposits/withdrawals from the ocean wallet. This is a breaking change and requires clients making use of such apis (dashboard) to be updated accordingly.
OperatorService/GetMarketBalance rpc - removed since duplicate of GetMarketInfo.
OperatorService/GetMarketCollectedFees rpc - merged into GetMarketReport.

Renamed:

WalletUnlockerService --> WalletService
WalletUnlockerService/IsReady --> WalletService/GetStatus
OperatorService/GetInfo --> WalletService/GetInfo
OperatorService/Get*Address --> OperatorService/Get*Addresses - moved to plural since the rpc returns a list of newly derived addresses.

Added:

WalletService/LockWallet rpc - useful to lock the daemon and its wallet without switching it off. This action disables every interface but the WalletService one like it was just started and waiting to be unlocked.

Other changes:

Replaced all bytes message fields with string
Fixed all enums with *_UNSPECIFIED = 0 first value as per buf recommended practice.
Expect a list of Receivers in all Withdrawal rpcs to make them consistent.
Use int64 for all timestamp message fields.
Balance returned by OperatorService/GetMarketInfo is expressed as map of pairs <asset_hash, { unconfirmed_balance, confirmed_balance, locked_balance, total_balance }>

Changes to tdex-daemon/v2 proto compared to v1:

cc/ @Janaka-Steph

Getting GetMarketReport failure - RpcError: not implemented, method seems missing.

Getting ListTrades failure - RpcError: Skip must be set to a positive number if page: { number: 0, size: 5 }. Number should be at least 1. What should be the behaviour of number: 0? Error message should be more explicit.

I want to re-discuss/challenge the way we are handling types in different layers in this PR.
Purpose of ports pkg is to isolate core from calls to 3rd party apps/infra.
In this PR we moved almost all types to ports, even some that doesnt "leave" our core.
I will use GetMarketReport method as an example and its arg TimeRange type.
TimeRange is simple struct or DTO used only to pass some data around as a whole, as opposed to several individual variables.
Before this TimeRange was defined as struct with its validation/toDate methods and now it is defined in ports as an interface.
Representing DTO(in this case TimeRange) as an interface, introduces only unnecessary complexity.
In this PR we are doing validation in upper layer(handler) which i find wrong as validation should happen in method using actual struct and not somewhere outside, since like this method is only "protected" in case it is called in this flow.
To summarise things i see problematic:

Representing simple structs as interfaces and then creating additional structs in different layers that implements that interface, dont see any pros only adding complexity
Moving types to ports that are not "leaving" core
Validation structs outside of method using actual struct, not protecting method using validated struct if called from somewhere else, eg. test

Please share your thoughts.
@tiero @altafan

2. Moving types to ports that are not "leaving" core

I agree on this, better to have little code replicated, but at least we can swap all layers without breaking anything in the future.

Validation structs outside of method using actual struct

+1

* Getting `ListTrades failure - RpcError: Skip must be set to a positive number` if `page: { number: 0, size: 5 }`. Number should be at least 1. What should be the behaviour of `number: 0`? Error message should be more explicit.

@Janaka-Steph fixed.

Purpose of ports pkg is to isolate core from calls to 3rd party apps/infra.
In this PR we moved almost all types to ports, even some that doesnt "leave" our core.

At the beginning, I didn't want to make the stuff defined in ports be used like cross-layer types.
You have to consider that most of the interfaces defined in ports are related to the WalletService (ocean), like for example WalletAccount, TxInput, TxOutputs, etc.
Even if we get back to the prev solution (ie. each layer has its own types, even if exactly the same), most of those interface will stay there anyway because they're related to an external service.
Then, it turned out that most of these interfaces are very similar to the proto types used by the interfaces layer, while the app layer have to deal with them in any case since it has to interact with the external wallet. Because of these reasons, I decided to use the interfaces as cross-layer types.
The main advantage of this approach is that we reduce the amount of repeated code, therefore the redundancy of potential bugs, and also allows us to write validators only once for any of those types.

Validation structs outside of method using actual struct, not protecting method using validated struct if called from somewhere else, eg. test

I definitely agree with this and I'm glad it came out because I just forgot about it. As I said just above, it would be enough to add validators (and related unit tests) and use them in both interface and application layers where the types are used.


[Joy][Table] Striped, hover, etc. don't adhere to border radius

Duplicates

[X] I have searched the existing issues

Latest version

[X] I have tested the latest version

Steps to reproduce 🕹
Link to live example: https://mui.com/joy-ui/react-table/#usage-with-sheet
Steps:

Enable striped, hover, or change the body color of a table

Current behavior 😯
When used in a Sheet (or other element), the bottom border radius will be 0 and as a result escape the radius of the parent element.
Expected behavior 🤔
When a table is in a Sheet or Card it should follow the border radius for the relevant fields.
Context 🔦
Including a table with anything but the basic setup will make the table bleed out of its component.
Your environment 🌎
No response

Thanks for the feedback. I decided not to make the border-radius follow the Sheet because it is not 100% for every use cases.
For example:
Adding padding top,bottom to the Sheet does not need a border radius on the table.

I think a proper solution for this case (similar to the List component) is to have utility styles that internally integrate with the component for developers to use.
import { applyTableCellRadius } from '@mui/joy/Table';

 tr:last-child > td:first-child': { …applyTableCellRadius(['bottom-left']) } }}
>…

Optimized for bundle size. You get what you use.
Joy provides the tools, you have full control of them.


Update the README

This updates the README to match the OSS advertisement at Stack Overflow. It also wipes out the example API keys to prevent any possible confusion from those who do not read the text below the block.

Coverage remained the same when pulling 7aa09d335b469d99f728cddcd9c19240edcccd6d on kevin-brown:patch_1 into 075e282ec7ffda0162d0ac86f684676769735be2 on gunthercox:master.

Awesome, looks great!


Feat: add support for Cloudflare Workers via --workerd option

Currently, the bindings generated by wasm-bindgen cannot be used on Cloudflare Workers out of the box (see "JavaScript Plumbing" section in the docs). This is a pain for developers, as shown in https://github.com/rustwasm/wasm-bindgen/issues/2972.
This PR proposes a new --workerd option for --target bundler that closes https://github.com/rustwasm/wasm-bindgen/issues/2972.
Tests and markdown documentation updates are included.
Example
Consider a wasm32-unknown-unknown crate called cf-worker.
The current cf_worker.js binding generated by wasm-bindgen --target bundler is:
import * as wasm from "./cf_worker_bg.wasm";
import { __wbg_set_wasm } from "./cf_worker_bg.js";
__wbg_set_wasm(wasm);
export * from "./cf_worker_bg.js";

This PR, following Cloudflare's own documentation and using wasm-bindgen --target bundler --workerd, creates the following binding instead:
import * as imports from "./cf_worker_bg.js";
import wkmod from "./cf_worker_bg.wasm";
const instance = new WebAssembly.Instance(wkmod, { "./cf_worker_bg.js": imports });
imports.__wbg_set_wasm(instance.exports);
export * from "./cf_worker_bg.js";

which is indeed compatible with Cloudflare Workers.

Unfortunately I know nothing about bundlers, so I will need some help understanding this.
Considering this PR doesn't actually touch the JS shim, only the final bundler file (the JS file without _bg), can't users not add this on their own before using the bundler? It seems straightforward to me, it's basically like users having to add the index.html file when using the web target.
I think I already addressed the original painpoint in https://github.com/rustwasm/wasm-bindgen/issues/2972#issuecomment-1625180501.

I guess one question I have is what would accomplishing the same thing look like from the user's perspective without this PR? Like, what steps would they have to take?

Like, what steps would they have to take?

Following the proposed solution of this PR, the only step the user would have to take is replace the output file with this:
import * as imports from "./cf_worker_bg.js";
import wkmod from "./cf_worker_bg.wasm";
const instance = new WebAssembly.Instance(wkmod, { "./cf_worker_bg.js": imports });
imports.__wbg_set_wasm(instance.exports);
export * from "./cf_worker_bg.js";

Some considerations for @asimpletune and @daxpedda:

this PR doesn't actually touch the JS shim

That's correct. Cloudflare Workers only require changes to the (very small) bundler file.

the only step the user would have to take is replace the output file with this

import * as imports from "./cf_worker_bg.js";
import wkmod from "./cf_worker_bg.wasm";
const instance = new WebAssembly.Instance(wkmod, { "./cf_worker_bg.js": imports });
imports.__wbg_set_wasm(instance.exports);
export * from "./cf_worker_bg.js";

Essentially, yes, keeping in mind that cf_worker is an example name and not a constant string.
Let's call the file above bundler-template.js and refer to the variable shim filename (without the postfix _bg.js part) as $WASM_SHIM_NAME. In this example, WASM_SHIM_NAME=cf_worker.

what would accomplishing the same thing look like from the user's perspective without this PR?

run wasm-bingen --target bundler
in the output folder, replace $WASM_SHIM_NAME.js's content with bundler-template.js' content
run a :s/cf_worker/$WASM_SHIM_NAME/g search&replace command on $WASM_SHIM_NAME.js

However, I'd like to point out that:

Cloudflare Workers explicit refer to wasm-bindgen in their documentation
Cloudflare Workers are one of the major platforms for deploying Wasm modules online in a serverless fashion
I think Cloudflare Workers are big enough to justify a first-class support (which, as you can see, doesn't require major changes in wasm-bindgen's source code) without having to resort to third-party utils / Wasm binding patch scripts.

This PR is an effort to make the task of deploying Rust+Wasm modules online more user-friendly :)

I see, and they would have to redo that process for every build?
I ask just because my initial reaction was that it felt a little strange to something that is vendor-specific as a first-class CLI option. Maybe, there's a way to accomplish the same thing, but to just have it as an argument?
Then cloudflare (or whomever) can just update their documentation, and if anything ever changed on Cloudflare's side of things they'd naturally be responsible for supporting those.

Maybe, there's a way to accomplish the same thing, but to just have it as an argument?

I wonder what the use cases outside of CF are that would be able to make use of the same thing.

Cloudflare Workers explicit refer to wasm-bindgen in their documentation

Not sure why this justifies us supporting it. Rather this is an issue on their side.

Cloudflare Workers are one of the major platforms for deploying Wasm modules online in a serverless fashion

True.

I think Cloudflare Workers are big enough to justify a first-class support (which, as you can see, doesn't require major changes in wasm-bindgen's source code) without having to resort to third-party utils / Wasm binding patch scripts.

I don't disagree, but on the other hand it would be better if they provide support when they are doing non-standard stuff.
Case in point: worker-build. From personal experience I know that Cloudflare itself doesn't maintain it very well, but it seems better to me to let them support it as long as we provide the tools necessary to do so.
I think it's gonna be problematic if the JS shim needs adjustment, then I would lean in favor of providing better support, but if it's only the bundle file, I think some effort from Cloudflare's side is appropriate.
I'm currently against merging this PR:

The web target already works as intended.
The bundle target can be used by modifying/fixing the bundler file.
A dedicated tool for Cloudflare Workers already exists: worker-build. So this should be out of our hands to begin with, aside from making sure we continue to support such tools.

I also want to note here that this problem in general has been a big issue in wasm-bindgen. The Web is pretty diverse and has a lot of very different tools. wasm-bindgen already supports a bunch of tools I personally have no clue about and it's already unmaintainable as-is, widening the scope even further is not really a good idea imo.
Instead we should push further towards making wasm-bindgen as agnostic as possible so other tools can be built on top of it to support all these diverse needs. worker-build is a perfect example on how we should handle this imho.

Thanks for your feedback @daxpedda. I see you're heavily hinting at moving this PR towards the worker-build tool. To shed additional context on this - Cloudflare Workers support WebAssembly in two different scenarios:

When defining workers in JavaScript/TypeScript, you can import Wasm modules (which, referencing this PR, requires wasm-bindgen --target bundler --workerd)
When defining workers purely in Rust via workers-rs, to compile to Wasm them via worker-build and upload them.

Afaik, the worker-build tool is only useful in this latter case. This PR addresses the first case.
I'll ask people from Cloudflare to chime in - whether this PR's logic ships in wasm-bindgen or in Cloudflare's own infrastructure is rather indifferent to me, as long as all Cloudflare Workers devs can avoid an additional "glue code patching" step (as it's currently the case).

When defining workers in JavaScript/TypeScript, you can import Wasm modules (which, referencing this PR, requires wasm-bindgen --target bundler --workerd)

That indeed would make sense to me then.

I'll ask people from Cloudflare to chime in - whether this PR's logic ships in wasm-bindgen or in Cloudflare's own infrastructure is rather indifferent to me, as long as all Cloudflare Workers devs can avoid an additional "glue code patching" step (as it's currently the case).

Great, thank you!

I'm not a fan of this. What about environments other than workerd? Cloudflare isn't the only one providing a worker environment where wasm-bindgen generated code will run

Is there any updates on this?
I'm able to run the tests on firefox with wasm_bindgen_test_configure!(run_in_worker);
or wasm_bindgen_test_configure!(run_in_browser);
But I wanted to run at least some of them on the workerd, because of the workerd extra stuff, like D1.
Is there a way for the runner to load them on workerd?
I believe that one way or another, this goes through either forking your runner or improving your runner to handle it.

I think both @hamza1311 and I covered how we want to proceed in previous posts.
If anything is unclear let us know.

Apologies, you are right your case is different, I didn't pay enough attention.
I think the same philosophy applies to the test runner as well. The preference here it not to have to cover every use case there is in wasm-bindgen. So as you said, CF should provide their own test runner.
That said, if the wasm-bindgen test runner can be adjusted to provide a general way for other runtimes/platforms to inject what they need to get things to work, that's great as well.
That would obviously require some design work, but considering the lack of time from maintainers this has to be done in small steps that can be easily discussed and reviewed.

I'm already in contact with someone from CF to see how this can be done, his comment was that my approach seemed reasonable.
But at the moment it seems they don't export enough yet to run the tests, but this is something that I do require, so I'm going to start experimenting soon anyway.
I'll update you when I have relevant info.

using wasm-bindgen produced test files to run the tests with your test runner using workerd is a bit of a different use case.

That is a use case I would love to support. There's also a point to be made of running wasm-bindgen output in non-node.js environments like Deno or Bun. I think this PR is out of scope for this discussion. Can you create a new issue for this so we can discuss how to handle this? I think we can take inspiration from the Jasmine testing framework in the JS ecosystem

I'm going to close this due to inactivity.
The request is continuing to be tracked in https://github.com/rustwasm/wasm-bindgen/issues/3891.


[#41] Add Kerberos example to README

Addresses #41
I've kept the commits separated for review purposes but I think we can just squash to one when this is ready.
The instructions are based on my own setup, so they have been confirmed to work on Ubuntu 24.04.

If you're happy with it, attach an issue number to that one commit or squash it.

Squashed

#'d, mergin


Use prometheus.io/scrape annotations for k8s service discovery

I'd like to configure Prometheus Operator to use prometheus.io/scrape annotations for Kubernetes Service Discovery. Is Prometheus Operator opinionated in using Service Monitors here or is there a way to configure a new Prometheus to do service discovery off of these annotations?

Question answered! https://github.com/coreos/kube-prometheus/pull/16#issuecomment-305953469


Trim filename in status line

Hey, thanks for this great plugin. I wanted to ask is there a way to trim the filename in the status line to exclude the relative path completely and only display the filename? I have a really nested file and currently the status line looks like p/w/c/s/A/M/C/C/C/S/filename

Sorry for the late reply, busy week on my end.
This statusline plugin is designed to display shortened relative paths for nested files and pathless files only for the current directory. So for now your problem has no solution (by design).
My preference is to keep that basic architecture. However, directory path trimming for very long paths does seem reasonable. Maybe something like four path components at most, for example:

p/w/c/filename --> p/w/c/filename (no change)
p/w/c/s/filename ---> .../w/c/s/filename
p/w/c/s/A/M/C/C/C/S/filename ---> .../C/C/S/filename

Ellipses, ..., like in Bash PROMPT_DIRTRIM, can be used to indicate truncation.
Would that suffice? Personally I would probably just implement this and make it the default.

Try the latest update, it now trims paths to a maximum of four components which should largely mitigate your issue whilst also keeping the functionality I like.
Cheers.

Thanks for getting back on this @bluz71 really appreciate it. And yes this will do thank you so much for making this change :D I also wanted to ask is there a way to display the git branch name towards the left of the filename like how airline/lightline do it?

I also wanted to ask is there a way to display the git branch name towards the left of the filename like how airline/lightline do it?

The positioning of the Git branch is a distinct layout choice which I would like to keep as is (basically because I like it where it is). I also prefer to keep the plugin as simple as possible so I am unlikely to offer an option to change it.
Your best bet would be to fork and move the Git branch to where you like if it is motivating enough.
Cheers.


Error when loading pretrained model for finetunning from a checkpoint of the pretrained model

Hi,
Very simple issue, this error:
"ValueError: loaded state dict contains a parameter group that doesn't match the size of optimizer's group"
Is displayed when I'm trying to load a a pre-trained model for finetunning while doing so from a checkpoint folder located in the pretrained model's folder. When loading the model from the  "root" folder of the pre-trained model (which contains the checkpoints) the finetunning is performed fine.
The error is thrown before the start of the training.
To reproduce, simply follow the steps in the example in README.md including the pretraining (just set to lower number of epochs) and then for the finetunning at step 3.3 set the path of the model to one of the checkpoint folders like:
If for the pretraining the output folder was set to:
export OUTPUT_PATH=output$KMER
Then for finetunning set:
export MODEL_PATH=output$KMER/checkpoint-1800/

Hello,
I'm having this same issue. I think there's a problem fine-tuning DNABERT from a checkpoint rather than training from a completed loop. Have you found an issue to this? What do you mean by loading the model from the "root" folder of the pre-trained model? Are you referring to the given sample pre-trained models?


Check to install package

Maybe we should ask Tiffany about that in order not to lose points for late submission


I would like a description of the "BackEnabled" property.

[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e4bbd1e3-f15f-e855-e4ee-2e10fd22920d
Version Independent ID: 48aa97f4-bceb-7ac9-b2ce-49cf721519cf
Content: App object - Power Apps
Content Source: powerapps-docs/maker/canvas-apps/functions/object-app.md
Service: powerapps
GitHub Login: @gregli-msft
Microsoft Alias: gregli

@s-kawara Thanks for reaching to us. I have searched for the word BackEnabled in the doc, but didn't find any reference? Can you share me the exact link to the doc that you have issues with.

Thanks for contacting us. As we haven’t heard back from you, I am closing this issue. If you need further help, please reopen the issue or open a new issue.


adds issue #1: feature: new data column for delivery_date

adds delivery_date to frontend and backend.

The DB table must be updated to include the extra field.


Could this package be merged with atomic-primops?

Hey! I just thought I'd float the idea of merging this package with atomic-primops, which also exposes a similar counter type.
It seems this package has a default cmm implementation, whereas atomic-primops does not. Perhaps that code could be contributed to atomic-primops as well?
Having two atomic counter packages is of course not the end of the world, but it would be (slightly) better for ecosystem discoverability and cohesion to have just one.

The point of atomic-counter is pretty silly: I wanted an atomic counter myself, tried to search for it on the Internet/Hackage and failed miserably. Stated simply, the idea is discoverability so that obvious search term will lead one to the package that does it. Maybe I'm just bad at searching on the Internet.
On a different note, some, perhaps questionable, design decisions were made in this package that will likely not feel at home in atomic-primops: the distinction between API in IO, ST and a raw one on Unlifted kind for guaranteed absence of any overheads. I can totally see why someone would not want to provide all 3 in their package, but I at least did use the Unlifted-kind-based one myself.
If anyone wants to contribute cmm to atomic-primops I'm all for it. Although this whole this is a bit experimental overall (i.e. could potentially be retracted, altough the results so far confirm that it works OK so most likely wouldn't).

I'm cleaning up stale issues, please reopen if you feel it's still relevant.


Block a keyboard for GameActivity

Remove the ability to hide a keyboard for GameActivity.

There is no a standard way to do it. I am really afraid of using hacks like https://stackoverflow.com/questions/25216749/soft-keyboard-open-and-close-listener-in-an-activity-in-android because it can lead to problems with returnig back on some devices. We discussed it with @greenEkatherine  and decided not to implement it because in her opinion it is ok to close keyboard with help of an additional action.

There is no a standard way to do it. I am really afraid of using hacks like https://stackoverflow.com/questions/25216749/soft-keyboard-open-and-close-listener-in-an-activity-in-android because it can lead to problems with returnig back on some devices. We discussed it with @greenEkatherine  and decided not to implement it because in her opinion it is ok to close keyboard with help of an additional action.


Use built-in Cloud Build vars in e2e-testing module

Thanks for stopping by to let us know something could be better!
PLEASE READ: This repository is dedicated to a sample application that uses multiple Google Cloud products. We will only be able to address issues related to this application, not to underlying issues in a client library or one of the products. Please visit the Support page to reach the most relevant engineers.
Problem
The build triggers for the e2e-testing module use a substitution variable _PROJECT to specify a project, and is currently set to the ops project id. We should use the built-in variable $PROJECT_ID instead, with no substitutions.
Expected behavior
The _PROJECT variable is used in the AR repo URL. We don't need to this to be a variable since this since it will always be the AR repo in ops project that's running the cloud build job.
Additional context
See https://cloud.google.com/build/docs/configuring-builds/substitute-variable-values

fixed by #762


Create a page for First Time Contributors

Create a page for first time contributors, explaining important topics for beginners such as:

first steps with git
how to prepare local environment with conda
create a local folder for development e.g. ~/dev
CI
run tests locally
pre-commit

@Satarupa22-SD feel free to share any content here that maybe would be helpful for first time contributors

okay!

@xmnlab @DanielaIgRo Please take a look at this page.

thanks @Satarupa22-SD ! very good resources!

we should also recommend to use git config --global pull.rebase true configuration


PHP cannot make concurrent calls

When making Unary method calls using PHP gRPC, it is not possible to start a call without also blocking on it, even though the UnaryCall object provides separate start() and wait() methods.
For example, the following Speech code:
    $startTime = microtime(true);
    $call = $stub->Recognize($request, [], ['call_credentials_callback' => $tokenCallback]);
    $callTime = microtime(true);
    $result = $call->wait();
    $endTime = microtime(true);
    echo "Time to call: " . ($callTime - $startTime) . PHP_EOL;
    echo "Time to wait: " . ($endTime - $callTime) . PHP_EOL;

produces this output:
    Time to call: 1.3245630264282
    Time to wait: 0.0020620822906494

(see full code here: https://gist.github.com/michaelbausor/d3baa9db9e981949b0eded149a515e23)
The expected output would be that Time to call would be only the time required for gRPC to start the call, and that Time to wait would be the much longer time required for the Speech API to process the request.
Currently, this makes it impossible to start multiple calls in parallel. For example, this code:
$requests = [*Array of N speech recognize requests*]
$calls = []
foreach ($requests as $request) {
  $calls[] = $stub->Recognize($request, [], ['call_credentials_callback' => $tokenCallback]);
}
foreach ($calls as $call) {
  $call->wait();
}

will always execute N requests in series, and thus take time equal to N * average_request_length. The expected behaviour would be that this would take time approximately equal to the time taken by the longest request.
This seems to be caused because currently the UnaryCall object sends the OP_RECV_INITIAL_METADATA code in the start() method. (See https://github.com/grpc/grpc/blob/master/src/php/lib/Grpc/UnaryCall.php#L45). This causes calls to the start() method to block.
Moving the OP_RECV_INITIAL_METADATA code from the start() method to the wait() method seems to resolve this issue.
cc @stanley-cheung @dwsupplee @jdpedrie @bshaffer

cc @murgatroid99

One thing to note regarding those measurements is that a full second is an extreme amount of time for a single call, which probably means that this is the first call you're making with a channel. If that is the case, then the call time you're measuring includes the time it takes to establish a connection. I expect that you will get somewhat different results if you make a call before you start measuring.
Regardless of that, there would be some benefit to moving OP_RECV_INITIAL_METADATA to the wait method as you suggest. If there are only send operations in the start method, then it will usually not have to wait for a network round trip. However, the current unary call API guarantees that once the call has been started, the metadata is present.
One possible solution would be to have the getMetadata method itself call the OP_RECV_INITIAL_METADATA code if it has not already been read.

@murgatroid99 Repeating the call does reduce the time a small amount (to about 0.8-0.9 seconds), but not significantly - see results below for looping 10 times.
In general, machine learning APIs such as Speech can make very long calls, with the duration being entirely dependent on the input. The synchronous recognition request for Speech can accept audio of up to around 1 minute in length, and according to the docs it is expected that "30 seconds of audio is processed in 15 seconds on average" (see here).
More significantly though is the second example, of making multiple calls in parallel. Using Pubsub instead of Speech, with the current gRPC code it takes approximately 20 seconds to make 1000 calls to the Publish method. Making the modification of moving OP_RECV_INITIAL_METADATA reduces this to around 1.6 seconds.
Having getMetadata make the call makes a lot of sense to me. We could then also conditionally include the OP_RECV_INITIAL_METADATA flag in wait(), similar to what is done for BidiStreamingCall::read() (https://github.com/grpc/grpc/blob/master/src/php/lib/Grpc/BidiStreamingCall.php#L49)
Output of looping Speech test 10 times:
Time to call: 1.2568538188934
Time to wait: 0.0019960403442383
Time to call: 0.90999507904053
Time to wait: 0.00068497657775879
Time to call: 0.89781498908997
Time to wait: 0.00091099739074707
Time to call: 0.92992115020752
Time to wait: 0.00061488151550293
Time to call: 0.90489912033081
Time to wait: 0.0004119873046875
Time to call: 0.91139006614685
Time to wait: 0.00066804885864258
Time to call: 0.91404891014099
Time to wait: 0.00060796737670898
Time to call: 0.93215799331665
Time to wait: 0.00067496299743652
Time to call: 0.90055298805237
Time to wait: 0.00074911117553711
Time to call: 0.9248321056366
Time to wait: 0.00066709518432617

I have created a PR that implements your suggestion for UnaryCall and ServerStreamingCall. Please take a look and let me know if this is what you had in mind: https://github.com/grpc/grpc/pull/12188

WIth the proposed change, what happens if I call start but never call wait? Will the RPC still finish on the server?

@vkedia In my testing of the proposed PR calls that simply execute start do in fact finish on the server - wait is only necessary to view the results of said call.

This is now included in v1.4.7


Error: A "use server" file can only export async functions.

I just started a new project and when I try to load the storefront for the first time, it crashes with Error: A "use server" file can only export async functions..
This happens right after running the setup with npm run dev, so no idea what's going on. NodeJS is v20.5.0 and tried with bun also.
Stack trace:
><EMAIL_ADDRESS>dev
> next dev -p 8000

next.config.js {}
next.config.js {}
   ▲ Next.js 14.1.2
   - Local:        http://localhost:8000
   - Environments: .env.local

/bin/sh: yarn: command not found
 ✓ Ready in 2.4s
 ○ Compiling /src/middleware ...
 ✓ Compiled /src/middleware in 2.3s (3797 modules)
 ⚠ Fast Refresh had to perform a full reload due to a runtime error.
 ○ Compiling /[countryCode] ...
 ✓ Compiled /[countryCode] in 2.1s (4549 modules)
 ⚠ Fast Refresh had to perform a full reload due to a runtime error.
 ⨯ node_modules/next/dist/build/webpack/loaders/next-flight-loader/action-validate.js (24:18) @ ensureServerEntryExports
 ⨯ Error: A "use server" file can only export async functions.
Read more: https://nextjs.org/docs/messages/invalid-use-server-value
    at ensureServerEntryExports (webpack-internal:///(rsc)/./node_modules/next/dist/build/webpack/loaders/next-flight-loader/action-validate.js:24:19)
    at eval (webpack-internal:///(rsc)/./src/lib/data/index.ts:564:91)
    at (rsc)/./src/lib/data/index.ts (/Users/marko/Dev/medusa-storefront/.next/server/app/[countryCode]/(main)/page.js:698:1)
    at __webpack_require__ (/Users/marko/Dev/medusa-storefront/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./src/modules/layout/templates/footer/index.tsx:9:67)
    at (rsc)/./src/modules/layout/templates/footer/index.tsx (/Users/marko/Dev/medusa-storefront/.next/server/app/[countryCode]/(main)/page.js:1115:1)
    at __webpack_require__ (/Users/marko/Dev/medusa-storefront/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./src/app/[countryCode]/(main)/layout.tsx:8:90)
    at (rsc)/./src/app/[countryCode]/(main)/layout.tsx (/Users/marko/Dev/medusa-storefront/.next/server/app/[countryCode]/(main)/page.js:577:1)
    at Function.__webpack_require__ (/Users/marko/Dev/medusa-storefront/.next/server/webpack-runtime.js:33:42) {
  type: 'Error',
  page: '/us'
}
null
 ⨯ node_modules/next/dist/build/webpack/loaders/next-flight-loader/action-validate.js (24:18) @ ensureServerEntryExports
 ⨯ Error: A "use server" file can only export async functions.
Read more: https://nextjs.org/docs/messages/invalid-use-server-value
    at ensureServerEntryExports (webpack-internal:///(rsc)/./node_modules/next/dist/build/webpack/loaders/next-flight-loader/action-validate.js:24:19)
    at eval (webpack-internal:///(rsc)/./src/lib/data/index.ts:564:91)
    at (rsc)/./src/lib/data/index.ts (/Users/marko/Dev/medusa-storefront/.next/server/app/[countryCode]/(main)/page.js:698:1)
    at __webpack_require__ (/Users/marko/Dev/medusa-storefront/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./src/modules/layout/templates/footer/index.tsx:9:67)
    at (rsc)/./src/modules/layout/templates/footer/index.tsx (/Users/marko/Dev/medusa-storefront/.next/server/app/[countryCode]/(main)/page.js:1115:1)
    at __webpack_require__ (/Users/marko/Dev/medusa-storefront/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./src/app/[countryCode]/(main)/layout.tsx:8:90)
    at (rsc)/./src/app/[countryCode]/(main)/layout.tsx (/Users/marko/Dev/medusa-storefront/.next/server/app/[countryCode]/(main)/page.js:577:1)
    at Function.__webpack_require__ (/Users/marko/Dev/medusa-storefront/.next/server/webpack-runtime.js:33:42) {
  type: 'Error',
  page: '/us'
}
null
 ⨯ node_modules/next/dist/build/webpack/loaders/next-flight-loader/action-validate.js (24:18) @ ensureServerEntryExports
 ⨯ Error: A "use server" file can only export async functions.
Read more: https://nextjs.org/docs/messages/invalid-use-server-value
    at ensureServerEntryExports (webpack-internal:///(rsc)/./node_modules/next/dist/build/webpack/loaders/next-flight-loader/action-validate.js:24:19)
    at eval (webpack-internal:///(rsc)/./src/lib/data/index.ts:564:91)
    at (rsc)/./src/lib/data/index.ts (/Users/marko/Dev/medusa-storefront/.next/server/app/[countryCode]/(main)/page.js:698:1)
    at __webpack_require__ (/Users/marko/Dev/medusa-storefront/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./src/modules/layout/templates/footer/index.tsx:9:67)
    at (rsc)/./src/modules/layout/templates/footer/index.tsx (/Users/marko/Dev/medusa-storefront/.next/server/app/[countryCode]/(main)/page.js:1115:1)
    at __webpack_require__ (/Users/marko/Dev/medusa-storefront/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./src/app/[countryCode]/(main)/layout.tsx:8:90)
    at (rsc)/./src/app/[countryCode]/(main)/layout.tsx (/Users/marko/Dev/medusa-storefront/.next/server/app/[countryCode]/(main)/page.js:577:1)
    at Function.__webpack_require__ (/Users/marko/Dev/medusa-storefront/.next/server/webpack-runtime.js:33:42) {
  type: 'Error',
  page: '/us'
}
null
 ✓ Compiled /_error in 303ms (4711 modules)
 ⚠ Fast Refresh had to perform a full reload due to a runtime error.
 ⨯ node_modules/next/dist/build/webpack/loaders/next-flight-loader/action-validate.js (24:18) @ ensureServerEntryExports
 ⨯ Error: A "use server" file can only export async functions.
Read more: https://nextjs.org/docs/messages/invalid-use-server-value
    at ensureServerEntryExports (webpack-internal:///(rsc)/./node_modules/next/dist/build/webpack/loaders/next-flight-loader/action-validate.js:24:19)
    at eval (webpack-internal:///(rsc)/./src/lib/data/index.ts:564:91)
    at (rsc)/./src/lib/data/index.ts (/Users/marko/Dev/medusa-storefront/.next/server/app/[countryCode]/(main)/page.js:698:1)
    at __webpack_require__ (/Users/marko/Dev/medusa-storefront/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./src/modules/layout/templates/footer/index.tsx:9:67)
    at (rsc)/./src/modules/layout/templates/footer/index.tsx (/Users/marko/Dev/medusa-storefront/.next/server/app/[countryCode]/(main)/page.js:1115:1)
    at __webpack_require__ (/Users/marko/Dev/medusa-storefront/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./src/app/[countryCode]/(main)/layout.tsx:8:90)
    at (rsc)/./src/app/[countryCode]/(main)/layout.tsx (/Users/marko/Dev/medusa-storefront/.next/server/app/[countryCode]/(main)/page.js:577:1)
    at Function.__webpack_require__ (/Users/marko/Dev/medusa-storefront/.next/server/webpack-runtime.js:33:42) {
  type: 'Error',
  page: '/us'
}
null
 ⨯ node_modules/next/dist/build/webpack/loaders/next-flight-loader/action-validate.js (24:18) @ ensureServerEntryExports
 ⨯ Error: A "use server" file can only export async functions.
Read more: https://nextjs.org/docs/messages/invalid-use-server-value
    at ensureServerEntryExports (webpack-internal:///(rsc)/./node_modules/next/dist/build/webpack/loaders/next-flight-loader/action-validate.js:24:19)
    at eval (webpack-internal:///(rsc)/./src/lib/data/index.ts:564:91)
    at (rsc)/./src/lib/data/index.ts (/Users/marko/Dev/medusa-storefront/.next/server/app/[countryCode]/(main)/page.js:698:1)
    at __webpack_require__ (/Users/marko/Dev/medusa-storefront/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./src/modules/layout/templates/footer/index.tsx:9:67)
    at (rsc)/./src/modules/layout/templates/footer/index.tsx (/Users/marko/Dev/medusa-storefront/.next/server/app/[countryCode]/(main)/page.js:1115:1)
    at __webpack_require__ (/Users/marko/Dev/medusa-storefront/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./src/app/[countryCode]/(main)/layout.tsx:8:90)
    at (rsc)/./src/app/[countryCode]/(main)/layout.tsx (/Users/marko/Dev/medusa-storefront/.next/server/app/[countryCode]/(main)/page.js:577:1)
    at Function.__webpack_require__ (/Users/marko/Dev/medusa-storefront/.next/server/webpack-runtime.js:33:42) {
  type: 'Error',
  page: '/us'
}
null

Having exactly the same error after loading the storefront for the first time.

Hey guys, I suspect this was caused by a redundant "use server" declaration in src/lib/data/index.ts. I've just pushed a fix, lmk if that solves the issue!

I also had to comment use server in /src/lib/data/index.ts.

Great, that fixed the issue for me. Thanks for the fast fix.


No exception information returned in Stream mode

The api returns a JSON when error, eg.
{
  "error": {
    "message": "'user_role' is not one of ['system', 'assistant', 'user'] - 'messages.0.role'",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }
}

I use following  code to fix
    def self.to_json_stream(user_proc:)

      proc do |chunk, overall_received_bytes, env|
        Rails.logger.debug("raw chunk: #{chunk}")

        if chunk.start_with?("data: {")
          chunk.each_line do |line|
            next unless line.start_with?("data: {")
            line.delete_prefix!("data: ")
            user_proc.call(JSON.parse(line))
          end
        elsif chunk.include?('"error": ')
          user_proc.call(JSON.parse(chunk))
        else
          Rails.logger.warn("unkown raw chunk: #{chunk}")
        end
      end
    end

The stream data may be split, here is the complete solution: https://gist.github.com/lingceng/afd30be6e102442fa48abf175efe3f6a
Any one can help merge to the project.
@itsderek23 @obie

FYI I have an open PR to solve the same issue, using a different approach: https://github.com/alexrudall/ruby-openai/pull/275

I'm working on properly fixing these issues with #338 (and I'll follow up with a breaking change to raise actual exceptions)

Released in v6


Integrating findbugs and fixing some of the reported issues.

There are way too many issues, so for now I have fixed whatever I found to be real issues or easy to fix.  Some packages and issue types are excluded, but this can be taken up in future and fixed.

And if findbugs also have max violation count, let's just set it and not exclude any packages. We may not have chance to include already excluded packages.

@HeartSaVioR addressed review comments and changed the module structure as suggested.

Btw, does findbugs cover checkstyle too? Some style issues seem to be also caught here.
Show outdated

It does not check for style issues. Only bug patterns.

And if findbugs also have max violation count, let's just set it and not exclude any packages. We may not have chance to include already excluded packages.

I could not find an option specify the number of allowed bugs. Yes the package level excluding needs to be removed. Maybe we can exclude specific classes. This can be taken up in future.


Support for PIMCore 10?

Just curious if it support PIMCore 10.x?

@Sen-Gupta did you try it on Pimcore X already?


SPIKE: Test improvements to navigation

Background
As part of the Admin refresh, we'd like to experiment with bringing the left hand menus up to the top of the screen so that we can use the full width of the screen to display exercise content. We'd like to achieve a layout like this:

Feature(s) Description
Experiment with the Exercise sub menu with the aim of delivering the above configuration. Ideally, we'd like to:

Have the items in the sub-menu display in a grey box underneath the top navigation
Have the selected item in the sub menu appear next to the top level menu item separated by a >
Have a blue line delineating which menu items are selected
The menu that appears could either float on top of screen content or push the screen content down

Hey team! Please add your planning poker estimate with Zenhub @drieJAC @HalcyonJAC @KoWeiJAC @tomlovesgithub

See the following packages if required:
https://www.npmjs.com/package/vue3-tabs-component
https://cristijora.github.io/vue-tabs/#/

Tasks will need some logic to decide which items should be visible