Queue output not readable after update to 9.42 in PHPStorm

Laravel Version: 9.42.2
PHP Version: 8.1.9
Database Driver & Version: MySQL 8.0.28
IDE: PHPStorm 2022.2.4
OS: Windows 10

Description:
Hello,
after upgrading Laravel localy to version 9.42 from 9.39 the queue:work command is not rendered correctly in PHPStorm. Either cmd nor powershell can be used for it because it can not be read. I'm fine with it but PHPStorm should work as an IDE and it worked before. My guess is is the PR https://github.com/laravel/framework/pull/44971 that broke it. Before I updated it looked like the screenshots in the PR. I hope this can be fixed. If more information is required I will try to provide them.
Screenshot PHPSotrm

Screenshot CMD

@nunomaduro reverted back to 9.41:

@PHPGuus You think you can try to reproduce this issue locally? Or assist this customer? For info, this change (https://github.com/laravel/framework/pull/44971) was released 14 days ago (9.41), and it should not cause any issues like the one the customer is describing.

Hi @PHPGuus,
output of the chcp on a CMD Promp:
Aktive Codepage: 850.
I'm using PHPStorm and not VSCode, can that make a differnece?
This is one of my PHPStorm installations:
PhpStorm 2022.3
Build #PS-223.7571.212, built on December 1, 2022
Licensed to xxxxxxxxxxxxxxx
Subscription is active until October 16, 2023.
Runtime version: 17.0.5+1-b653.14 amd64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
Windows 10 10.0
GC: G1 Young Generation, G1 Old Generation
Memory: 4064M
Cores: 16
Registry:
    debugger.new.tool.window.layout=true
    run.processes.with.pty=TRUE
    ide.experimental.ui=true

Non-Bundled Plugins:
    com.mnr.java.intellij.idea.plugin.base64helper (2.0.3)
    org.intellij.plugins.postcss (223.7571.117)
    com.intellij.ideolog (<IP_ADDRESS>)
    Statistic (4.2.6)
    com.github.inxilpro.intellijalpine (v0.4.2)
    ru.adelf.idea.dotenv (2022.3)
    de.espend.idea.php.annotation (8.3.0)
    de.beyondco.tinkerwell.tinkerwell (2.0)
    com.laravel_idea.plugin (<IP_ADDRESS>)
    com.pestphp.pest-intellij (1.8.3)
    zielu.gittoolbox (213.10.3)

Still no changes after updating from PHPStorm 2022.2.4 to 2022.3 .
I did some changes to my vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php and reverted line after line back to track down the issue. After I changed this line:
        if (Terminal::hasSttyAvailable()) {
            $this->components->info(
                sprintf('Processing jobs from the [%s] %s.', $queue, str('queue')->plural(explode(',', $queue)))
            );
        }

to not use the if statement, the console output was fine again. No other change was made to the code. Only the if statement needs to be commented out. What do I need to change for the if statement?

Hi @PHPGuus,
our display language is German (Germany)/Deutsch (Deutschland). But a college tested it with English (United Kingdom) as the display language and has the same errors.
Output of stty in PHPStorm, CMD and Powershell (same for my college):
speed 38400 baud; line = 0;
-imaxbel iutf8

The stty command is only available because we need to use Git for windows (and it includes this command). The version is
git version 2.31.1.windows.1
My Powershell version with $PSVersionTable (same for my college):
Name                           Value
----                           -----
PSVersion                      5.1.19041.2364
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.19041.2364
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>

Hi @PHPGuus,
I fixed the problem.
My GIT for Windows version needs to be 2.38.1 or 2.39 because the stty.exe could not handle something from the Termwind extension (my guess). Now it is working all fine. Thanks for the help! You got me on the right track.


plugin no longer in plugins list or plugin hub

actively used plugin last night (and have been using for months)
logged in today and it's completely gone.

Most likely failed to build from the last runelite update, will take a look when I finish work

https://repo.runelite.net/plugins/<IP_ADDRESS>/fashionscape/ log file here
> Task :compileJava
/tmp/pluginhub-package921256631347546234/fashionscape/repo/src/main/java/eq/uirs/fashionscape/data/ItemInteractions.java:476: error: cannot find symbol
		WEAPON_TO_IDLE.put(ItemID.EVENT_RPG, IdleAnimationID.OVERHEAD_LAUNCHER);
		                         ^
  symbol:   variable EVENT_RPG
  location: class ItemID
1 error

item name changed this update.

We're back


Zarf support for EKS

Story:
As a user of EKS and Zarf, I want the ability to deploy Zarf packages to EKS, so that I can use Zarf in my environment.
Acceptance Criteria:

The Games example deploys successfully to EKS (Commercial) (definition of "successfully" is that all pods are running
A new E2E test has been created that deploys the Games example to EKS (Commercial)

Potential future backlog items:

EKS (GovCloud)
Big Bang example deploys to EKS (Commercial then GovCloud)
Software Factory package (a.k.a. DI2-ME) deploys to EKS (Commercial then GovCloud)

Blocked by #237

@team, looking for engineering feedback on this before it gets moved to Ready to Start

Is the outcome and acceptance criteria well understood?
Is it reasonable?
Can the backlog item be made smaller without arbitrarily slicing it into little valueless chunks?

Adding to the "DI2-ME" milestone, since it is needed for that work to truly be valuable, but this backlog item is also very valuable outside of the context of the "DI2-ME" package.

Blocked by #335

This shouldn't be closed by #329 actually since it needs an E2E test that deploys to EKS

@jeff-mccoy we need to reopen this since the description calls for an E2E test, unless we want to make a new issue for that.

Closing this one, updated AC to remove the E2E for this one as we've been working/testing on EKS for a couple of months now.


11.4 CFW line

why is dsiware are not in 11.4 CFW line?
I mean..

There is DSIWARE in 11.5, and not int 11.4

I think you can make mistake but..
like 1.1.0, people can think there is no CFW way in 11.4.

Currently there is no documented method on 11.4 other than via 11.5. The previous method was DSiWare which no longer works (latest version required).

Maybe instead of a blank cell you can fill it with "Upgrade 11.5, then proceed."

oh, I see. thx!


Ag grid

Added

new, beta Application grid ()

[Delivers #]

@MarkBaker Please set up your IDE as per docs, to put the author tags automatically.

👍 Since there are no concrete requirements for the grid I cannot say whether they have been met but everything works and sufficient tests have been made.


Break up snapshotter into separate binary

WIP
This creates two binaries:

changes-client
changes-snapshotter

It will remove the configuration to output snapshots from the client and adjust the reporting mechanisms to support the future snapshot changes described in dropbox/changes#29

We're sorry, but we're no longer updating this repo, and we're closing this PR without action. Of course you're welcome to fork it!


Code not matching tutorial

Hello, I've done extensive testing and found the following: on step 4, part 4 of Add a reticle the following code is on the tutorial:
async onSessionStarted() {
  // ...

  // Setup an XRReferenceSpace using the "local" coordinate system.
  this.localReferenceSpace = await **_session_**.requestReferenceSpace("local");

  // Add these lines:
  // Create another XRReferenceSpace that has the viewer as the origin.
  this.viewerSpace = await this.**_session_**.requestReferenceSpace("viewer");
  // Perform hit testing using the viewer as origin.
  this.hitTestSource = await this.**_session_**.requestHitTestSource({ space: this.viewerSpace });

  // ...
}

and it doesn't work, it throws the onNoXRDevice exception.
The functional code, found in step-04 folder is the following:
// Setup an XRReferenceSpace using the "local" coordinate system.
this.localReferenceSpace = await this.**_xrSession_**.requestReferenceSpace('local');

// Create another XRReferenceSpace that has the viewer as the origin.
this.viewerSpace = await this.**_xrSession_**.requestReferenceSpace('viewer');
// Perform hit testing using the viewer as origin.
this.hitTestSource = await this.**_xrSession_**.requestHitTestSource({ space: this.viewerSpace });

Thanks for the catch! I can't immediately update the website at the moment (colleagues are in a far-off timezone), but the patch is waiting for submission. Thanks for bringing this up!


How to work with FormAction?

Rasa Core version: 0.9.0a3
Rasa NLU version: 0.12.2
Python version: 3.6
Operating system (windows, osx, ...):
The bot got stuck after trigger sentence "I want to report a conflict of interest". Then no reply (no matter what I enter)
when I enter fallback sentence, then 10 times the fallback reply comes back.
Can you provide a simple comprehensive example on how to use FormAction (your partial examples seem to lack some key information or I'm simply too stupid to understand it), ideally complemented with an example on how to fill the slot with the "coi_gi_receivername" (!)
Thx in advance!
Content of domain file (if used & relevant):
intents:
 - greet
 - default
 - goodbye
 - thankyou
 - reportcoi
 - affirm
 - inform

entities:
 - username

slots:
  username:
    type: text
    initial_value: "human"
  slot_result:
    type: text
  requested_slot:
    type: unfeaturized

templates:
  utter_greet:
    - "hey there {username}!"    # variable will be filled by slot with the same name or by custom code
  utter_goodbye:
    - "goodbye {username}"       # multiple templates will allow the bot to randomly pick from them
  utter_default:
    - "I don't understand what you mean!"
  utter_affirm:
    - "affirmative"
  utter_thankyou:
    - "you're welcome"
  utter_entername:
    - "Good to know you, {username}"
  utter_reportcoi:
    - "Sure, let me guide you through the reporting process."
  utter_coi_gi_receivername:
    - "Who is the receiver of the gift/invitation?"
  utter_coi_gi_give_or_take:
  - text: "Do you give or receive the gift/invitation to {coi_gi_receivername}?"
    buttons:
    - title: "give"
      payload: "give"
    - title: "receive"
      payload: "receive"
  utter_coi_amount:
    - "How much is the gift/invitation worth (in EUR)?"
  utter_coi_gi_is_publicofficial:
    - "Is the receiver/giver a public official?"
  utter_coi_remarks:
    - "Please amend any further information necessary"

actions:
  - utter_greet
  - utter_default
  - utter_goodbye
  - utter_thankyou
  - utter_reportcoi
  - utter_affirm
  - utter_entername
  - utter_coi_gi_receivername
  - utter_coi_gi_give_or_take
  - utter_coi_gi_is_publicofficial
  - utter_coi_amount
  - utter_coi_remarks
  - test_rasa.ActionReportCoI

class ActionReportCoI(FormAction):

    RANDOMIZE = False

    @staticmethod
    def required_fields():
        return [
            EntityFormField("coi_gi_receivername", "coi_gi_receivername"),
            EntityFormField("coi_gi_give_or_take", "coi_gi_give_or_take"),
            EntityFormField("coi_gi_is_publicofficial", "coi_gi_is_publicofficial"),
            EntityFormField("coi_amount", "coi_amount")
        ]

    def name(self):
        return 'action_report_coi'

    def submit(self, dispatcher, tracker, domain):
        results = RestaurantAPI().search(
            tracker.get_slot("coi_gi_receivername"),
            tracker.get_slot("coi_gi_give_or_take"),
            tracker.get_slot("coi_gi_is_publicofficial"),
            tracker.get_slot("coi_amount"))
        return [SlotSet("search_results", results)]

## intent:greet
- good morning
- good evening
- hey
- howdy
- hey there
- hello
- hi
- dear sir

## intent:default
- bla bla
- do you want to marry me
- i hate you
- i want something else
- can you help me?
- do you love me?
- fuck you

## intent:goodbye
- goodbye
- good bye
- stop
- end
- farewell
- Bye bye
- have a good one

## intent:affirm
- great
- right, thank you
- correct
- great choice
- sounds really good
- yes
- yep
- yeah
- indeed
- that's right
- ok

## intent:reportcoi
- report conflict of interest
- i would like to report a conflict of interest
- i want to report a conflict
- how can i report a conflict of interest
- where can i report a conflict of interest
- can you support me in reporting a conflict of interest

## intent:intent_coi_gi_receivername
- the receiver's name is [Eric Wagner](coi_gi_receivername)
- the name is [Karin](coi_gi_receivername)
- my name is [Philipp Mayer](coi_gi_receivername)
- the receiver will be [Oliver Mueller](coi_gi_receivername)
- the receiver is [Constantine Breitenfeld](coi_gi_receivername)

## intent:thankyou
- thanks
- thank you
- thank you very much
- nice
- thx alot
- thx
- appreciated

## synonym:chinese
+ Chines
* Chinese

## synonym:vegetarian
- vegg
- veggie

## regex:zipcode
- [0-9]{5}

## regex:greet
- hey[^\s]*

## greet path
* greet
  - utter_greet

## goodbye path
* goodbye
  - utter_goodbye

## default path
* default
  - utter_default

## affirm path
* affirm
  - utter_affirm

## thankyou path
* thankyou
  - utter_thankyou

## report_coi path 1
* reportcoi
  - action_report_coi
  - slot{"requested_slot": "coi_gi_give_or_take"}
* inform("coi_gi_give_or_take": "give")
  - action_report_coi
  - slot{"requested_slot": "coi_gi_is_publicofficial"}
* inform{"coi_gi_is_publicofficial": "affirm"}
  - action_report_coi
  - slot{"requested_slot": "coi_amount"}
* inform{"coi_amount": "1000"}
  - action_report_coi
  - utter_goodbye

Guys ... any chance for feedback? Does this FormAction really work? Did anybody succeed in bringing that to life?

Does the report_coi path 1 not work either if you try to follow it with the trained bot?

I'm not sure, as I don't get any feedback from bot - prompt remains and I can enter further input(s) with same effect. This stops when I only hit the enter (i.e. no input) and then 10 times the utter_default appears. Then I can start over with "hi" again (which works, by the way). Only if I enter the report_coi_path 1, I have the above mentioned effects.

Can you run the bot with --debug flag set to increase the output? If you don't mind, please also add the comandline log from this session.

2018-05-05 20:32:10 INFO     rasa_core.policies.keras_policy  - Done fitting keras policy model
2018-05-05 20:32:10 INFO     rasa_core.agent  - Persisted model to '/home/eric/data/test_rasa/models/dialogue'
2018-05-05 20:32:10 INFO     rasa_nlu.components  - Added 'nlp_spacy' to component cache. Key 'nlp_spacy-en'.
Bot loaded. Type a message and press enter:
hi
2018-05-05 20:32:21 DEBUG    rasa_core.tracker_store  - Creating a new tracker for id 'default'.
2018-05-05 20:32:21 DEBUG    rasa_core.processor  - Received user message 'hi' with intent '{'name': 'greet', 'confidence': 0.5368560066640731}' and entities '[]'
2018-05-05 20:32:21 DEBUG    rasa_core.processor  - Logged UserUtterance - tracker now has 2 events
2018-05-05 20:32:21 DEBUG    rasa_core.processor  - Current slot values:
requested_slot: None
slot_result: None
username: human
2018-05-05 20:32:21 DEBUG    rasa_core.policies.memoization  - Current tracker state [
[('slot_username_0', 1)]
[('intent_greet', 1), ('slot_username_0', 1), ('prev_action_listen', 1)]]
2018-05-05 20:32:21 DEBUG    rasa_core.policies.memoization  - Used memorised next action '2'
2018-05-05 20:32:22 DEBUG    rasa_core.policies.ensemble  - Predicted next action 'utter_greet' with prob 1.00.
hey there human!
2018-05-05 20:32:22 DEBUG    rasa_core.processor  - Bot utterance 'BotUttered(text: hey there human!, data: null)'
2018-05-05 20:32:22 DEBUG    rasa_core.processor  - Action 'utter_greet' ended with events '[]'
2018-05-05 20:32:22 DEBUG    rasa_core.policies.memoization  - Current tracker state [
[('intent_greet', 1), ('slot_username_0', 1), ('prev_action_listen', 1)]
[('intent_greet', 1), ('slot_username_0', 1), ('prev_utter_greet', 1)]]
2018-05-05 20:32:22 DEBUG    rasa_core.policies.memoization  - Used memorised next action '0'
2018-05-05 20:32:22 DEBUG    rasa_core.policies.ensemble  - Predicted next action 'action_listen' with prob 1.00.
2018-05-05 20:32:22 DEBUG    rasa_core.processor  - Action 'action_listen' ended with events '[]'
2018-05-05 20:32:22 DEBUG    rasa_core.processor  - Current topic: None
I want to report a conflict of interest
2018-05-05 20:32:55 DEBUG    rasa_core.tracker_store  - Recreating tracker for id 'default'
2018-05-05 20:32:56 DEBUG    rasa_core.processor  - Received user message 'I want to report a conflict of interest' with intent '{'name': 'reportcoi', 'confidence': 0.517626934387094}' and entities '[]'
2018-05-05 20:32:56 DEBUG    rasa_core.processor  - Logged UserUtterance - tracker now has 6 events
2018-05-05 20:32:56 DEBUG    rasa_core.processor  - Current slot values:
requested_slot: None
slot_result: None
username: human
2018-05-05 20:32:56 DEBUG    rasa_core.policies.memoization  - Current tracker state [
[('intent_greet', 1), ('slot_username_0', 1), ('prev_utter_greet', 1)]
[('intent_reportcoi', 1), ('slot_username_0', 1), ('prev_action_listen', 1)]]
2018-05-05 20:32:56 DEBUG    rasa_core.policies.memoization  - Used memorised next action '14'
2018-05-05 20:32:56 DEBUG    rasa_core.policies.ensemble  - Predicted next action 'action_report_coi' with prob 1.00.
2018-05-05 20:32:56 ERROR    rasa_core.processor  - Encountered an exception while running action 'action_report_coi'. Bot will continue, but the actions events are lost. Make sure to fix the exception in your custom code.
2018-05-05 20:32:56 ERROR    rasa_core.processor  - name 'EntityFormField' is not defined
Traceback (most recent call last):
File "/home/eric/.local/lib/python3.6/site-packages/rasa_core/processor.py", line 301, in _run_action
events = action.run(dispatcher, tracker, self.domain)
File "/home/eric/.local/lib/python3.6/site-packages/rasa_core/actions/forms.py", line 118, in run
events = self.get_requested_slot(tracker) + self.get_other_slots(tracker)
File "/home/eric/.local/lib/python3.6/site-packages/rasa_core/actions/forms.py", line 100, in get_requested_slot
required = self.required_fields()
File "/home/eric/data/test_rasa/test_rasa.py", line 60, in required_fields
EntityFormField("coi_gi_receivername", "coi_gi_receivername"),
NameError: name 'EntityFormField' is not defined
2018-05-05 20:32:56 DEBUG    rasa_core.processor  - Action 'action_report_coi' ended with events '[]'
2018-05-05 20:32:56 DEBUG    rasa_core.policies.memoization  - Current tracker state [
[('intent_reportcoi', 1), ('slot_username_0', 1), ('prev_action_listen', 1)]
[('intent_reportcoi', 1), ('slot_username_0', 1), ('prev_action_report_coi', 1)]]
2018-05-05 20:32:56 DEBUG    rasa_core.policies.memoization  - Used memorised next action '0'
2018-05-05 20:32:56 DEBUG    rasa_core.policies.ensemble  - Predicted next action 'action_listen' with prob 1.00.
2018-05-05 20:32:56 DEBUG    rasa_core.processor  - Action 'action_listen' ended with events '[]'
2018-05-05 20:32:56 DEBUG    rasa_core.processor  - Current topic: None

Guys ... any help would be appreciated.

So as for 1:
You have to set the payload to "/give" - this directly passes on the intent.
How are you sending this utterance? Is it with dispatcher.utter_button_template? If it's with dispatcher.utter_button_message, you have to .format the string with the slot value. With the template it should be set automatically though.

I'd say you have to add stories for the fallback action then.

I have to be honest - you've lost me ...
ad 1a) there's no  "give". I simply want to fill the slot "coi_gi_give_or_take" with either "give" or "receive". Ideally, I would like to do that with a button, but I don't know how that works with FormAction?
ad 1b) can you help me on that as well?
ad 2)

do you really mean separate STORIES for fallback action? If yes, HOW do I do that ??
I thought separate intent examples in nlu.yml are sufficient for that?  (see attached .zip file above with complete py and yml and md file-set)

ad 3a+b) can you help me on that as well?

https://github.com/RasaHQ/rasa_core/pull/458 should fix the slot issue

for 1a) if you want a button to pass entities you can also do that, with e.g. /coi_gi_give_or_take{"give_take":"give"} as a payload (assuming you have an entityformfield which sets entity give_take to the slot you want to set.

easiest is just to grab master from github

I'm too stupid ...
sudo -H python3 setup.py install --user
installs everything in (correct) ~/.local/lib/python3.6/site-packages/
EXCEPT rasa_core, which is installed under /root/.local/lib/python3.6/site-packages/ (!?)
see:
root@fractal:/root/.local/lib/python3.6/site-packages# ls -l
total 300
-rw-r--r-- 1 root root    220 Mai 14 22:20 easy-install.pth
-rw-r--r-- 1 root root 295766 Mai 14 22:20 rasa_core-0.9.0a5-py3.6.egg
-rw-r--r-- 1 root root     46 Mai 14 22:20 setuptools.pth
root@fractal:/root/.local/lib/python3.6/site-packages# cat easy-install.pth
./rasa_core-0.9.0a5-py3.6.egg
/usr/local/lib/python3.6/dist-packages/colorhash-1.0.2-py3.6.egg
/home/eric/.local/lib/python3.6/site-packages
/usr/local/lib/python3.6/dist-packages/tensorflow-1.8.0-py3.6-linux-x86_64.egg
root@fractal:/root/.local/lib/python3.6/site-packages# cat setuptools.pth
/home/eric/.local/lib/python3.6/site-packages

You need to add the name requested_slot to your slots in the domain file

Thank you! The error disappeared. Wish the documentation was little clear. Did not realize the slot name it self was, "requested_slot'

+1 we should raise an error if a domain contains a FormAction but there is no requested_slot in the domain

another suggestion / request :
Is it possible to pull the VALIDATE function from forms.py and be referenced / called from custom actions?
Though this validations implementation is nice, don't like some custom code creeping in core packages.

I made some progress. I am able to get the validations properly. It  does go to the submit function after the last action_fillForm. But, story does not go to "utter_goodbye". Is this a training issue or something else?
I am yet to locate a proper example to see, if I am at least on the right track.
My story ends like this...
....
....
....

action_fillForm
slot { "requested_slot": "addr"}

nameNo {"addr":"103, 1xxxxx"}

action_check
action_write
action_fillForm
slot { "requested_slot": "idProof"}

nameNo {"idProof":"xxxxx"}

action_fillForm
utter_goodbye
action_restart

@ewagner70 , did you manage to get this working. Been doing several attempts and also finding very similar to your observations.

@Egalite123 yes, I've downloaded the most recent version from github (via pip install -U git+https://github.com/rasahq/rasa_core.git --user ) what I found out is that the training set is sometimes quite tricky to compose (if you are referring to your last comment "... but story does not go to 'utter_goodbye' ...") especially if you have lengthy similar paths in various stories and it can sometimes lead to a shitty model, which does not really work (then simply re-train again and re-use that model accordingly)
Don't know if my answer really helps - maybe you can be more specific which of the aboves issues you're referring to.

@Egalite123  could you please share your project here.. I am facing the same issue :(

@Egalite123  I am having the same issue as well. It goes into submit form but does not go into the next action.

could one of you share your stories file/form action so we can try and reproduce this? In general it might help to try interactive learning for this. It might be that you've forgotten a slot set event somewhere in your stories

Hi @akelad , here is my stories file.
When I run the bot, I do get "success". But, does not goto "utter_otp".
Please let me know, if you need anything more.
FundsTransferConfirm

transferFunds

action_fillForm
slot{"requested_slot": "fromAccount"}

dataInput

action_fillForm
slot{"fromAccount": "A0001"}
slot{"requested_slot": "toAccont"}

dataInput

action_fillForm
slot{"toAccount": "B0002"}
slot{"requested_slot": "currency"}

dataInput

action_fillForm
slot{"currency": "USD"}
slot{"requested_slot": "amt"}

dataInput

action_fillForm
slot{"amt": "1000"}
slot{"searchResults": "success"}
utter_otp

otp

utter_confirmTransfer
action_restart

actions.py -->
class trustCheck(object):
def search(self, *args):
    print ("= = = = inside api  = = = = =")
    return

class ActionFillForm(FormAction):
RANDOMIZE = False
print ("Surya...")
@staticmethod
def required_fields():
    return [
        FreeTextFormField("fromAccount"),
        FreeTextFormField("toAccount"),
        FreeTextFormField("currency"),
        FreeTextFormField("amt")
    ]

def name(self):
    return ACTION_FILLFORM

def submit(self, dispatcher, tracker, domain):

    results = trustCheck().search(
        tracker.get_slot("fromAccount"),
        tracker.get_slot("toAccount"),
        tracker.get_slot("currency"),
        tracker.get_slot("amt"))

    template = "utter_thanks"
    dispatcher.utter_template(template)

    results = "success"
    print ("==== in submit =====")
    print ("Results...... ", results)
    return [SlotSet("searchResults",results)]

In continuation of the above, while testing last night noticed this -
a. While I just start with intent (in this example - just saying "transfer funds" with out entities) and if I let FormAction control the dialogue with the user (filling all slots sequentially), it is behaving normally and goes onto "utter_otp"
b. If I start with something like - "transfer 100 from my account AC1234", bot identifies that , two slots amt and fromAccount are filled and asks only currency and toAccount. In this case, though the FormAction has all the entities, it does out come of the loop and goes to "utter_otp".
May be this is a clue. But, was still elusive to me.

Ah ok, so the issue here is you need to have a story for that case too, when the user provides some information at the start already. Otherwise Core may not have the correct prediction

Not sure, I follow you. But, when the user supplies other info (e.g., amount ), I see that the required slot is already is filled and FormAction skips the question. That is an expected behaviour, right?

Yes it is, but you also need to add the stories for those cases to your training data otherwise future predictions of core might be wrong.

If i have fields in the FormAction that are more or less identical in its range and type of values, how do I train my NLU to recognise the correct intent for it. Eg inform{"latitude": "10"} vs inform{"longitude": "10"}. Is there any way for me to automatically force whatever input the user has to a particular intent?

could one of you share your stories file/form action so we can try and reproduce this? In general it might help to try interactive learning for this. It might be that you've forgotten a slot set event somewhere in your stories

Strangely, in interactive learning, it seems to be executing it correctly.

what do you mean? Those examples have the same intent, are you talking about entities?

I have hit up on a problem. I have two different  story snippets (or, use cases). One FormAction to make a FundsTransfer and another one to collect user information for call back. I have named them two different action names and also two different intents to collect user inputs (one like illustrated above - "dataInput" and  other formAction has "inform" intents - just to avoid confusion.
Both the flows collect numerical data (like amount in the example above and other form action also collects customer account number).
Some times, while running, bot picks up the other intent (instead of "dataInput" picks the intent as "inform" and jumps into the middle of other story.
How can I avoid this situation?

@Egalite123 Can you post this kind of question in our forum please? We'll answer it there.
Also this issue thread has been going on for a while and they're all mostly questions rather than issues/bugs, so I'll close this for now.


Gradio patch should be dimension agnostic

Error message from https://github.com/John-WL/sd-webui-inpaint-difference/issues/4#issuecomment-1996211262:
Traceback (most recent call last):
  File "F:\stablediffusion\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "F:\stablediffusion\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1434, in process_api
    data = self.postprocess_data(fn_index, result["prediction"], state)
  File "F:\stablediffusion\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1335, in postprocess_data
    prediction_value = block.postprocess(prediction_value)
  File "F:\stablediffusion\stable-diffusion-webui\venv\lib\site-packages\gradio\components\image.py", line 316, in postprocess
    return processing_utils.encode_pil_to_base64(y)
  File "F:\stablediffusion\stable-diffusion-webui-1-8-0-rc\extensions\sd-webui-inpaint-difference\lib_inpaint_difference\gradio_hijack
s.py", line 13, in encode_pil_to_base64_new
    image_arr = np.asarray(pil_image)[:, :, ::-1]
IndexError: too many indices for array: array is 2-dimensional, but 3 were indexed

The issue seems to be apparent when inpaint-diff and the mosaic extension are used at the same time, though I believe the problem is more general than this conflict, and it should be fixed asap.

Should be fixed now.


Add Travis

Thanks @kuraga
I will merge this after merging the PR related to extending Freezer.

Thanks!
P.S. I would be very pleased if you see open PRs and issues and than push new version :smile:


Improve User-Agent

You need to complete

[ ] unit tests and/or feature tests
[ ] documentation is changed or added

Codecov Report

Merging #15 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master    #15   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           2      2
  Lines         165    170    +5
=====================================
+ Hits          165    170    +5

Impacted Files
Coverage Δ

lib/aliyunsdkcore/roa_client.rb
100% <100%> (ø)

lib/aliyunsdkcore/rpc_client.rb
100% <100%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fe12c9c...a90402d. Read the comment docs.


Implement pretty error in cli

Adresses #143

Codecov Report

Merging #144 into master will decrease coverage by 0.23%.
The diff coverage is 17.39%.

@@            Coverage Diff             @@
##           master     #144      +/-   ##
==========================================
- Coverage   82.58%   82.35%   -0.24%
==========================================
  Files         130      130
  Lines        5776     5797      +21
==========================================
+ Hits         4770     4774       +4
- Misses        685      700      +15
- Partials      321      323       +2

Impacted Files
Coverage Δ

pkg/prettyerr/text_formatter.go
18.51% <6.25%> (-12.26%)
:arrow_down:

pkg/prettyerr/error.go
65.00% <42.85%> (-11.93%)
:arrow_down:

I just noticed this PR was still a draft, I have removed the reviewers for now. Feel free to update the PR when it is ready to be reviewed

@jeroenrinzema  PR is ready for review.


Why is there no content on how to configure things for AD FS

So looking at the content for Configure AD FS for Hybrid deployments it seems to be lacking a whole lot of configuration steps when compared to the on-premises certificate trust configuration.
It actually calls out the permissions required on the certificate templates, and how to setup the Enrollment Agent certificate and how to validate that it got the RA certificate.
Also the Hybrid Cert Trust talks about using a Service Account for AD FS rather than gMSA in the other one.  Is there a reason for this?
If the reason for not really covering AD FS configuration is because it is expected that the AD FS is already being used for Azure Federation setup then it would be good to call this out, as well as pointing to content on how to do this!  You should not assume that the tenant is already federated.  I do not see anywhere it states that the Azure tenant needs to be federated for this to work.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 24d7753d-30e1-db13-429a-c0b4eaf2c9c5
Version Independent ID: 5be668ea-fc9d-0fb5-585d-4f5d5575b416
Content: Configure Active Directory Federation Services in a hybrid certificate trust model
Content Source: windows/security/identity-protection/hello-for-business/hello-hybrid-cert-whfb-settings-adfs.md
Product: windows-client
Technology: itpro-security
GitHub Login: @paolomatarazzo
Microsoft Alias: paoloma

Hello @RobG-MSFT, this article describes how to configure AD FS specifically for the WHFB deployment model. It's not intended to cover how to deploy AD FS from scratch or how to configure device registration. Those are considered prerequisites. Since the content is already described in other articles, it's called out in the prerequisites section.

Regarding the observation about gMSA, I didn't get a chance to test and refresh that part of documentation to use gMSA. If you already have the process outlined, please feel free to make a contribution to the deployment guide.


Empty tags in jsx mode are coloured as if there was a syntax error

Hi,
Empty tags are coloured as if they had a syntax error on jsx mode but they seem to be a valid shorthand for React.Fragment

Can you reproduce this on http://codemirror.net/mode/jsx/?

Oh!  it seems to work there, it might be my configuration, the imported files or a new version maybe? I see I got mode:'jsx I'll try text/jsx and see.
I usually think the configuration and needed files could be detailed on example pages on the website as I usually go and inspect the page code to see what is needed (maybe I'm missing something and this is explained somewhere)

text/jsx does not change anything. I'm importing these files:
codemirror.min.css,  codemirror.min.js, codemirror/jsx.js, javascript.js, selection-pointer.js, xml.js and css.js
I got the files from cdnjs.com


feature request: add config option to not show anything while in idle mode

Discussed in https://github.com/skuethe/MMM-Spotify/discussions/140

Originally posted by CharlesMougenot February 10, 2021
MMM-Spotify is a great module.
However I liked it better without spotify logo: if spotify is unused the logo uses space on the mirror for nothing. Maybe an option to hidden it would be nice.

In addition: Free the space the module reserves when it is hidden! Please also for minibar mode (I really like this mode)


add rss

#2
Haven't been able to properly sanitize the posting text. Everything else should be all right

Did you try to put description and title into a <![CDATA[...]]> tag like this?
  <title><![CDATA[<%= sanitized_inline_markdown posting.title %>]]></title>
  <description><![CDATA[<%= sanitized_markdown posting.text %>]]></description>

Will try that

Thanks! :+1:


pngout not found, linux

Image_optim v 0.11.1 causes:
rake aborted!
pngout not found
on a debian machine, all packages (advancecomp gifsicle jhead jpegoptim libjpeg-progs optipng pngcrush) are installed.

pngout worker: pngout not found; please provide proper binary or disable this worker (--no-pngout argument or :pngout => false through options) on ------rake db:migrate-----
i am getting this error , how can i disable it
@toy

@avinashdvv You either need to make pngout available in path or disable the worker in yml config or through options to image_optim initialiser. Which version are you using?


Make foods menu UI better

I want a UI made using bootstrap cards, three cards in one line, each card representing one food with order and add to card button.

each card should have an image of food too, if there.

Fixed in #10


[FEATURE] Add confirm password field in sign up page

I want to add confirm password field in sign up page and let users to set up strong passwords only

@MitulSonagara please assign this issue to me

@meghanakn473
You can start working...


Better version negotiation in the peer connection handshake

We should check zcashd to ensure we are constructing version handshake messages with fields that have the same meaning as zcashd.

We never actually check and agree on a version so uh, we should do that.

Added the research label since there's some looking into how zcashd behaves involved.

This is 'done' by #295 , there may be an edge case if version changes happen in the future.
https://github.com/ZcashFoundation/zebra/pull/295#discussion_r388549034


Enable the back arrow in requests coming from Onboarding (Import, Create)

Might as well come from LOGIN in case of CREATE or vice versa. So a more generally applicable name was chosen.
Feel free to recommend a renaming.

Hm, you are right. Ok, lets go with as it is now.


Add test for presets

add extras installation
add test for TabularNLPAutoML
add pickling test for all presets

the problem with numpy and python 3.10 still exists:
https://github.com/numpy/numpy/issues/22520


Nearly Ready: Use new float/double apis

Rework client to use new float/double api for cells. Swap "Numeric" column type for "Float".

Only question on this is whether we should allow numeric strings in the case when a column description is defined.
Example
For a row that has 1 cell, SQL type Float.

Create Storevalue command
withColumns = [TsColumnType.Float]
withRows = [ [ '4.5' ] ]

Storevalue command would then put '4.5' in the numericValue register of the cell only in this case.
Without the columns definition, it would store the string value '4.5' to the string cell register.
See https://github.com/basho/riak-nodejs-client/pull/105/files#diff-f26b0b0dee3b0c66618f90c8dfddf3bbR147.

:ship: :it:


Implement endpoint to store the builder onboarding seen status

Backend?

Yes, I was going to ask you to whom could I assign the ticket?

hi @javitonino! :)

Give him details in the description please :)

I feel we should extend this functionality for other purposes, not only for builder thing, so take a moment for the implementation @javitonino.

So current idea for implementing this is as follows:

Create a new model, UserNotification which consists basically of a 1:1 reference to User and a JSON field to store notifications.
Create new API to store notifications. This will be a PUT of the complete notifications JSON. Does it makes more sense for this to replace the data, or to merge it? Merging allows to include just the data that changed (the notification that was marked as do not show again), while replacing allows to delete keys.
Include the notifications JSON in the builder .erb so it is available to the JS. Only needed in the builder view, right?

To be able to support different use cases (onboarding vs notification messages) without having to always set and retrieve the whole notification object, the plan is to separate the notifications in different categories, such as builder or messages. That way, the backend can pass only the necessary notifications for each view and the api calls only apply to the relevant category. The API's will have category as part of the URL.
PUT /api/v1/users/:id/notifications/builder
{
  "onboarding": {
    "import": {"show": false }
  }
}

---

PUT /api/v1/users/:id/notifications/messages
{
  "welcome_dashboard": { "read": true }
}

The database will store:
"builder": {
  "onboarding": {
    "import": {"show": false }
  }
},
"messages": {
  "welcome_dashboard": { "read": true }
}

When displaying a page, all of them will include messages (when that is implemented), but only the builder view will include builder, as that is only relevant there.
I think that covers everything that was needed. Thoughts?

Oh, I like the idea of categorize the notifications, so :+1:

Could this thing cover your expectations @javierarce?

I think this is great.
Regarding your first question: for this particular case, merge and replace are both ok. I don't have any preference for the broad use of this feature right now, so pick whichever you want.
Regarding your second question, yes, currently we just need this in the builder view.

Merged!
You should be getting the notifications in a builderNotifications variable. The API to update them is:
PUT (/u/user)/api/v3/notifications/builder
{ "notifications": {[...]} }

Let's try to agree on the format of the data that's stored inside. Although the backend won't be doing anything with it now, it would be good to have an extensible an easily parseable format in case we need to start doing so in the future (thinking of statistics gathering). I propose something like this, but you know the needs better than anyone:
{
  "onboarding": {
    "import": {"show": false },
    "add_layer": {"show": false }
  },
  "analyses": {
    "georeference": { "show": false },
    "point-in-polygon": {"show" : false }
  }
}

We have a categorization to easily separate onboarding from analyses, and inside each key we store if we want to show the message or not. It is inside a dictionary to allow expansion, which I thunk could be useful (but maybe I'm overengineering this).

👍


Use default environment for MACOSX_DEPLOYMENT_TARGET

I have defined MACOSX_DEPLOYMENT_TARGET myself. pyenv should read my
settings as the default value.
I need to build flat 10.6-intel pythons for compatible wheels.
Related article: http://lepture.com/en/2014/python-on-a-hard-wheel

Sorry for delay. I added a fix and some tests for it in 22ecefd5729d1634bd1c498a43aa30ad87dc351c :disappointed_relieved:


Change license to be compatible with MIT requirements

Mistakes were made. Some files from other depots were MIT, probably switch to MIT.

@rntz @dwiel @fidgetingbits
I'm going to go ahead and switch this to the MIT license to be properly compatible with some of Ryan's scripts. Let me know if you have any concerns; it doesn't really have much of a practical impact.


title: Add import and export subcommand for kosmosctl

Description: Add import and export subcommand for kosmosctl

What type of PR is this?

feat: new feature

What does this PR do?
##export
#if svc elasticsearch-logging-data in master k8s, create ServiceExport elasticsearch-logging-data in master k8s
kosmosctl export service elasticsearch-logging-data -n foo
#if svc elasticsearch-logging-data-1 in knode knode-worker1, create ServiceExport elasticsearch-logging-data in master k8s
kosmosctl export service elasticsearch-logging-data-1 -n foo --kubeconfig /tmp/master --knode knode-worker1
##import
#if svc elasticsearch-logging-data in master k8s, create ServiceImport elasticsearch-logging-data in master k8s
kosmosctl import service elasticsearch-logging-data -n foo
#if svc elasticsearch-logging-data-1 in knode knode-worker1, create ServiceImport elasticsearch-logging-data in knode knode-worker1
kosmosctl import service elasticsearch-logging-data-1 -n foo --kubeconfig /tmp/master --knode knode-worker1
#if svc elasticsearch-logging-data-1 in knode knode-worker1, create ServiceImport elasticsearch-logging-data-1 in knode knode-worker2
kosmosctl import service elasticsearch-logging-data-1 -n foo --kubeconfig /tmp/master --knode knode-worker1 --to-knode knode-worker2
Which issue(s) does this PR fix?
Fixes #100
Screenshots (if applicable):

Additional Comments:
If svc has exported or imported, now we don't update it

@GreatLazyMan please check the golint ci problem

@GreatLazyMan please check the golint ci problem

Commited new code, fix golint ci problem

/lgtm
/cc @ONE7live

/lgtm
/cc @wuyingjun-lucky

/cc @xyz2277 @wuyingjun-lucky
please help check

/lgtm
/approve


statsmodels 0.9.0 appears to break existing tests

Should investigate updating tests, pinning statsmodels, or some other unconsidered option(s).

Failing tests on busywork.

I think just update the tests. It looks like statsmodels may have made the row names in the summary results more clear, so we should just propagate that.

Sounds good - we should probably update the conda recipe to pin the minimum version of statsmodels.


Improve "CStrings" handling

During #2753 I made the "mistake" of using String() instead of CString() and I was wondering why the returned string had a bunch of trailing null characters. It took me a while to understand I should've used CString() instead. I think there is not any reason to provide CString() as users will need to understand when to use it, hence this PR removes it, hopefully making users' life easier.
Testing
I used the following code to test it:
// Copyright 2024 The Inspektor Gadget authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package main

import (
	"context"
	"fmt"
	"os"

	"github.com/inspektor-gadget/inspektor-gadget/pkg/datasource"
	gadgetcontext "github.com/inspektor-gadget/inspektor-gadget/pkg/gadget-context"
	"github.com/inspektor-gadget/inspektor-gadget/pkg/operators"
	_ "github.com/inspektor-gadget/inspektor-gadget/pkg/operators/ebpf"
	ocihandler "github.com/inspektor-gadget/inspektor-gadget/pkg/operators/oci-handler"
	"github.com/inspektor-gadget/inspektor-gadget/pkg/operators/simple"
	"github.com/inspektor-gadget/inspektor-gadget/pkg/runtime/local"
)

func do() error {
	const opPriority = 50000
	myOperator := simple.New("myHandler", simple.OnInit(func(gadgetCtx operators.GadgetContext) error {
		for _, d := range gadgetCtx.GetDataSources() {
			// Access some specific fields
			commF := d.GetField("comm")

			d.Subscribe(func(source datasource.DataSource, data datasource.Data) error {
				comm := commF.String(data)
				fmt.Printf("command %s, len(comm): %d\n", comm, len(comm))

				if comm == "cat" {
					fmt.Printf("cat was called!!!\n")
				}
				return nil
			}, opPriority)
		}
		return nil
	}))

	gadgetCtx := gadgetcontext.New(
		context.Background(),
		"ghcr.io/inspektor-gadget/gadget/trace_open:latest",
		gadgetcontext.WithDataOperators(ocihandler.OciHandler, myOperator),
	)

	runtime := local.New()
	if err := runtime.Init(nil); err != nil {
		return fmt.Errorf("runtime init: %w", err)
	}

	if err := runtime.RunGadget(gadgetCtx, nil, nil); err != nil {
		return fmt.Errorf("running gadget: %w", err)
	}

	return nil
}

func main() {
	if err := do(); err != nil {
		fmt.Printf("Error running application: %s\n", err)
		os.Exit(1)
	}
}

Notice that is calls String() instead of CString(), I did this mistake while working on #2753.
Before this PR
The cat was called.. is never printed and the length of the string is always 16.
...
command wdavdaemon, len(comm): 16
command wdavdaemon, len(comm): 16
command cat, len(comm): 16
command wdavdaemon, len(comm): 16
command cat, len(comm): 16
command wdavdaemon, len(comm): 16
command cat, len(comm): 16
command cat, len(comm): 16
command wdavdaemon, len(comm): 16
command wdavdaemon, len(comm): 16
command wdavdaemon, len(comm): 16
command wdavdaemon, len(comm): 16
...

After this PR
The cat was called.. message is printed and the length of the string is correct.
...
command wdavdaemon, len(comm): 10
command wdavdaemon, len(comm): 10
command wdavdaemon, len(comm): 10
command wdavdaemon, len(comm): 10
command wdavdaemon, len(comm): 10
command wdavdaemon, len(comm): 10
command cat, len(comm): 3
cat was called!!!
command cat, len(comm): 3
cat was called!!!
command cat, len(comm): 3
cat was called!!!
command cat, len(comm): 3
cat was called!!!
command wdavdaemon, len(comm): 10
command wdavdaemon, len(comm): 10
command wdavdaemon, len(comm): 10
command wdavdaemon, len(comm): 10
command wdavdaemon, len(comm): 10
....

Not sure about this one - null-terminated strings is especially a C thing, whereas null-chars in Go strings are totally valid. IMHO we should support both and default to CString for anything coming from eBPF only. We also have to check for example the args member of trace_exec - afair - splits the arguments using null-chars.

Not sure about this one - null-terminated strings is especially a C thing, whereas null-chars in Go strings are totally valid. IMHO we should support both and default to CString for anything coming from eBPF only.

I don't see how an user of our API will be interested in the String() method returning a string with null characters on it. Do you think there is any use case for it? In the end, it's always possible to use Get() (or a new Bytes() method) to access the raw bytes of the field.

IMHO we should support both and default to CString for anything coming from eBPF only.

Can you extend a bite more on what you exactly suggest in this case?

We also have to check for example the args member of trace_exec - afair - splits the arguments using null-chars.

I see this more like an array of bytes than a string. We can handle it with the Get()` method as described above.

Not sure about this one - null-terminated strings is especially a C thing, whereas null-chars in Go strings are totally valid. IMHO we should support both and default to CString for anything coming from eBPF only.

I don't see how an user of our API will be interested in the String() method returning a string with null characters on it. Do you think there is any use case for it? In the end, it's always possible to use Get() (or a new Bytes() method) to access the raw bytes of the field.

I think the difference between String() and Bytes() would be text encoding mainly. A string as handled in Go (and our API) would be interpreted as utf8 (where 0-chars are valid), while the raw buffer (using Get()/Bytes()) is just binary content. I also think it's important to differentiate (and we are not 100% clean on that right now) between strings and binary content as it should affect the representation in the frontend (strings should just be printed, while binary data might be printed as hex/base64) automatically (e.g. no necessity of doing it using annotations or such).

IMHO we should support both and default to CString for anything coming from eBPF only.

Can you extend a bite more on what you exactly suggest in this case?

The eBPF operator should - if known - report/tag string fields as CString if they are null-terminated by default. Right now it's a bit of a guessing game between binary content and strings which we should improve upon by maybe introducing new types. Most gadgets use strings, but if we go ahead and for example forward packet data, it should be specified as binary (or: not as string), so that whenever printed, it could be handled in a specific way from the frontend.

We also have to check for example the args member of trace_exec - afair - splits the arguments using null-chars.

I see this more like an array of bytes than a string. We can handle it with the Get()` method as described above.

I'd be fine with that - but if you want to output for example those entries separated as strings in JSON, it would need to get a dedicated type (or at least annotation) to be able to properly do that.

I also think it's important to differentiate (and we are not 100% clean on that right now) between strings and binary content as it should affect the representation in the frontend (strings should just be printed, while binary data might be printed as hex/base64) automatically (e.g. no necessity of doing it using annotations or such).

The eBPF operator should - if known - report/tag string fields as CString if they are null-terminated by default. Right now it's a bit of a guessing game between binary content and strings which we should improve upon by maybe introducing new types. Most gadgets use strings, but if we go ahead and for example forward packet data, it should be specified as binary (or: not as string), so that whenever printed, it could be handled in a specific way from the frontend.

This makes sense to me. I think the common way to define a string in C is using char [], so I'd say we can treat this as a string, other things like uint8_t [], etc. should be handled as a byte array.

I'd be fine with that - but if you want to output for example those entries separated as strings in JSON, it would need to get a dedicated type (or at least annotation) to be able to properly do that.

IMHO the args field in the trace_exec gadget is something we need to handle with wasm, I think it's too specific and I wouldn't introduce logic for it on Inspektor Gadget.
I think we kind of agree on this discussion, but I'm not sure how to move it forward, I'd propose:

Improve the logic to only recognize char [] as c-strings. Other combinations will be considered an array of bytes.
Improve the json marshaller (and the cli operator?) to encode []byte as hex
Provide a Bytes[] (and PutBytes) method to keep it uniform with other Uint/Int/String fields.
Drop the CString method, add logic to return a "c-string" on String() if the thing is coming from ebpf.

Thoughts on that?

I also think it's important to differentiate (and we are not 100% clean on that right now) between strings and binary content as it should affect the representation in the frontend (strings should just be printed, while binary data might be printed as hex/base64) automatically (e.g. no necessity of doing it using annotations or such).

The eBPF operator should - if known - report/tag string fields as CString if they are null-terminated by default. Right now it's a bit of a guessing game between binary content and strings which we should improve upon by maybe introducing new types. Most gadgets use strings, but if we go ahead and for example forward packet data, it should be specified as binary (or: not as string), so that whenever printed, it could be handled in a specific way from the frontend.

This makes sense to me. I think the common way to define a string in C is using char [], so I'd say we can treat this as a string, other things like uint8_t [], etc. should be handled as a byte array.

That sounds like a good way to approach it to me.

I'd be fine with that - but if you want to output for example those entries separated as strings in JSON, it would need to get a dedicated type (or at least annotation) to be able to properly do that.

IMHO the args field in the trace_exec gadget is something we need to handle with wasm, I think it's too specific and I wouldn't introduce logic for it on Inspektor Gadget.

Tbh, I think we should add that to the core. I can imagine there are many more situations where you need to add a dynamically sized list of things from eBPF (even if for now it'd only be strings). It might be tricky (or even not worth it) to add helpers right away that allow manipulating those, but if at least the exporters (columns, JSON mainly) could be made aware, it would already help.

I think we kind of agree on this discussion, but I'm not sure how to move it forward, I'd propose:

Improve the logic to only recognize char [] as c-strings. Other combinations will be considered an array of bytes.

Sounds good.

Improve the json marshaller (and the cli operator?) to encode []byte as hex

JSON: as hex makes sense
CLI: maybe add an option to choose from:

printable chars only (replace non-printables with dots, it's basically how many hex editors show the binary content)
as hex

Reason for introducing the former is that many times it really helps seeing at least the alphanumeric values directly instead of going through an external decode process (imagine raw dns packet contents, for example.) Could be added later on, but I'd prefer that to be the default for the CLI output.

Provide a Bytes[] (and PutBytes) method to keep it uniform with other Uint/Int/String fields.

👍

Drop the CString method, add logic to return a "c-string" on String() if the thing is coming from ebpf.

👍

Thoughts on that?

🚀


Chess / board-game flip button no longer works.

Probably caused by the recent changes to eye rotation that allows a players view to be misaligned from the grid?

Couldn't reproduce with current master in DEBUG configuration on Windows 11. Flip button works properly.

Works fine for me as well.


Relationship routes are created without adding them in the routes.rb

I've defined the following Resource:
class CommentResource < BaseResource
      has_one :activity
      has_one :user
end

 class ActivityResource < BaseResource
      has_many :comments
end

jsonapi_resources :activities, only: %w(index show)
jsonapi_resources :comments, only: %w(create destroy index show)

And checking with rake routes I see that it also created all the relationship/related routes for both Activity and Comment even if I haven't added jsonapi_links / jsonapi_related_resources in the routes file

By default all the nested routes are created. If you want control of the nested routes that are created you should use a block for jsonapi_resources. See https://github.com/cerebris/jsonapi-resources#nested-routes

I feel like a noob, even if I read the whole readme twice :sob:
I've noticed that even if the relationship routes are not created, the json response still includes them:
{
  "data": {
    "id": "908005740",
    "type": "comments",
    "links": {
      "self": "http://example.org/api/v2/comments/908005740"
    },
    "attributes": {
      "body": "You rock!",
      "created_at": "2015-08-06T15:57:33.401Z"
    },
    "relationships": {
      "activity": {
        "links": {
          "self": "http://example.org/api/v2/comments/908005740/relationships/activity",
          "related": "http://example.org/api/v2/comments/908005740/activity"
        }
      },
      "user": {
        "links": {
          "self": "http://example.org/api/v2/comments/908005740/relationships/user",
          "related": "http://example.org/api/v2/comments/908005740/user"
        }
      }
    }
  }
}

And this causes to have broken links on the relationships /api/v2/comments/908005740/relationships/activity and /api/v2/comments/908005740/activity are broken

@xdmx Yes, the routes are not tied to the serialization and links generation code. That's an oversight we should get fixed. I'll see if I can come up with a decent solution.

@lgebhardt how about defaulting to flat uris to the relationship resources instead of the nested ones? this would have less of a chance of pointing to inexistent routes.
of course, I'm plugging my own need here - I'm actually about to override the relationship uri building because I prefer the api to be as flat as possible. Less potential headaches, imo

btw, is it possible to generate these default relationship routes if one isn't using the jsonapi_resources helper? I've been including ActsAsResourceController into my controllers to be able to do things like declaring a before_action before loading JR:
class ResourceController < ApplicationController
  rescue_from Forbidden, with: :forbidden_response

  before_action :authenticate!, except: [:index, :show]

  include JSONAPI::ActsAsResourceController

  def create_operations_processor
    TempMongoidOperationsProcessor.new
  end

  def context
    # FIXME: method is being passed because of authorization kludge
    { current_participant: current_participant, request_method: request.method }
  end
end

Nevermind, managed to use the route helpers in the end. I wrongly assumed they wouldn't work unless the controllers extended the resource controller (instead of including the module), but they do. About the flat uris, it sounds like they'd be possible for the related link, but self should point to an endpoint for managing the relationship itself, something I just grasped about te spec. So even if related is flat, self needs to be nested, so not sure if there's any gain in making the former flat.

Maybe using something like Rails.application.routes.recognize_path in the serializer could help with this integration between serializers and routes? (sorry for the multiple comments, btw. convoluted thought process lol)

routes are not tied to the serialization and links generation code. That's an oversight we should get fixed.

@lgebhardt I realized it's been... a while. But has that changed at all? In particular, it seems OK that the  self link is there (e.g., "self": "http://example.org/api/v2/comments/908005740/relationships/activity") b/c it's implemented by default. But the related link (e.g., "related": "http://example.org/api/v2/comments/908005740/activity") isn't, and is thus broken.


First draft of MDS high level design document.

This is the first draft of what we discussed on Friday...... I will call something for Tuesday to discuss. I will bring a blue tin hat........

Minor revisions from Jan. 20 meeting:
For any (*disaster-free!) run that's initiated, you should be guaranteed to get a start_run event and an end_run event.
The beamline_config will not be nested into the start_event. Many runs may point to the same beamline_config.
Major revision involving trigger events becoming event-wise start and end time.


Wraps UnityEditor specific code in #if UNITY_EDITOR to ensure that th…

Wraps UnityEditor specific code in #if UNITY_EDITOR to ensure that the project will build for any platform.

Thanks for the great little tool.
My spelling is bad too. They had the red squiggles under them and I couldn't resist.  :)


Add Vowpal wabbit or docker(-compose) support

Is your feature request related to a problem? Please describe.
I try to play with vowpal wabbit, more explicit a derived version of your sagemaker RL examples. I tried to lunch a docker container but docker and docker-compose is not available. So I tried to execute the training files in the notebook itself, but vowpal wabbit is not available.
Describe the solution you'd like
Either support docker & docker-compose to play with provided examples in studio labs, or at least allow setting up vowpal wabbit locally.
Describe alternatives you've considered
enable docker and docker-compose to woraround and be able to execute the provided RL examples that leverages docker-compose for local training
Additional context
Add any other context or screenshots about the feature request here.

Hi there, Is it possible to use the default installation via pip -> pip install vowpalwabbit ? Currently there are no plans to support running containers within the studio lab environment.

Hey there,
I guess yes. The main point I want to highlight is, there are many examples provided by amazon that require docker. It would be great to support it from this perspective. You could event make the examples really interactive and connect it closer to studio lab.

Thanks for this feedback!


TrayMenu: changelog link

Description
At the moment the only way to get to change log is to open CHANGELOG.md fille in OpenPype codebase (not everyone knows how to get there) or find it on Github (also not everyone knows how to).
Solution
Easiest way would be to have the Github link available at our OpenPype info widget (TrayMenu). This link would open Githhub page in web brouwser.

[cuID:OP-1133]

I didn't even know there was a changelog link. So I went to that part in UI and there was none:

I suppose there isn't any nowadays?

Well. that's exactly what the issue is about. Adding one :)

I must have been reading this with my eyes closed - you are right, of course.

Considering this had no traction since start and it'll become more complex than this in AYON (addons split to multiple repos with their own changelogs). I'm closing this.


[RESOLVED] No SNMP data collection

Problem started at 17:40:47 on 2023.04.11
Problem name: No SNMP data collection
Host: HP_iLO
Severity: Warning
Operational data: Current state: not available (0)
Original problem ID: 16095
Event details in Zabbix: http://<IP_ADDRESS>/tr_events.php?triggerid=23274&eventid=16095

Problem has been resolved in 2d 13h 25m 0s at 07:05:47 on 2023.04.14
Problem name: No SNMP data collection
Host: HP_iLO
Severity: Warning
Original problem ID: 16095


NTR: resistance to 2-DOG induced cell lysis

def
resistance to cell lysis induced by the presence of higher concentration of 2-deoxyglucose in the vegetative growth phase of life cycle of cell
this is a user request. It might seem a bit odd but it is what they show...
"Lysis is more efficient in cells grown in a medium containing high glucose than cells grown in the presence of low glucose. Cells of the four deletion strains lyse in the presence of 2-DG when grown at the low glucose concentration but not so at the high concentration. This indicates that the four genes are involved in the control of 2-DG induced lysis and that this control is glucose dependent. Independent of the glucose concentration, cells of the strain contain- ing the overexpressed gene odr1 are not lysed by 2-DG."]
PMID:29319508

and 2-deoxyglucose induced cell lysis
A phenotype in which a cell lyses, i.e. the plasma membrane ruptures and cytoplasm is lost, in presence of higher concentration of 2-deoxyglucose.

resistance to 2-DOG induced cell lysis

I would be inclined to just use the existing resistance to 2-deoxyglucose term (FYPO:0000064). We haven't folded in the mechanism for any other chemical sensitivity or resistance terms.

2-deoxyglucose induced cell lysis

Is this worth pre-composing? We do pretty much everything except cell population growth as term+conditions (so lysis FYPO:0000647 + condition PECO:new 2-deoxyglucose added).

edits to the summary don't change my position

I haven't edited anything?

@Antonialock  you have more experience with this type of paper. Should this just be done with conditions. There  seems to be more here than just resistance, but I just don't know how to capture it....

I haven't edited anything?

from the top of this very page:

Yes that was the comment I added with the second request...
I didn't change/edit the first comment.

Well not after your comment. it might have been about a minute apart....as I went to the paper for the text. before the second part and your comment anyway....

I agree with Midori.
2DG messes up glycolysis.
I've added + 2-DG

I would be inclined to just use the existing resistance to 2-deoxyglucose term (FYPO:0000064). We haven't folded in the mechanism for any other chemical sensitivity or resistance terms.

Is

this worth pre-composing? We do pretty much everything except cell population growth as term+conditions (so lysis FYPO:0000647 + condition PECO:new 2-deoxyglucose added).

OK I know how to do it (to get the differntiation of the phenotypes)
Its the lysis phenotype with the  different conditions  + 2DG and low or high glucose...
I was having a blank.
I didn't want to ignore because the first author  so clearly understood the annotation process and did the annotation so precisely
https://curation.pombase.org/pombe/curs/b466cec3cdc7c5c0
can I get + 2DG as a condition?

I added it already, that's what I meant by "I've added + 2-DG" :-)

Got you ;)


Random Crash

=========================== FCNPC Exception Handler ============================
Address: 0x702FD679

Registers:
EAX: 0x0000FFFF - EBX: 0x00000003 - ECX: 0x0071DAE8 - EDX: 0x0012FA00
ESI: 0x00000000 - EDI: 0x2ace848 - EBP: 0x0012F9A0 - ESP: 0x0012F9A0

Stack:
0x0: 0x0012F9B4 - 0x4: 0x702F7404 - 0x8: 0x0000FFFF - 0xC: 0x02ACE848 - 0x10: 0x00000001 - 0x14: 0x00000036
0x18: 0x0046C2E4 - 0x1C: 0x01449458 - 0x20: 0x0012FA00

Information:
  System: Windows
  SA-MP: 0.3.7 R2-1
  FCNPC: 1.0.2

Functions:
  FUNC_CPlayerPool__DeletePlayer: 0x466570
  FUNC_CPlayer__Kill: 0x484620
  FUNC_CPlayer__EnterVehicle: 0x484c70
  FUNC_CPlayer__ExitVehicle: 0x484f50
  FUNC_CPlayer__SpawnForWorld: 0x486d30
  FUNC_GetVehicleModelInfo: 0x488240
  FUNC_CConsole__GetIntVariable: 0x48b5b0
  FUNC_ClientJoin_RPC: 0x4918f0

Pointers:
  VAR_ServerAuthentication: 0x4f5fe8
  VAR_NetVersion: 0xfd9

Offsets:
  OFFSET_RemoteSystemManager: 0x33c
  OFFSET_RemoteSystemSize: 0xcb8
  OFFSET_RemoteSystem__ConnectMode: 0xcb0
  OFFSET_RemoteSystem__Unknown: 0xcb5
=========================== FCNPC Exception Handler ============================

Just created simple loader for NPC's and wanted to see if its gonna work as i expected.
Server crashed only after a minute or two without any reason.
Only three functions are used inside filterscript init (no FCNPC publics at all).
FCNPC_Create
FCNPC_Spawn
FCNPC_SetAngle
Plugins list: crashdetect mysql sscanf streamer MapAndreas PathFinder FCNPC
Pretty sure other plugins doesnt have anything to do with it. Also i didnt load height map or init MapAndreas in FCNPC (like i said only 3 functions above are used from FCNPC) .
After that i started server again, tested for like 5 minutes, shooting and killing NPC's, driving over them, not a single problem...
EDIT: Okey...maybe im insane but i just recreated crash, twice... When you drive a bike, and you fell from it (hit something and fly over), server crashes...Insane right? Can someone could confirm these?
EDIT2: Tested everything a little bit more and i concluded that crashes are happening due to player taking damage. (If you bow up yourself with rocked once, server crashes, even if you dont die). I tried to add FCNPC_OnTakeDamage callback and return both 1 or 0, it still crashes. But im now 100% sure its the damage system that makes problems.

Okay, im bloody idiot...I didnt check for updates, i was using 1.0.2, its fixed in 1.0.4.
Sorry :smile:


Publish Board Meeting Minutes from August 2021 Meeting

Please check the draft minutes in HackMD, make necessary edits, and publish the minutes here.

Could the minute writer see whether they can resolve this issue? Was that @Teebusch?

closed by df79a7505bb7842da964c198da78fac222b093a9


Field name modified if showsize is enabled

From<EMAIL_ADDRESS>on October 06, 2011 17:07:41
When the option showsize is enabled, each time you click onto a field to modify it, its size value between brackets is appended to the field name.
I.e., if the field name is "name", type char(80), when you modify the field the first time, its name becomes "name (80)", the second time "name (80) (80)" and so on.
Original issue: http://code.google.com/p/wwwsqldesigner/issues/detail?id=138

From ondrej.zara on October 06, 2011 08:16:05
Oops, sorry! Will fix this...
Status: Accepted
Owner: ondrej.zara

From ondrej.zara on October 06, 2011 23:45:26
Fixed in changeset 103:b06737612b95.
Status: Fixed


ValueError: Sensor sensor.nozzle provides state value '2', which is not in the list of options provided

I think I am getting this error because it is a different model to the one that you are using. Would need to do some more checking to see what the 2 value is.
Logger: homeassistant
Source: components/sensor/__init__.py:572
First occurred: 07:51:15 (32 occurrences)
Last logged: 08:06:45

Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 918, in _update_entity_states
    await asyncio.gather(*tasks)
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 713, in async_update_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 809, in _async_write_ha_state
    state = self._stringify_state(available)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 748, in _stringify_state
    if (state := self.state) is None:
                 ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/sensor/__init__.py", line 572, in state
    raise ValueError(
ValueError: Sensor sensor.nozzle provides state value '2', which is not in the list of options provided

What was connected to your steam generator then you get this message?

Machine is not even switched on but does have milk carafe attached to the nozzle

#80 did not resolve this issue unfortunately. Must be somewhere else.

Resolved


Add support for XML comments

Would be nice if XML comments could flow through to the HTML, maybe as tooltips on each of the elements.

But maybe an option to include them in the visible HTML.

@IanMercer I would have no clue how to do that and still make the OMD readable. This isn't API documentation. It's an OMD.


Revert "Update to python 3.6"

Need to revert because web3 v3 requieres old version of gevent which is not compatible with python 3.6
This reverts commit e2e34e852e954826f8a6f7270b11308d456819e2.
Closes #172

Codecov Report

Merging #176 into develop will decrease coverage by <.01%.
The diff coverage is 54.54%.

@@             Coverage Diff             @@
##           develop     #176      +/-   ##
===========================================
- Coverage    59.99%   59.98%   -0.01%
===========================================
  Files           42       42
  Lines         2702     2699       -3
===========================================
- Hits          1621     1619       -2
+ Misses        1081     1080       -1

Impacted Files
Coverage Δ

relay/blockchain/proxy.py
77.38% <100%> (-1.2%)
:arrow_down:

relay/blockchain/currency_network_proxy.py
80% <100%> (-0.96%)
:arrow_down:

relay/utils.py
33.33% <100%> (ø)
:arrow_up:

relay/api/tokens/resources.py
56.25% <100%> (+0.93%)
:arrow_up:

relay/streams.py
96.51% <100%> (ø)
:arrow_up:

relay/ethindex_db.py
39.81% <60%> (-0.56%)
:arrow_down:

relay/relay.py
28.73% <7.14%> (+0.24%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7fdf1d4...e66632f. Read the comment docs.


possible bug in zipkin-ruby or in the visualization

After updating to 2.12.2 we have seen that many of our traces get smashed to the left of the graph

When clicking on one of those I get:

Only SA information.
Looking at the json, these could be the relevant sections:
  {
    "traceId": "7309633b16097a5c",
    "parentId": "7309633b16097a5c",
    "id": "4f829b6d0eb7bfb5",
    "name": "post",
    "localEndpoint": {
      "serviceName": "hermes-sandbox",
      "ipv4": "<IP_ADDRESS>",
      "port": 443
    },
    "tags": {
      "sa": "1"
    }
  },

  {
    "traceId": "7309633b16097a5c",
    "parentId": "7309633b16097a5c",
    "id": "4f829b6d0eb7bfb5",
    "kind": "SERVER",
    "name": "post /path_info_in_here_edited_out",
    "timestamp":<PHONE_NUMBER>539229,
    "duration": 999430,
    "localEndpoint": {
      "serviceName": "hermes-sandbox",
      "ipv4": "<IP_ADDRESS>",
      "port": 3000
    },
    "tags": {
      "http.path": "/apps/exd/api/mauth_proxy"
    }
  },

Note how the SA notation is alone in a different section of the JSON even when they have the same ID.
I assume the UI is rendering any one of the other depending on the orther it finds this information. Thus it gets an annotation with no further info and gets rendered this way.
These traces are being sent by the zipkin-ruby tracer. I suspect a bug there,  but please verify.
Zipkin-ruby does not have support for v2 traces yet which can be the root of this.
If you determine this is a bug in the current UI, I do not care really as long as Lens do not have the same bug as we will migrate as soon as it is available in the stable image.

CC @johnduhart

what this smells like is v1 json encoding 1 instead of true for the "sa"
binary annotation.. could that be the case? in thrift boolean true is
encoded as a byte with one bit set..
On Tue, Feb 26, 2019, 3:51 AM Jordi Polo Carres<EMAIL_ADDRESS>wrote:

After updating to 2.12.2 we have seen that many of our traces get smashed
to the left of the graph
[image: screen shot 2019-02-25 at 11 41 28 am]
https://user-images.githubusercontent.com/1803073/53363826-5a50da00-38f2-11e9-8816-a638ba8337ff.png
When clicking on one of those I get:
[image: screen shot 2019-02-25 at 11 42 36 am]
https://user-images.githubusercontent.com/1803073/53363908-7eacb680-38f2-11e9-9646-543ae893540b.png
Only SA information.
Looking at the json, these could be the relevant sections:
{
"traceId": "7309633b16097a5c",
"parentId": "7309633b16097a5c",
"id": "4f829b6d0eb7bfb5",
"name": "post",
"localEndpoint": {
"serviceName": "hermes-sandbox",
"ipv4": "<IP_ADDRESS>",
"port": 443
},
"tags": {
"sa": "1"
}
},
{
"traceId": "7309633b16097a5c",
"parentId": "7309633b16097a5c",
"id": "4f829b6d0eb7bfb5",
"kind": "SERVER",
"name": "post /path_info_in_here_edited_out",
"timestamp":<PHONE_NUMBER>539229,
"duration": 999430,
"localEndpoint": {
"serviceName": "hermes-sandbox",
"ipv4": "<IP_ADDRESS>",
"port": 3000
},
"tags": {
"http.path": "/apps/exd/api/mauth_proxy"
}
},
Note how the SA notation is alone in a different section of the JSON even
when they have the same ID.
I assume the UI is rendering any one of the other depending on the orther
it finds this information. Thus it gets an annotation with no further info
and gets renderingg this way.
This traces are being sent by the zipkin-ruby tracer. I suspect a bug
there, but please verify.
Zipkin-ruby does not have support for v2 traces yet which can be the
root of this.
If you determine this is a bug in the current UI, I do not care really as
long as Lens do not have the same bug as we will migrate as soon as it is
available in the stable image.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/openzipkin/zipkin/issues/2414, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAD61-vFuOyUJPe878OuqAbv1azn8dDeks5vRD6pgaJpZM4bQptj
.

"sa" aside it looks like the span is sent in two documents instead of one.
it is odd that the IP address would change during the same request on the
same host. usually that is symptomatic of a loopback RPC. Even if it
weren't, we would have to guess which side of the RPC to stick the extra
span doc (the client or the server side).
thanks for posting.. will think about it. meanwhile if there is a way to
keep stable endpoint info and post a single doc instead of two that will be
good anyway.

We are hardcoding sa to 1 somehow we thought that was the way. We'll change that to true (boolean or the string "true"?) if you can verify.
More weird still, I've looked at the code and I see where we set the sa, it is in the client span, which in fact it is in the trace also. At that point in the code there are other annotations, and they do show up:
  {
    "traceId": "7309633b16097a5c",
    "parentId": "4f829b6d0eb7bfb5",
    "id": "d1887feaeb2591de",
    "kind": "CLIENT",
    "name": "post",
    "timestamp":<PHONE_NUMBER>543929,
    "duration": 993347,
    "localEndpoint": {
      "serviceName": "hermes-sandbox",
      "ipv4": "<IP_ADDRESS>",
      "port": 3000
    },
    "tags": {
      "http.method": "POST",
      "http.path": "/v1/protocol_versions/94365fe0-d84d-4ed7-af48-3991b12bde4a/publish",
      "http.status": "200",
      "sa": "1"
    }
  },

and nowhere else the sa is setup, so really no idea where this particular sa with no extra info comes from.
So to summarize, we have 3 pieces here , a SERVER, a CLIENT, both I can find in the client code. And a SA which has no extra info which I can't find.
Also we have the SA == 1 thing which if you tell me so I'll change to the boolean true.

We are hardcoding sa to 1 somehow we thought that was the way. We'll
change that to true (boolean or the string "true"?) if you can verify.

definitely boolean true (even if string might work)
https://github.com/openzipkin/zipkin-api/blob/master/zipkin-api.yaml#L298
More weird still, I've looked at the code and I see where we set the sa, it

is in the client span, which in fact it is in the trace also. At that point
in the code there are other annotations, and they do show up:
{
"traceId": "7309633b16097a5c",
"parentId": "4f829b6d0eb7bfb5",
"id": "d1887feaeb2591de",
"kind": "CLIENT",
"name": "post",
"timestamp":<PHONE_NUMBER>543929,
"duration": 993347,
"localEndpoint": {
"serviceName": "hermes-sandbox",
"ipv4": "<IP_ADDRESS>",
"port": 3000
},
"tags": {
"http.method": "POST",
"http.path": "/v1/protocol_versions/94365fe0-d84d-4ed7-af48-3991b12bde4a/publish",
"http.status": "200",
"sa": "1"
}
},
and nowhere else the sa is setup, so really no idea where this particular
sa with no extra info comes from.

not sure.. maybe when late route info is sent.

So to summarize, we have 3 pieces here , a SERVER, a CLIENT, both I can
find in the client code. And a SA which has no extra info which I can't
find.
Also we have the SA == 1 thing which if you tell me so I'll change to the
boolean true.

So, the reason why the "sa" gets separated is  "sa" means "server address"
and that is a client attribute. "sa" should never be set on a server span.
So, when the attribute is sent standalone, the parser assumes this is a
client span, as if a remote address in v1 format was set on a server span,
it would be "ca" (client address) not "sa" (server address)
obv doing native v2 encoding would be  a win at some point (nudge nudge) as
that gets rid of the whole awkwardness and confusion around "sa" etc, as
remoteAddress is harder to get wrong.

The only place where I see SA being set is here
        span.record_tag(Trace::BinaryAnnotation::METHOD, method.upcase, Trace::BinaryAnnotation::Type::STRING, local_endpoint)
        span.record_tag(Trace::BinaryAnnotation::PATH, url.path, Trace::BinaryAnnotation::Type::STRING, local_endpoint)
        span.record_tag(Trace::BinaryAnnotation::SERVER_ADDRESS, SERVER_ADDRESS_SPECIAL_VALUE, Trace::BinaryAnnotation::Type::BOOL, remote_endpoint)
        span.record(Trace::Annotation::CLIENT_SEND, local_endpoint)
        response = @app.call(env).on_complete do |renv|
          record_response_tags(span, renv[:status].to_s, local_endpoint)
        end
        span.record(Trace::Annotation::CLIENT_RECV, local_endpoint)

Which is a client span. I never see it in a server span.
Yeah I want to do V2 also, not enough hands

ok so in the initial example, it looks like maybe the data was missing some of the client part (only getting the "sa" info).. weird. anyway this is more a troubleshooting type of conversation. We should probably not do this in an issue as it spams everyone watching. Let's move to gitter? https://gitter.im/openzipkin/zipkin

It is not missing information, the client span is there, the server span is there, and then there is this extra sa thing which I can't see where is coming from

Below is the complete json that creates this concern. I was able to reproduce it with the canonical frontend/backend app hacked here https://github.com/openzipkin/zipkin-ruby-example
The only option I can think of in span conversion are hacks:

for address keyword in "sa" "ca" "ma"

leniently treat any binary annotation with address keyword as truthy (ex ignore the value which would let "1" be same as true. The value is actually unimportant.
drop any binary annotation which has address keyword and a value besides true.

@openzipkin/core @drolando any opinons?
[
  {
    "traceId": "23ecf2011083ef43",
    "parentId": "23ecf2011083ef43",
    "id": "e323382e9c9a9653",
    "kind": "SERVER",
    "name": "get",
    "timestamp":<PHONE_NUMBER>864992,
    "duration": 5466,
    "localEndpoint": {
      "serviceName": "backend",
      "ipv4": "<IP_ADDRESS>",
      "port": 80
    }
  },
  {
    "traceId": "23ecf2011083ef43",
    "id": "23ecf2011083ef43",
    "kind": "SERVER",
    "name": "get",
    "timestamp":<PHONE_NUMBER>856589,
    "duration": 21271,
    "localEndpoint": {
      "serviceName": "frontend",
      "ipv4": "<IP_ADDRESS>",
      "port": 80
    },
    "tags": {
      "http.path": "/"
    }
  },
  {
    "traceId": "23ecf2011083ef43",
    "parentId": "23ecf2011083ef43",
    "id": "e323382e9c9a9653",
    "kind": "CLIENT",
    "name": "get",
    "timestamp":<PHONE_NUMBER>857207,
    "duration": 14067,
    "localEndpoint": {
      "serviceName": "frontend",
      "ipv4": "<IP_ADDRESS>",
      "port": 80
    },
    "tags": {
      "http.method": "GET",
      "http.path": "/api",
      "http.status": "200"
    }
  },
  {
    "traceId": "23ecf2011083ef43",
    "parentId": "23ecf2011083ef43",
    "id": "e323382e9c9a9653",
    "name": "get",
    "localEndpoint": {
      "serviceName": "backend",
      "ipv4": "<IP_ADDRESS>",
      "port": 9000
    },
    "tags": {
      "sa": "1"
    }
  }
]

fwiw personally I am fine with conversions of address keywords ignoring the binary annotation value. As you can see in the image of the original trace, mistaking remote address for local one really wrecks data conversion.

https://github.com/openzipkin/zipkin/pull/2415 should work around this, but fixing the encoding should go out asap


Unable to run multiple BatchSchedulers

I am calling the below code snippet inside a loop
BatchScheduler scheduler = new BatchScheduler(engine);
Date startDate = new Date(new Date().getTime() + 1);
scheduler.scheduleAtWithInterval(startDate, intervalInMinutes);
scheduler.start();
--
basically I am trying to schedule multiple engines , each with its own reader,mapper and processor.
stack trace is as below:
org.easybatch.integration.quartz.BatchSchedulerException: An exception occurred during scheduler startup
at org.easybatch.integration.quartz.BatchScheduler.start(BatchScheduler.java:124)
at com.lmkt.we.job.JobManager.startDataPull(JobManager.java:64)
at com.lmkt.we.bootstrap.BootStrap.contextInitialized(BootStrap.java:35)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4765)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5260)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1525)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1515)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.quartz.ObjectAlreadyExistsException: Unable to store Job : 'DEFAULT.batch-job', because one already exists with this identification.
at org.quartz.simpl.RAMJobStore.storeJob(RAMJobStore.java:279)
at org.quartz.simpl.RAMJobStore.storeJobAndTrigger(RAMJobStore.java:251)
at org.quartz.core.QuartzScheduler.scheduleJob(QuartzScheduler.java:886)
at org.quartz.impl.StdScheduler.scheduleJob(StdScheduler.java:249)
at org.easybatch.integration.quartz.BatchScheduler.start(BatchScheduler.java:119)
... 12 more

Hi Imran,
Thank you for pointing out this issue. In fact, scheduling multiple engines fails even if you use separate instances of the BatchScheduler. Here is how I reproduced the issue:
Engine engine1 = new EngineBuilder().build();
Engine engine2 = new EngineBuilder().build();

BatchScheduler scheduler1 = new BatchScheduler(engine1);
scheduler1.scheduleAtWithInterval(new Date(), 1);
scheduler1.start();

BatchScheduler scheduler2 = new BatchScheduler(engine2);
scheduler2.scheduleAtWithInterval(new Date(), 1);
scheduler2.start();

I get the same exception:
org.easybatch.integration.quartz.BatchSchedulerException: An exception occurred during scheduler startup
	at org.easybatch.integration.quartz.BatchScheduler.start(BatchScheduler.java:124)
	at org.easybatch.integration.quartz.BatchSchedulerTest.testJobScheduling(BatchSchedulerTest.java:31)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:77)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:195)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
Caused by: org.quartz.ObjectAlreadyExistsException: Unable to store Job : 'DEFAULT.batch-job', because one already exists with this identification.
	at org.quartz.simpl.RAMJobStore.storeJob(RAMJobStore.java:279)
	at org.quartz.simpl.RAMJobStore.storeJobAndTrigger(RAMJobStore.java:251)
	at org.quartz.core.QuartzScheduler.scheduleJob(QuartzScheduler.java:886)
	at org.quartz.impl.StdScheduler.scheduleJob(StdScheduler.java:249)
	at org.easybatch.integration.quartz.BatchScheduler.start(BatchScheduler.java:119)
	... 27 more

This is due to the use of the same in-memory SchedulerFactory with the same (default) job name.
I'll fix that asap in version 3.0.1-SNAPSHOT and I'll keep you informed once the fix is deployed to maven central.
Cheers
Mahmoud

yes that is exactly how I it occurred, the workaround is to implement your own RecordReader and use single scheduler for the multiple jobs.
Also please note that if this code is to be called inside a J2EE webapp, the container starts complaining about a thread that has not finished and maycause memory leak
SEVERE: The web application [/we] appears to have started a thread named [DefaultQuartzScheduler_QuartzSchedulerThread] but has failed to stop it. This is very likely to create a memory leak.
.

yes that is exactly how I it occurred, the workaround is to implement your own RecordReader and use single scheduler for the multiple jobs.
Also please note that if this code is to be called inside a J2EE webapp, the container starts complaining about a thread that has not finished and maycause memory leak
SEVERE: The web application [/we] appears to have started a thread named [DefaultQuartzScheduler_QuartzSchedulerThread] but has failed to stop it. This is very likely to create a memory leak.
.

yes that is exactly how I it occurred, the workaround is to implement your own RecordReader and use single scheduler for the multiple jobs.
Also please note that if this code is to be called inside a J2EE webapp, the container starts complaining about a thread that has not finished and maycause memory leak
SEVERE: The web application [/we] appears to have started a thread named [DefaultQuartzScheduler_QuartzSchedulerThread] but has failed to stop it. This is very likely to create a memory leak.
.

As you may have read in the job scheduling tutorial, the BatchScheduler API is a wrapper around the Quartz's low level APIs. so behind the scene, it's quartz  that is creating threads and easy batch has no control over them.
If you are using Easy Batch in a JEE server, you should use the thread executor service of your server and not rely on the BatchScheduler API. The engine is actually a java.util.concurrent.Callable so it can be scheduled using a java.util.concurrent.ScheduledExecutorService in a Java SE environment, or a javax.enterprise.concurrent.ManagedExecutorService in Java EE environment.
That said, I am not sure I understand your workaround using a custom record reader. A possible solution I am thinking about is to give every engine instance a unique Id, and use this id as the job Id for the quartz scheduler (This will prevent creating a scheduler with an existing id).
Last question, why has your thread failed to finish? Can you give me more details please?

sorry for late reply, I agree on ManagedExecutorService  in JavaEE, but that would make the application container dependent, for instance, the web app should be deployable on Jetty and tomcat also, ManagedExecutorService requires a container like JBoss or Weblogic etc.
About the custom record reader, I implemented the open() method in such a way that it handled all the jobs, eventually creating a single record,which is actually an array of records.

Hi Imran,
I have deployed version 3.0.1-SNAPSHOT in maven central repository with the fix for this issue. Every job has now a unique Id which allow scheduling multiple engines (This test is passing now).
Moreover, I have added a method to stop the scheduler (BatchScheduler.stop()) which will allow you to control when to start/stop each scheduler. You can also check the status of the scheduler with
the new isStarted and isStopped methods.
Could you please retest with this new version and keep me informed if it is ok for you?
Many thanks upfront
Mahmoud

Hello Imran,
FYI, this fix has been released in version 3.0.1.
Regards
Mahmoud


apidocs: Move stray descriptions to OpenAPI.

Testing plan:
GIFs or screenshots:

@timabbott This should be ready for review after the prep PR is merged. This step crosses the last hurdle in creating a common template, and I think we have reached maximum coverage once this gets merged :)

@timabbott Addressed most of the comments above. I think the PR would be easier to review after the prep one gets merged(since the things can't be verified directly on this PR as of now).

I just merged the prep commit, so you can rebase this now.  It'd be great if you can post the HTML diff output after completing this one.
(As a sidenote, did you end up writing a tool for that sort of HTML diff work?  I wonder if it'd be worth checking it and documenting it).

@timabbott Done with the rebase. Attaching the diff of the files:
old-archive-stream new-archive-stream
246d245
<
Note: The following parameters are all URL query parameters.

old-get-events new-get-events
320,324d319
<
Note: The parameters documented above are optional in the sense that
< even if you haven't registered a queue by explicitly requesting the
< http://localhost:9991/api/v1/register endpoint, you could pass the parameters for
< the http://localhost:9991/api/v1/register endpoint to this
< endpoint and a queue would be registered in the absence of a queue_id.

4534,4535c4529,4534
<
If the target event queue has been garbage collected, you'll get the
< following error response:

---
>
This error occurs if the target event queue has been garbage collected.
> A compliant client will handle this error by re-initializing itself
> (e.g. a Zulip web app browser window will reload in this case).

>
See the /register endpoint docs for details on how to
> handle these correctly.

>
The following is the error response in such case:

4543,4546d4541
<
A compliant client will handle this error by re-initializing itself
< (e.g. a Zulip web app browser window will reload in this case).

<
See the /register endpoint docs for details on how to
< handle these correctly.

old-get-messages new-get-messages
402,404d401
<
When a request is successful, this endpoint returns a dictionary
< containing the following (in addition to the msg and result keys
< present in all Zulip API responses).

601c598,601

When a request is successful, this endpoint returns a dictionary
> containing the following (in addition to the msg and result keys
> present in all Zulip API responses).
> A typical successful JSON response may look like:

old-get-stream-id new-get-stream-id
263d262
<
Note: The following parameters are all URL query parameters.

old-get-streams new-get-streams
270d269
<
Note: The following parameters are all URL query parameters.

old-get-user new-get-user
257d256
<
Note: The following parameters are all URL query parameters.

old-get-user-by-email new-get-user-by-email
264d263
<
Note: The following parameters are all URL query parameters.

old-get-users new-get-users
276d275
<
Note: The following parameters are all URL query parameters.

old-update-message new-update-message
218a219,220
>
Note: See configuring message editing for detailed documentation on under what
> circumstances users are allowed to edit topics.

278,285d279
<
Permissions

<
You only have permission to edit a message if:

<

<
You sent it, OR:

<
This is a topic-only edit for a (no topic) message, OR:

<
This is a topic-only edit and you are an admin, OR:

<
This is a topic-only edit and your realm allows users to edit topics.

<

old-upload-custom-emoji new-upload-custom-emoji
246c246
<

curl -sSX POST http://localhost:9991/api/v1/realm/emoji/demo \
---
>

curl -sSX POST http://localhost:9991/api/v1/realm/emoji/smile \
254,257c254,260
<
As described above, the image file to upload must be provided in the
< request's body.

<
Emoji name

<
The emoji name can only contain letters, numbers, dashes, and spaces.
---
>

>
emoji_name string required in path

>

>         Example: "smile"
>

>

The name that should be associated with the uploaded emoji image/gif.
> The emoji name can only contain letters, numbers, dashes, and spaces.
260c263,268
< handles emoji).

---
> handles emoji).

>

>

>
>
As described above, the image file to upload must be provided in the
> request's body.

Also, I'm using combination of a python script and a bash script to store raw HTMLs in files old-<endpoint-name> and new-<endpoint-name>, and showing diff of the respective older and newer files if they not exactly same(using cmp bash command).

For upload-custom-emoji, I think smile is a terrible example, since that's a built-in emoji name, and custom emoji are primarily for adding new custom emoji. We can merge this, but we should change that example to something that's a reasonable custom emoji name; I think demo (as was previously presentis OK, butmy_new_emoji` might be better.

The emoji name can only contain letters, numbers, dashes, and spaces.

Is this accurate?  I would have thought that we used underscores, not -s, for this purpose.

Merged after a small whitespace change for the "delete message" one to add a blank line after the heading.  Thanks @MSurfer20!
Can you take care of the small follow-up issues mentioned in my last batch of comments?
It's exciting how close we are to being able to delete the templates.