Prototype: Course recommendation engine using Apache Spark

I have been thinking of this idea for a while, but never had the time to implement it until now.
Proposal
Since UW Flow is approaching 12000 users and growing, I think this a good time to setup a course recommendation engine.
After some research, I believe Apache Spark is a great tool to use for this purpose since it offers fast, parallelized cluster computing as well as a lot of rich tools and libraries for machine learning algorithms.
I also want to use Collaborative Filtering as the algorithm for the recommendation engine. The advantages of this algorithm is that it takes account of similar user's preferences to make recommendations.
Prototype Explanation
The prototype I made sends a list of triples (user_id, course_id, rating) to spark. The rating, for now, is our Liked It button. Spark uses the data to train a model using [ALS] (http://bugra.github.io/work/notes/2014-04-19/alternating-least-squares-method-for-collaborative-filtering) (Alternating least squares). The model is trained to predict the course rating (0.0-1.0) of all courses not yet rated by the user. Then given a user and the trained model, it returns the top 5 highest courses rated by the user as recommendations.
Running this PR
If you want to test this PR, you need to install and build spark and then set $SPARK_HOME to your spark install directory (uncomment some code in engine.py to set this up). Also install these python modules:
pip install numpy
pip install py4j

Then call
make recommendations user_id=<user_id>

user_id is the m.User.id property of each user
user_id parameter is optional, if not provided, the code will recommend courses to a random user in the database

Next Steps
If we decide to go with this approach, here are some next steps and problems I think we need to solve:
Optimization
In order to optimize the engine, we should run metrics against UW flow's real user data. (I am currently running the algorithm with some test data that I generated)
The most common step to evaluate learning algorithms is to split the available data into 3 sets: training (60%), validation (20%) and test (20%). Then we choose the optimal parameters based on the lowest error on the cross validation set. Finally, we measure the final error of the algorithm on the test set.
To do this, I need access to the entire or at least a subset of our user data.
Another possible optimization is to train the algorithm using more data (eg. shortlisted courses, easiness, usefulness)
Integration
If we go with the approach, I suggest we integrate the engine with the application like this:

The model is trained every day with the new user data - run it with aggregator.py?
Whenever /user is loaded, front-end queries the recommender for the top "n" recommendations
Display the top "n" results to the user in the front-end, maybe we can reuse this design

Deployment
We need to set-up spark for deployment.
The biggest advantage to using machine learning is that since our data is infinitely growing, our recommendations should also get much better over time.

Hey Charles! Thanks for the PR, I'm going to give it a shot locally, and I'll comment with my thoughts.

So I'm trying to test this locally, and am running in to an issue. (Note, I have production data on my machine, so it's actually training with real data)
I got spark by running brew install apache-spark, and am running your script with PYTHONPATH=.. pyspark data/engine.py userId where 'userId' is the id for my local profile.
However, it's throwing an error, saying ERROR:__main__:User has not rated any courses yet after it says the model has been trained. I've definitely rated courses, so any idea what's happening here?
I really love the idea, and it's great you put together a prototype PR so we can play around with it before putting up a formal one :)
Let me know when you get the chance, I want to get this working!

Hey Jeff, thanks for taking the time to try out my PR!
As for the error you're getting, checkout my latest code, it will give you a more informative error message.
If you're still an error, reply back to me with the error message or message me on fb.
Can you also check that you rated "Liked It" for any courses? The spark library only gives recommendations to users that "Liked" or "Disliked" at least one course, and throws an error otherwise. I suspect this is what's happening with you.

Alright, new error (and more informative!)
ERROR:__main__:'MatrixFactorizationModel' object has no attribute 'recommendProducts'
I double checked and I've liked courses, but this seems like a different error. Any thoughts? Would love to get this running, it'd be a great feature to have!

I'm wondering if maybe it's a versioning issue? What version of Spark are you using?

So it turns out it was a versioning issue, brew installed an old version of spark (1.3.1) that didn't have that method.
Now that that's figured out, I was able to run it successfully, and get recommendations! However, they were all courses I'd already taken, so I think we should show the top 5 that the user hasn't taken yet. I'm not quite sure what you meant in your original description by " Then given a user and the trained model, it returns the top 5 highest courses rated by the user as recommendations."
Does that mean the user has to rate a course to have it as a recommendation? Once we clear that up, then we can look at improving the algorithm and moving this from a prototype to a full-blown feature!

Amazing. So cool to see this kind of stuff come in as PRs.
Remember that you'll need to update https://github.com/UWFlow/rmc/blob/master/aws_setup/setup.sh to ensure that Spark gets installed on the server if you have to rebuild from scratch, as well as installing it for dev usage too: https://github.com/UWFlow/rmc/blob/master/linux-setup.sh and https://github.com/UWFlow/rmc/blob/master/mac-setup.sh

@JGulbronson  glad to see its working :smile:
I just updated my PR so that it will only recommend courses not already taken by the user, give it a try!
What I meant by
Then given a user and the trained model, it returns the top 5 highest courses rated by the user as recommendations.
is that the trained model will have the predicted ratings of all courses not taken by the user, and my script will return the top n highest predictions by the model. So if the model predicts for user x:
Course | Rating
A | 0.21
B | 0.89
C | 1 (actual user rating)
D | 0.78
E | 0.99
F | 0 (actual user rating)
and we want to recommend 3 courses, it will recommend E, B, D to the user.
For the case where the user liked no courses, the algorithm can't make a prediction since it does not have any information about the user. In this case, I think we should just recommend the most liked courses in our database to the user. This is not implemented in this PR yet.

From my understanding, collaborative filtering's base case would recommend
the most popular courses if there's no user-specific input, no?
Awesome stuff, btw. :)
On Sat, Dec 12, 2015 at 9:24 PM, Charles Qi<EMAIL_ADDRESS>wrote:

@JGulbronson https://github.com/JGulbronson glad to see its working [image:
:smile:]
I just updated my PR so that it will only recommend courses not already
taken by the user, give it a try!
What I meant by
Then given a user and the trained model, it returns the top 5 highest
courses rated by the user as recommendations.
is that the trained model will have the predicted ratings of all courses
not taken by the user, and my script will return the top n highest
predictions by the model. So if the model predicts for user x:
Course | Rating
A | 0.21
B | 0.89
C | 1 (actual user rating)
D | 0.78
E | 0.99
F | 0 (actual user rating)
and we want to recommend 3 courses, it will recommend E, B, D to the user.
For the case where the user liked no courses, the algorithm can't make
a prediction since it does not have any information about the user. In this
case, I think we should just recommend the most liked courses in our
database to the user. This is not implemented in this PR yet.
—
Reply to this email directly or view it on GitHub
https://github.com/UWFlow/rmc/pull/254#issuecomment-164228166.

@divad12 Yes, that's true. Unfortunately, the spark method I'm calling to make the recommendations throws an error whenever I input a user that has no ratings:
IN > model.recommendProducts(<user_id_with_no_ratings>, 5)
: java.util.NoSuchElementException: next on empty iterator```

 so we have to implement the *base case* ourselves.

@JGulbronson I implemented the base case, so users that doesn't rate any courses will get recommended the highest rated courses with the most ratings. Give it a try when you have time, and let me know when we can start implementing this as a full blown feature.

Awesome! I'm just finishing up work this week, but I should be able to pull and try it locally either tomorrow night or Friday at the latest.
In terms of implementing this as a full blown feature, I think there are a couple things that we'd need to do (in no particular order)
Train spark with real data
It'd be great to train the algorithm using real user data from Flow. I'm looking in to providing you with user data, with Personally Identifiable Information (PII) removed, such as names, emails, password hashes, etc. The courses taken, program, as well as friend graph would thus remain intact, which I think should be sufficient as a dataset for training, while also respecting the privacy of our users. @divad12 @jlfwong @jswu @mduan I'd like your input on this one, to make sure I'm handling things appropriately.
Front-end work
There needs to be some front-end component to this, so that users can actually see what we're recommending! At this point, it could be as simple as a view saying "Flow recommends these courses!" and then has links to 3-5 courses. The idea is we build it, get it out there, and then iterate on it. I'd be more than happy to take this on, as it shouldn't take me very long to complete, and we don't have to hold up the deploy. Which brings me to
Installing Spark on our server
This is something I'll have to do, and set up cron jobs for running the script. Should be fairly straightforward, I'll just have to make sure it doesn't drastically affect the server's performance while the cron job is running. I can look in to installing it on the server over the break. Note, you'd also want to add spark as a dependency to the project, though perhaps an optional one as it's not strictly necessary for getting everything running locally.
Clean up the code
I'll go through and give your existing code a review in the next couple days. Seems the library is pretty powerful, so there's not a lot of code (which is great), so we'll just make sure we have it nice and clean, with some good error reporting incase we encounter an edge case in production.
Code without tests is broken code
I really want all new code to have associated tests. It'd be great to have some here, perhaps some unit tests, as well as system tests if we're feeling particularly adventurous. We should have some examples already in the code base, but we can always talk about what we want to test in the Hipchat room.
Deploy and promote
Once we finish all the above stuff, I think it'll be good to merge/deploy. It'd be great to have it out at the start of next term, as some people might still be looking to add a new course in that first week of classes. That said, we certainly don't want to rush it, and make sure our first iteration is a functional feature that is of use to our users. Once it is deployed, then one of us will write a blog post, and I'll put it on all our social media accounts.
Alright, that was a lot of stuff. If you have any questions, let me know! Like I said, I'll look at getting you anonymized data, and giving you a code review to start. From there, we'll kinda check things off as they get done. Really liking the potential on this one, and I think after being trained with real user data, it'll be a  great feature!

@JGulbronson any chance to take a look at my PR?

@JGulbronson Alright, I addressed your comments and make a couple of changes to integrate with the flow backend.

make train_engine will train the model and write user.json, course.json and trained_model to data/recommendations folder.

When the server starts up, it will load the data that were saved to data/recommendations folder into memory. Unfortunately, this means that our server will not run if we did not already train the model, as the files wouldn't exist and we wouldn't be able to make recommendations. What are you opinions on how to error handle this case? This also means every time we retrain the model, we need to reload the server to load the updated model.

I added an api endpoint: /api/v1/user/<user_id>/recommendations. This will return the top n recommendations for the user and could be used by the frontend.
You can use curl to test this, although you need to authenticate the user first - you could uncomment    the authenticate user code, although I am not sure if that will be ethical.

I moved all the spark and pythonpath dependencies to shell scripts instead of python.

Tell me what you think of the changes.
For next steps, I could look into writing some tests. I wrote some python nosetests for my last co-op, so they shouldn't take too long for me to write.
Also, are you able to provide me with real user data (without PII) anytime soon?

So, I think we should run this as a cron job around midnight, and update/save the courses to each person's profile. This way, the API call won't cause issues, and we choose when Spark is taking up the processor. Does that make sense? Code wise, it's looking good, I'll get you that info without PII tomorrow. Sorry for the delay, it's been a busy week.
I think if you can make it so that it will (efficiently) run recommend_course for every User, that'd be great, and hopefully it'd only have to generate the matrix once. Does that make sense?
Again, sorry for the delay, you'll have the data tomorrow!

I made a prototype design for the front-end. I copied the general "courses" style (slightly smaller font) with the "add to shortlist" button on the right.

@JGulbronson @jlfwong @divad12 @jswu @mduan Any feedback, comments, suggestions for improvements before I implement this?

Cool!
What do you think about putting that beneath the course schedule?
On Sat, Jan 9, 2016 at 11:23 AM, Charles Qi<EMAIL_ADDRESS>wrote:

I made a prototype design for the front-end. I copied the general
"courses" style (slightly smaller font) with the "add to shortlist" button
on the right.
[image: image]
https://cloud.githubusercontent.com/assets/6456601/12217854/ecda5b2c-b6db-11e5-863f-8c3afa2af253.png
@JGulbronson https://github.com/JGulbronson @jlfwong
https://github.com/jlfwong @divad12 https://github.com/divad12 @jswu
https://github.com/jswu @mduan https://github.com/mduan Any feedback,
comments, suggestions for improvements before I implement this?
—
Reply to this email directly or view it on GitHub
https://github.com/UWFlow/rmc/pull/254#issuecomment-170272794.

@JGulbronson @divad12 I made a separate pull request for the front end, check it out!

@JGulbronson Here are some updates I made recently and I believe this pull request is complete and ready to be deployed provided it passes final code review and testing.

I added a new field to the user model: recommended_courses and saved the recommendations to all users in the database with my script make train_engine. The script only takes 5-10 minutes on my local machine, so I expect the speed will be even better on the server.

I optimized some machine learning parameters with the real data.

I added some metric reporting for the engine, each time my script runs, it will write to a timestamped json file in "data/recommendations/metrics/." The file will contain the number of ratings and the training and test errors. If my model is effective, the error is  expected to decrease with more ratings coming in. This is also useful information for anyone who wants to optimize the learning model in the future.

Finally, I added spark setup for linux development. My script should install spark (v1.6.0) on the root directory and setup some environment variables

As @jlfwong said, for deployment, we still need to set up spark for mac and aws.
For production, spark recommends we run the script through bin/spark_submit. Here are some documentation.
For my script, this will be: PYTHONPATH=..:${SPARK_HOME}/python ${SPARK_HOME}/bin/spark-submit data/engine.py. I think we should we should run this as a cronjob every day/week. Also see the above documentation for additional parameters we might want to consider.

@JGulbronson any time to take a look at this? Would like to get this merged before the term gets too busy (midterms and interviews)

Going to take a look this weekend. Sorry for the wait!

Would you be able to squash some commits together, and give me an idea of what dependencies this has? Then I'll look at getting them on the server.

Ok I squashed all my commits.
Dependencies are:
numpy==1.10.1
py4j==0.9
spark 1.6.0 with hadoop v2.6
Look at my linux-setup.sh changes to see how I installed spark for local development.

Tests broke... it looks like some dependency changed?

@ccqi @JGulbronson The tests are failing with this message:
No module named pyspark

Which is unsurprising, because it relies upon dependencies that are not installed in the Docker image https://hub.docker.com/r/jgulbronson/uwflow/.
It seems like the right fix is to update the Docker image to have the missing dependencies (both Spark and pyspark), and push a new version of the Docker image.


[KOGITO-8923] Passing function definition directly to action

Jenkins retest this

Jenkins retest this

(tests) - serverless-workflow-examples job #590 was: UNSTABLE
Possible explanation: This should be test failures
Please look here: https://eng-jenkins-csb-business-automation.apps.ocp-c1.prod.psi.redhat.com/job/KIE/job/kogito/job/main/job/pullrequest/job/kogito-runtimes.tests.downstream.serverless-workflow-examples/590/display/redirect
Test results:

PASSED: 77
FAILED: 1

Those are the test failures:

org.kie.kogito.examples.DataIndexRestIT.testDataIndexRest
1 expectation failed.JSON path data.ProcessInstances[0].state doesn't match.Expected: is "COMPLETED"  Actual: ACTIVE


Can't login to Grafana UI with own Grafana admin credentials[Bug]

Describe the bug
Deployed Grafana v10.0.1 through grafana operator version v5.2.0. I had set the admin credentials as user root and password secret.
Kubernetes secret is created with the below credentials.
       apiVersion: v1
       data:
          admin-password: secret
         admin-user: root
      kind: Secret
      metadata:
          annotations:
              meta.helm.sh/release-name: grafana-operator
              meta.helm.sh/release-namespace: grafana-test2
          labels:
             app.kubernetes.io/managed-by: Helm
             name: grafana-deploy-secret
             namespace: grafana-test2
       type: Opaque

Referring the above secret in grafana deployment
    env:
                - name: GF_SECURITY_ADMIN_USER
                  valueFrom:
                    secretKeyRef:
                      name: {{ .Values.grafana.clusterName }}-secret
                      key: admin-user
                - name: GF_SECURITY_ADMIN_PASSWORD
                  valueFrom:
                    secretKeyRef:
                      name: {{ .Values.grafana.clusterName }}-secret
                      key: admin-password

I am able to login with the configured credentials root/secret.
The issue is, If i change the credentials to admin/admin or testadmin/testpassword then can't login to grafana UI and it is throwing the error Invalid username or password.
I can login with only the credentials root and secret.
Version
Grafana Operator Version: https://github.com/grafana-operator/grafana-operator/tree/v5.2.0
Environment: GKE
Grafana Version: 10.0.1
To Reproduce
Steps to reproduce the behavior:

Deploy grafana operator version 5.2.0
Deploy grafana version 10.0.1
Create kubernetes secret with admin-user(testadmin) and admin-password(testpassword)
Refer the above secret in grafana deployment.
Now try to login to Grafana UI with the above credentials. It will throw you Invalid username or password

Expected behavior
I should be able to login with any other credentials which I set other than root/secret.
Runtime (please complete the following information):

OS: Linux
Grafana Operator Version v5.2.0
Environment: GKE
Deployment type: Deployed through helm chart

Hey @sshingarapu , are you updating the credentials directly in the secret? In that case, the Operator won't know the credentials have changed, since it doesn't own that secret. You could restart the Operator to force it to re-read that secret.

Hi @pb82 I am not updating the existing credentials. I have done two deployments. One is with root/secret and the other one is with testadmin/testpassword.
The deployment with root/secret is working without any issues. The other deployment with testadmin/testpassword is throwing Invalid username or password.

@HubertStefanski Please let me know if you need any more details

@sshingarapu please share your entire deployment so we can recreate the issue.

Confirming initially, seems like there's an issue somewhere, I'm still figuring out whether it's with upstream grafana or the operator
Looks like the custom secrets are being loaded into the pod properly, but somehow grafana isn't picking it up, this seems to be the case on both grafana 9 and 10

might be relevant https://github.com/grafana/grafana/issues/49055

It think it's the same issue that was discussed a while ago in the context of v4: https://github.com/grafana-operator/grafana-operator/issues/774
Basically, Grafana picks up credentials from environment variables only when it's just been created (DB is not initialized yet):

When you're using an ephemeral instance (no PVC / DB), Grafana will get recreated by the operator from scratch, thus the credentials rotation should work just fine;
When you're using a stateful Grafana, you have to fall back to grafana-cli to rotate credentials (+ update them in your CR/secret). Grafana DB schemes keep on changing, and grafana-cli imposes restrictions on password complexity (refuses to accept certain passwords), so it's not really a straightforward procedure when you want to automate it.

@sshingarapu something similar happened to me too when trying to reset admin password, i'm using grafana 10.0.3.
i saw that the login is working with "Admin" user with a Capital letter with the new password.
hope it helps somehow :)

From the issue that @HubertStefanski linked:

https://github.com/grafana/grafana/issues/49055#issuecomment-1695635466
=> So, in theory, we can instrument an ugly workaround with an init container, though, like I mentioned above, the usage of grafana-cli is not that straightforward (unlike the initial provisioning, grafana-cli is opinionated on password complexity). It should better get fixed on grafana side IMO.

I have been playing allot with this latley @weisdd , I'm thinking about writing a blog around using the operator together with a postgresDB, and then I also used the CLI to reset my password at startup.

Closing, as this is not something we can support due to the way which Grafana initializes these credentials in memory, as described above by @weisdd and @NissesSenap
We hope the other methods suggested here are a sufficient workaround for your use case


api-docs: Remove incorrect schema reference in unread_msgs object.

Removes an extraneous BasicStream schema reference in the unread_msgs object returned by the /register-queue endpoint.
Current documentation - find on page "unread_msgs"

API documentation diff
diff -r -B -u current_api_html/register-queue.html new_api_html/register-queue.html
--- current_api_html/register-queue.html	2022-08-08 11:17:52.404009817 +0200
+++ new_api_html/register-queue.html	2022-08-08 11:17:18.551780006 +0200
@@ -1614,7 +1614,7 @@

-
unread_msgs: (object)[]

+
unread_msgs: object

Present if message and update_message_flags are both present in
 event_types.

A set of data structures describing the conversations containing
@@ -1712,90 +1712,6 @@
 unread messages.

Changes: New in Zulip 4.0 (feature level 44).

-

-
stream_id: integer

-
The unique ID of the stream.

-

-

-
name: string

-
The name of the stream.

-

-

-
description: string

-
The short description of the stream in text/markdown format,
-intended to be used to prepopulate UI for editing a stream's
-description.

-

-

-
date_created: integer

-
The UNIX timestamp for when the stream was created, in UTC seconds.

-
Changes: New in Zulip 4.0 (feature level 30).

-

-

-
invite_only: boolean

-
Specifies whether the stream is private or not.
-Only people who have been invited can access a private stream.

-

-

-
rendered_description: string

-
The short description of the stream rendered as HTML, intended to
-be used when displaying the stream description in a UI.

-
One should use the standard Zulip rendered_markdown CSS when
-displaying this content so that emoji, LaTeX, and other syntax
-work correctly. And any client-side security logic for
-user-generated message content should be applied when displaying
-this HTML as though it were the body of a Zulip message.

-

-

-
is_web_public: boolean

-
Whether the stream has been configured to allow unauthenticated
-access to its message history from the web.

-

-

-
stream_post_policy: integer

-
Policy for which users can post messages to the stream.

-

-
1 => Any user can post.

-
2 => Only administrators can post.

-
3 => Only full members can post.

-
4 => Only moderators can post.

-

-
Changes: New in Zulip 3.0 (feature level 1), replacing the previous
-is_announcement_only boolean.

-

-

-
null, the default, means the stream will inherit the organization
-  level setting.

-
-1 encodes retaining messages in this stream forever.

-

-
Changes: New in Zulip 3.0 (feature level 17).

-

-

-
history_public_to_subscribers: boolean

-
Whether the history of the stream is public to its subscribers.

-
Currently always true for public streams (i.e. invite_only=False implies
-history_public_to_subscribers=True), but clients should not make that
-assumption, as we may change that behavior in the future.

-

-

-
first_message_id: integer | null

-
The id of the first message in the stream.

-
Intended to help clients determine whether they need to display
-UI like the "more topics" widget that would suggest the stream
-has older history that can be accessed.

-
Null is used for streams with no message history.

-

-

-
is_announcement_only: boolean

-
Whether the given stream is announcement only or not.

-
Changes: Deprecated in Zulip 3.0 (feature level 1). Clients
-should use stream_post_policy instead.

-

-

-

-

message_retention_days: integer | null

-
Number of days that messages sent to this stream will be stored
-before being automatically deleted by the message retention
-policy. There are two special values:

-

-
null, the default, means the stream will inherit the organization
-  level setting.

-
-1 encodes retaining messages in this stream forever.

-

-
Changes: New in Zulip 3.0 (feature level 17).

-

-

-
history_public_to_subscribers: boolean

-
Whether the history of the stream is public to its subscribers.

-
Currently always true for public streams (i.e. invite_only=False implies
-history_public_to_subscribers=True), but clients should not make that
-assumption, as we may change that behavior in the future.

-

-

-
first_message_id: integer | null

-
The id of the first message in the stream.

-
Intended to help clients determine whether they need to display
-UI like the "more topics" widget that would suggest the stream
-has older history that can be accessed.

-
Null is used for streams with no message history.

-

-

-
is_announcement_only: boolean

-
Whether the given stream is announcement only or not.

-
Changes: Deprecated in Zulip 3.0 (feature level 1). Clients
-should use stream_post_policy instead.

-

Screenshot of updated `unread_msgs` object in register response doc

Register response in dev environment (via curl)
Note: with event types "messages" and "update_message_flags"
{
   "last_event_id" : -1,
   "max_message_id" : 99,
   "msg" : "",
   "queue_id" : "1659950209:0",
   "result" : "success",
   "unread_msgs" : {
      "count" : 0,
      "huddles" : [],
      "mentions" : [],
      "old_unreads_missing" : false,
      "pms" : [],
      "streams" : []
   },
   "zulip_feature_level" : 135,
   "zulip_merge_base" : "",
   "zulip_version" : "6.0-dev+git"
}

Self-review checklist

[x] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[ ] Explains differences from previous plans (e.g., issue description).
[ ] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[ ] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[x] Each commit is a coherent idea.
[x] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[x] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[ ] End-to-end functionality of buttons, interactions and flows.
[ ] Corner cases, error conditions, and easily imagined bugs.

Merged, thanks @laurynmm!
It's too bad that our automated tests don't have a way to catch this class of error (fields that are documented as possible but never actually appear).


Add support for Slack multiselect

Description of Problem:
Slack multiselect element values posts are not supported in Rasa
2021-04-10 07:27:00 WARNING  rasa.core.channels.slack  - Received input from a Slack interactive component of type 'multi_static_select', for which payload parsing is not yet supported.
Overview of the Solution:
Slack multiselect values payload should be parsed to comma separated values that will be enough

Thanks for submitting this feature request 🚀 @b-quachtran will get back to you about it soon! ✨


Change CSS to be More Flexible

As it stands. the CSS for AHI is a little wonky. When the browser is enlarged from a small state, the blue background is only in the middle, with a white background behind the boxes.  The CSS should be changed so that it smoothly transitions between smaller and larger sizes.
Also, the two boxes are very large compared to the rest of the screen when shrunk, making it hard to see the entire screen. The boxes do not have a button form until shrunk to a width of 600px or less, so there should also be a size for the boxes when they are in-between full-screen and 600px.

This doesn't really apply anymore with our new ux!


Allow non-string content as breadcrumb divider

Fixes #25579
Allow non string content as breadcrumb divider. $breadcrumb-divider can also be set to none , that's why padding-left: $breadcrumb-item-padding; is set on .breadcrumb-item and not the ::before pseudo element.

Can we see a before/after example? And do we need to make any docs updates here to guide folks through customizing this?

There is no visual difference for this implementation. The static quotes in _breadcrumb.scss prevented other content than strings to be used as separator, this is now solved by using the quote() function in _variables.scss.
https://github.com/twbs/bootstrap/blob/0205fc8bdbd66c5c9101a95cd15c648a134a59a8/scss/_breadcrumb.scss#L18
I just updated the docs a bit to where I explained how the divider can be changed. It's also possible to use the url() function now, but this would in most cases require extra styling (as mentioned in #25579), so I didn't mention that in the docs.

Sorry for commenting on a old PR but I just was this...
https://github.com/twbs/bootstrap/blob/v4-dev/scss/_breadcrumb.scss
What is the point of the hack selectors? (line: 30 and 34)
Looks as a misstake.. or?
The last selector will always be used..
Also there is comments to suppress the linting error.. ?

@mackelito
https://github.com/twbs/bootstrap/blob/47ef0a877e6fc18c1e516f7a0855e0531c77af96/scss/_breadcrumb.scss#L24-L29

@MartijnCuppens Sorry for this.. we used the breadcrumb css/html in a non bootstrap project and realised that this is just a BS bug and not a "native hack thing".. so the hack was not needed for us 👍


Fe homework 8

https://video.drift.com/v/abqqfDe2KQ6MzBRwmwVWqrWuvvOok5NNupy4aeayxgdU/
Desktop view

Mobile view

Metric
Value

alert_status
:white_check_mark: OK

bugs
0

code_smells
3

reliability_rating
A (0 Bugs)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0

Metric
Value

alert_status
:x: ERROR

bugs
0

code_smells
3

reliability_rating
A (0 Bugs)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0

Go through the code smells
Follow the commit message's format
Work on the directory structure, code modularity and readability
Format the code properly


specify raw types

Codecov Report

Merging #581 into master will not change coverage.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master     #581   +/-   ##
=========================================
  Coverage     72.63%   72.63%
  Complexity      761      761
=========================================
  Files            49       49
  Lines          3625     3625
  Branches        453      453
=========================================
  Hits           2633     2633
  Misses          802      802
  Partials        190      190

Impacted Files
Coverage Δ
Complexity Δ

...bv2/mt/mappers/sharedtable/SharedTableBuilder.java
86.28% <ø> (ø)
58 <0> (ø)
:arrow_down:

...modbv2/mt/mappers/sharedtable/impl/FieldValue.java
52.94% <100%> (ø)
5 <0> (ø)
:arrow_down:

...haredtable/impl/MtAmazonDynamoDbBySharedTable.java
62.86% <100%> (ø)
71 <0> (ø)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8a8b1ca...e5588c7. Read the comment docs.


Misleading description on pub.dev

Please change the description on pub.dev for this package. Windows, Linux, MacOS, iOS and Android don't work. This is misleading for people looking for a genuine cross platform solution.

The supported platforms list on pub.dev is generated on their end.
It's true that this package only supports web. But it will nicely compile in your project if you target both native code and web code. It just won't work.
Also the package is called flutter_web_bluetooth. And the description says "Flutter web bluetooth is a Flutter plugin to add support for the web bluetooth api."
It's the same with this package for Bluez the linux Bluetooth library.
https://pub.dev/packages/bluez

Thanks for the update. Is there a way for you to merge this into flutter_reactive_ble? It already supports android and ios, hence giving support for all platforms (Android, iOS from their package, and Mac, Windows and Linux through web). That will be the true cross platform solution.

I'm not sure if it would work since Bluetooth web requires a pair flow. Where the user allowed access to a device and scanning for all devices in the area is not allowed.
But if the developers of flutter_reactive_ble want to add this library as a dependency to implement web support they are free to do do do.

Hey @jeroen1602  @realrk95 , i was trying to merge this package with flutter_reactive_ble , you guys can contribute here ,  we can atleast merge main features like scanning , connecting and subscribing to characterstics ,
checkout my fork here,
https://github.com/rohitsangwan01/flutter_reactive_ble/tree/master/packages/reactive_ble_web


Shuffle may get stuck if disk becomes full

Disco can get stuck (non-fatally) as follows if disk is full during shuffle
WARN: Creating index file failed at disco2-33: {badrpc,
{'EXIT',
{{badmatch,{error,enoent}},
[{ddfs_util,concatenate,2},
{shuffle,process_url,3},
{shuffle,
'-process_task/3-lc$^0/1-0-',
3},
{shuffle,
'-merged_index/3-fun-0-',
4},
{lists,foldl,3},
{shuffle,merged_index,3},
{shuffle,
combine_tasks_node,4},
{rpc,
'-handle_call/3-fun-0-',

Same problem here, no full disk
Erlang R13B03 (erts-5.7.4) [source] [64-bit] [smp:24:24] [rq:24] [async-threads:0] [hipe] [kernel-poll:false]


ci: parallel pytest execution

Experiment the use of pytest-xdist to speed up test execution in CI and dev environment:
see for example this job where tests are executed in under 10min on ubuntu.
Following issues with parallel execution of tests are solved :

some checks depended on the consistency at 2 moments in time of the result of disk_usage, which was anyway not a good idea (dependent on what happens elsewhere on the machine). It's now replaced with mocking
some parameterized tests had random sort order, which is not allowed by pytest-xdist (all workers must have exactly the same tests in the same order)
the command_factory test checked something in its teardown, assuming all tests were executed in the same process, which is not the case anymore. The same check is done as a separate unit tests.

Closed in favor of https://github.com/AntaresSimulatorTeam/AntaREST/pull/2133 which fixes naming issues


[#160176077] Fix bug with cached tags when restoring from snapshot

What?
The restore from snapshot feature relies on the LastOperation broker endpoint
that will be called periodically by the API. That would read/update the
tags of the DB instance to implement a short of state machine to execute
the required steps to restore an instance from snapshot[1]
In [2] we added some caching logic to avoid querying the instance tags in
every Describe() operation, but that broke the previous mentioned feature.
In this PR we add a feature to override the cache of tags by passing an
option to the Describe*() methods, and change the LastOperation handler
to use it.
We add a new integration test for this feature. This test has to run in a sequence an instance creation, snapshot, and another instance creation, so it adds some time to all the acceptance tests. Now it would last ~24 minutes.
[1] https://github.com/alphagov/paas-rds-broker/pull/35
[2] https://github.com/alphagov/paas-rds-broker/pull/83
How to review?

Code review
Check that the test pass

Integration test in verbose mode can be checked in: https://concourse.build.ci.cloudpipeline.digital/teams/main/pipelines/rds-broker/jobs/integration-test/builds/239, it failed because it had a focus.

Who?
Not @keymon

LGTM


Parse node selection from the URL query parameters

[x]  selected nodes are read from URL query parameters on App launch
[x]  user node selections are written to the URL query parameters
[x]  base outcome: I can copy/paste URL with node names and recreate node selection state

:rocket: Issue was released in v0.1.0 :rocket:


Update sql-database-vcore-resource-limits-single-databases.md

Adding SQL limits page

@dcbrown16 : Thanks for your contribution! The author, @CarlRabeler, has been notified to review your proposed change.

@dcbrown16 : Thanks for your contribution! The author, @CarlRabeler, has been notified to review your proposed change.

@sign-off

#sign-off


Other plots

The app currently is very much tailored to demonstrate the differences between the models, which is very useful and fits the paper very well. However other users (say, ICCI) will have interest in a few other plots which I think are relatively easy to add without jeopardizing the original intent of the app. A few ideas:

[x] When selecting one model in the main frame, the resulting plots in the "Basin information" box could show all SSPs for that model instead of one single line (Model for 1 currently selected SSP). This way a user can explore the influence of SSPs as well.
[x] If possible, it would be great to have the option to see relative changes as well (less focus on absolute values) -> useful for people less interested in inter-model differences and absolute values.

More longer term and out of scope, but of course OGGM (or pygem maybe?) have the monthly components and melt components available, which would be a fantastic addition one day. Something to think about, perhaps in a companion app focussed on OGGM output already.

Created and uplaoded new figures which have two panels. If using the "multi-model median" input the panels show annual runoff volume (panel #1) and percent change in runoff compared to historical mean (panel #2) for all three models and the selected SSP. If using a single glacier model as the input, panel #1 shows the annual runoff volume (for only that model as well as the selected SSP) while panel #2 shows projections for all 4 SSPs given by a 20-year rolling mean.


Add toChicago

Resolve #4

Hey @dungwinux , if you could rebase and your branch it would be awesome!

@vorillaz, I've done the rebase.

@dungwinux Merged, it will be released soon.


added text typing effect and animations

here i have added typing effect animation to make the tag line of Home Page even more appealing and also some color style changes are made
added relevant screenshots

@Anu27n sir i have raised a PR without conflicts and also attached screen shots
please review .


Integration with other models

Hello! Awesome work and quick question.
The available engines listed in the boilerplate SDK code are:
# Available engines: stable-diffusion-v1 stable-diffusion-v1-5 stable-diffusion-512-v2-0 stable-diffusion-768-v2-0
# stable-diffusion-512-v2-1 stable-diffusion-768-v2-1 stable-inpainting-v1-0 stable-inpainting-512-v2-0

Are there plans to allow simple integration of models build on Stable Diffusion? (for example: Protogen)

Hi there! @TimFinnigan Please accept our apology for the delay in response. At the moment we're only supporting the models that we have listed, if that changes however we'll be sure to note it on https://platform.stability.ai in an update.


Error displaying PDF converted to AFP.

Hello,
I am creating a converter from PDF files to AFP files. To do this, I have used examples that you have in the code and I have reviewed other issues that have asked you related questions.
However, I have a problem. Once I generate the AFP file, depending on the viewer I use, I get different results. If I use your viewer or papyrus software, they return the file correctly but others such the IBM Internter Explorer plugin returns a red background document or in the case of the printer software to be used to print the documents it gives an error.
I have been researching and I think it is related to the code "T1000395" as it is the error message displayed when parsing and validating the AFP document. I have tried to modify it and use other types of fonts based on an original document that they have given me and that they use to print, but it still gives the same error in those programs.
I appreciate your help and if you need more information I will explain you without problems.
Thank you.

Hi Jesus,
If you are just encapsulating PDF in AFP then you wouldn't need the code page T1000395 nor the font C1N20000 - you can just remove it (so basically the whole MCF and BPT, PTX and EPT), but most likely your problem is related to PDF object container. Older viewers like the IBM Internet Explorer plugin are not able to render PDF object container. Also some printers, esp. black and white printers are not able to print pdf object container. Most modern color (production) printer however have no problem with these files.

Hi Yan,
I give you a little more information about the problem we have with the AFP file. I have generated a new file removing both the encoding and the source as you said in your answer to my post. However, the printer software gives an error that an ENG structure was expected instead of the end-of-document EDT structure. To give you more feedback and see if you can help me, I am attaching the image with the error provided by the printer and an AFP file generated with the same program (the file itself is not the same since for confidentiality reasons I cannot provide it).

Thanks,
Greetings
Attached files.zip
.

Hello,
After carrying out the steps that you indicated in the previous one, the printing process of the AFP file has worked correctly.
Thank you.


kind of supervision

I would like to ask what kind of supervision the ThreaTrace algorithm is? Is it self-supervised?

I find the question interesting; I have already asked myself the same question, and my conclusion is that it depends on the perspective.
The authors themselves write that the unsupervised training mode of GraphSAGE is inappropriate for the task, and therefore, supervised training is used. In the discussion, the authors call the method unsupervised but describe it as semi-supervised.
Self-supervised fits as a description since the classification withholds information for learning derived from the original data.
Therefore:
A supervised setup is used in the training, but it gets by with information from the raw data, hence self-supervised training.
Regarding attack detection, the system manages without attack knowledge and can therefore be seen as Unsupervised Attack Detection.
I disagree least with Semi-Supervised (which is mentioned in the latest version of IEEE-Xplore), as I would think it learns with a few attack examples, which it does not.
Can you follow the reasoning? Comments and corrections are welcome!


vendor: bump grpc/protobuf

This bumps the grpc/protobuf dependencies.
Fixes: #3582

This is necessary to be up-to-date with the etcd protobuf/grpc dependencies.

This was reported by @timothysc on the k8s slack.

Fixes: https://github.com/coreos/rkt/issues/3582

The Debian failure is a known flake.

Would it be possible to patch this to v1.24 and tag a point release?
If not, what is the eta on 1.25.0?

1.25.0 ETA is 16th Feb, see https://github.com/coreos/rkt/milestone/59. We don't plan to have a 1.24.1 at this point.

Do folks have any concerns about vendoring in the current master api into kubernetes?


implement sku_tier for IAC-639

This adds sku_tier to variables.tf.  Choosing "Free" (the default) results in a cluster "optimized for availability".  Choosing "Paid" optimizes for availability (and cost more $).  Default behavior does not change.  Tested and working

tflint caught a typo.  will get that cleaned up.


selection sort added

Hi @Spymik, Please first click on the problem which one you want to solve and that link redirect you to codeforces,
and submit your code there and when got your solution accepted there and then commit your code.
Thank You!


⚠ BCR Bot has degraded performance

In 136445c, BCR Bot (https://bcr.alternativasinteligentes.com/BCRApi/web/chatbot) experienced degraded performance:

HTTP code: 200
Response time: 1276 ms

Resolved: BCR Bot performance has improved in dca1a4f.


page/logo

PR

Types of changes

[x] New feature

Description

Checklist

[x] update cases
[x] added new cases

testing

testing


Can we build the app to run in android?

Does Flet supports running in android?

#78

@lemon37564 Thank you


Stop camera

This library does not have function to show preview of the snapshot taken, is there any way we can do that

Hi! What about this issue?
I want to stop camera after picture was taken and see a preview.

Hi! What about this issue?
I want to stop camera after picture was taken and see a picture preview instead of a running camera.

#158
it has been fixed.


Debug mode

Work in progress...
Description
Debug mode to debug Fragments processing.
Motivation and Context
Help developers to understand what happens during Fragment processing.
Screenshots (if appropriate)

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[x] My code follows the code style of this project.
[x] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.

I hereby agree to the terms of the Knot.x Contributor License Agreement.

Replaced by updated #39


Exit our of video

How do you exit out of video?

@robertjones1 we are not getting your question. From where do you want to exit the video? You can take the back button or anything like this. you can pop as well.

Closing this issue due to no reply. Reopen if require with proper details. Thanks.


Telescope upgrade

Fixed all the comments, casing, and issues from before, should be in better shape!

[ ] do we need to include the ICA types from https://github.com/confio/cosmjs-types/issues/36

do we need to include the ICA types from https://github.com/confio/cosmjs-types/issues/36

Yeah, let's include them. You find them in wasmd-0.28/third_party/proto/ibc/applications/interchain_accounts now

the check is giving:
src/cosmos/tx/v1beta1/service.ts:924:14 - error TS7056: The inferred type of this node exceeds the maximum length the compiler will serialize. An explicit type annotation is needed.

I think I know how to solve this, but will make a large diff.

the check is giving:
src/cosmos/tx/v1beta1/service.ts:924:14 - error TS7056: The inferred type of this node exceeds the maximum length the compiler will serialize. An explicit type annotation is needed.

I think I know how to solve this, but will make a large diff.

https://app.circleci.com/pipelines/github/confio/cosmjs-types/123/workflows/569fd5f5-81d4-49c0-b0ec-27d44cbead66/jobs/115
btw — we can try doing this for just the google module if you want to keep the Exact<DeepPartial business. The plugin system should support that.

OK, I think that may be it!
Only difference should be Exact vs DeepPartial, etc., but this one works and is much faster for the TS compiler to deal with.
We can roll back the Exact/DeepPartial stuff, but then the CI can't process things because of TS compiler memory.

wow this marks the end of an era! and a new beginning. Excited to start improving things like DecCoin and Long 🙌🏻


Implement string type output parameters for queries

Please implement Pod and other required impl's for string CDatatypes Char and WChar. There are outputs for numeric types, but none for string-based types.

Hello @TylerJMcLean ,
thing is a String is not Pod. These are variadic types, which means we do not know how big they are in memory. Best way to fetch them is using a columnar buffer. See the documentation on how to fetch values: https://docs.rs/odbc-api/latest/odbc_api/buffers/struct.ColumnarBuffer.html

If you want to fetch your data row wise, and know your table schema at compile time, you can also use RowVec and use the derive feature.
See: https://docs.rs/odbc-api-derive/latest/odbc_api_derive/derive.Fetch.html

Hey @pacman82, I ended up figuring that out after another half an hour of digging through the project and meant to come back to close this.
It was not clear to me while reading through the documentation that I could do something like the following with out parameters to obtain a string
let mut out_msg: VarWCharArray<21> = VarWCharArray::NULL;

connection.execute("CALL PROCEDURE_NAME(?)", (Out(&mut out_msg),))?;

as the only example that was available used Nullable's exclusively here. Perhaps the example should be updated to reflect this? Regardless I will close this ticket as my issue is resolved. Thanks for the quick reply!

Hello @TylerJMcLean ,
I am happy you did not close the issue immediately. Improving can bring just as much value as improving the code. I clarified the documentation of Out and added an example to VarCharArray.
Best, Markus


lets use the promtool to replace the python querier.

lets build a docker image that uses the promtool to send queries to the tested Prometheus servers

This is the Python querier that we want to replace with the Promtool
and here is the manifest how it is deployed

Hmm, just curious. The Python querier also exposes metrics about the query
latencies. How do you propose to do that using just promtool?
Further if we're going to build a wrapper around promtool for it, we're
better off with the Python querier no?
Thanks
Goutham
On Fri, Mar 22, 2019, 1:41 AM Hrishikesh Barman<EMAIL_ADDRESS>wrote:

continuing working on this one.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/prometheus/prombench/issues/125#issuecomment-475384409,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHA3H1BrtnvKcxXo-pfsJ9YWbccCbuZ-ks5vY-eHgaJpZM4V1HAn
.

metrics in the python querier:
1 - loadgen_query_duration_seconds
2 - loadgen_queries_total
3 - loadgen_failed_queries_total

I am not very happy with what I thought of but was thinking about starting with replacing L116 with async subprocess call to promtool.

loadgen_queries_total it's just a increment every time promtool is called
loadgen_failed_queries_total can be set using return codes, if we need more granularity on return codes we can change how promtool behaves.
loadgen_query_duration_seconds just calculate this once the async process finishes.

https://github.com/prometheus/prombench/blob/93113c77da24f189ff58139e4df01f1cfebd4b9e/cmd/load-generator/main.py#L116-L129
If we don't want to use subprocess, I assume we can write a go implementation and import the promtool package but that will be using the HTTP API anyway.
What do you suggest?
cc @gouthamve

Or we could extract querier out as a separate tool like scalar and also expose the /metrics endpoint from there.
Where are the loadgen_* metrics used? grepping the metrics just shows the declarations.

I looked at the promtool code quickly and @gouthamve 	 is right using the promtool wouldn't make any sense here.
It seems that re-implementing the python querier in golang is the way to go.
The main benefit for this is that most Prom devs are already familiar with golang so can contribute easily.

Yes, that'd be a better approach. I'll follow that.


casm-4232-iuf-logging

Summary and Scope
This change is to support enhanced logging for scripts used by IUF operations. The "loaders.py" script is used by the "ims-upload-content" operation of the "deliver-product" stage of IUF. The changes introduce log levels to each statement that logs information for user.
Issues and Related PRs
None
Testing
Planned for test in #mug .
Tested on:
Test description:
Run the "deliver-product" stage for the "COS" product to verify.
Risks and Mitigations
None

I am confused by the nature of the changes. The changelog text and the PR description state that this is meant to enhance the logging. However, almost no new logging is added. In some cases the log message is changed in a tiny way (removing the final period from the output). In all of the other cases, the logging is changed from "info" level to "debug" level, which means it will show up LESS often in the logs (since DEBUG is a more detailed logging level than INFO). In a couple of instances the log message is improved to include the filename (where it currently does not), but it also changes the log level from info to debug, meaning that the message itself will show up only if debug logging is enabled.
In other words, the effect of this PR will only SLIGHTLY increase the information being logged if debug logging is already enabled (in which case you will get those filenames in the places where previously you did not). Otherwise you will get LESS information (if a log level less than DEBUG is set). Is this the desired behavior? Is there a reason we want these log messages to be rarer?
The only part of this PR which makes sense to me is adding the filename to the messages where it is absent. I don't see why the info messages are changed to debug.

Hi @mharding-hpe. I completely get where you are coming from. So these changes are made majorly from the IUF perspective. There are so many images that IUF uses in various stages. So when an IUF command is run, the log messages comes from various places - IUF itself, NLS and from these images that IUF calls. So from the user perspective its a bit unclear of actually what is happening. User would be getting loads of messages which might not be quite information for them. That is the reason we planned to change most of the messages which are not much critical to 'DEBUG'(which would be going into the logs in IUF) and leave critical messages alone in 'INFO' which would appear on screen when IUF commands are run so it will be less cluttered. Hope this answers your queries.


itemcreation_ui_init gets altered when image is rebased

Something alters the code, resulting in garbage code / error.
This happens in "vanilla" too (Moebius patch).

Hey shai, is this still current? Or is this fixed by replacing the Moebius stuff we already replaced?

Nope
I've also found this in the Moebius DLL notes:
"If you're feeling really adventurous, you can go back and figure out what sorts of relocs
you need, and edit the .reloc segment in temple.dll.  I've done this, but I'm not sure if
my edits are correct, as temple.dll is always loaded at it's preferred base, and no
relocations are necessary."
Maybe he just made an error there...

Replaced the itemcreation init


[Subtask] Separate FilesetCatalog, TableCatalog for connector and client

Describe the subtask
The server side and client side need different API for FilesetCatalog, TableCatalog; so will separate today's one interface class to two interface classes;
Parent issue
#https://github.com/datastrato/gravitino/issues/3626

Now I think the interface of FilesetCatalog, TableCatalog are clear, can be used for both server side and client sider, especially after we make the "NameIdentifier" class as a generic object. So close this issue for now.


Began Tracking/Finished Tracking Callbacks + Improved Thumb Tracking

Thanks, but I think isn't necessary

Ok. I understand. I do think the improved thumb tracking is very important, however, as I found it very frustrating to have to click directly on the ThumbLayer to activate it. This implementation allows the user to click anywhere on the track to activate the nearest ThumbLayer.


Hoyolab check in?

Does Hoyoverse remove the Hoyolab check in API? I still use genshinstats but when I use Hoyolab check in it throw an error genshinstats.errors.GenshinStatsException: Internal database error, see original message, did they remove it now?

May I see the original message? (error.original)
I am unable to test this for myself because I have already claimed the daily reward.

May I see the original message? (error.original / repr(error))
I am unable to test this for myself because I have already claimed the daily reward.

The error says HoYoLAB is currently carrying out server migration. This function is not available at the moment~ screenshot

Daily check-in seems to be working just fine for me. Maybe the migration has already finished?

Daily check-in seems to be working just fine for me. Maybe the migration has already finished?

It's not finished yet, see hoyolab_check_in() function from genshinstats library this function doesn't work, I'm not using genshin.py yet

Oh, that one. I thought I got rid of that one completely, I'm surprised the function still exists.
Are you asking me to look into and potentially fix the genshinstats function or to implement it in genshin.py?

Just implement it in genshin.py I think, but that API still not work 🤔 just keep update about that API and implement it in genshin.py will be a pleasure.


Properties that may be overridable as methods.

Hello,
I'm dealing with an issue specific to TypeScript on a Backbone project, but this seems more generic to any project using underscore/lodash and the return of the _.result method:
// With _.result you can work with an object property or function...
return _.isFunction(value) ? value.call(object) : value;

Is it possible to have a TypeScript interface definition that includes an instance property that can optionally be overridden with an instance method?  Example from Backbone's .d.ts file to include both:
url: any
url(): any

Currently this produces an error for a duplicate identifier.
Thanks!
-Eric

You can write  x: string|(() => string) to mean "a string or a function returning a string"

Excellent - that was just what I was looking for.  Thank you!

Reopening for one more question - is it possible to do the same thing in a class definition or will the base class require a more concrete definition of the type (e.g. "string vs. function returning a string")?

Should be a few ways you could approach it. One example:
declare var _: any;

class Foo {
	url: string | (() => string);
	constructor(value, object) {
		this.url = _.isFunction(value) ? value.call(object) : value;
	}
}

var f = new Foo(1, 2);
var r = f.url; // need to copy to a local for type guards to work
if (r instanceof Function) {
	r();
} else if (typeof r === "string") {
	var s = r;
}

Thanks @danquirk - the issue appears to be in classes that extends another class with a union type defined this way.  Here's an example I just threw up:
http://www.typescriptlang.org/Playground#src=class Foo {
	bar%3A string | (() %3D> string)%3B%0A%09bat()%20%7B%0A%09%09let%20bar%20%3D%20this.bar%3B%0A%09%09if%20(bar%20instanceof%20Function)%20%7B%0A%09%09%09bar()%3B%0A%09%09%7D%0A%09%09else%20if%20(typeof%20bar%20%3D%3D%3D%20%22string%22)%20%7B%0A%09%09%09let%20b%20%3D%20bar%3B%0A%09%09%09b%20%3D%20%22b%22%3B%0A%09%09%7D%0A%09%7D%0A%7D%0A%0Aclass%20MyFoo%20extends%20Foo%20%7B%0A%09bar()%3A%20string%20%7B%0A%09%09return%20%22bar%22%3B%0A%09%7D%0A%7D%0A
As long as I create that local copy the type guards work, but the very nature of extends means we don't create that local copy before the property/method definition.  It seems like the compiler should work within the realm of check if the definition satisfies one of the types in the union, and usage of the property/method would still require local copying.
Does that make sense or seem like a reasonable request?
Thanks,
Eric

If you define bar as an instance member in MyFoo it will work:
class MyFoo extends Foo {
	bar = () => {
		return "bar";
	}
}

likewise using an interface:
interface hasURL {
	url: string | (() => string);
}

class Bar implements hasURL {
	constructor(public configured: boolean) { }
	url = () => '' // or could be url = ''
}

Looks like this issue has been addressed. please reopen if there more questions.


Improved deno run --watch support, watch support inside Docker

Description
This PR improves support for running the connector using deno run --watch where the connector gets restarted by Deno whenever the function files change. It adds support for running the connector in watch mode using Docker.
docker build . -t "typescript-deno:latest" # only because we don't publish a docker container anywhere
docker run -it --rm -p 8080:8080 -e WATCH=1 -v ./functions:/functions/src typescript-deno:latest

Part of the improvements to make watch mode work better include

rearranging when we infer the functions schema to reduce work done, but also reduce watch-induced reloads caused by vendoring during inference
rearranging the dockerfile and entrypoint script so that the generated vendor, node_modules and schema.json artifacts are stored outside of the functions source directory, so that we don't pollute the user's volume mount

Various version updates have also been made:

Version of deno used in Docker has been updated
Deno github action has been updated
the deno std library version has been updated and made consistent across the entire project

Solution Design
We now have a difference between RawConfiguration and Configuration. We perform our inference only in validate_raw_configuration and capture the results in the Configuration. This then powers every usage of the inference, including the schema endpoint and actual queries. It also provides a central place for configuration defaulting (such as defaulting to "INFER").
The entrypoint script now looks for a WATCH env var and if it is "1" or "true" it generates a different configuration that enables INFER mode and turns on deno's --watch flag.

Things to test:

CI published docker images
Use of these images from Sendgrid example project
Connector create workflow
NPM Deps
Deno Deps
Local file deps

It would be good if we can introduce some automated / e2e CI testing for these categories at some point - extending the current tests beyond inference. But we can just manually test for this PR.

Should probably have a better error if the config is missing the functions:
> deno run -A --watch --check ~/hasura/ndc-typescript-deno/src/mod.ts serve --configuration config.json
Watcher Process started.
Check file:///Users/lyndon/hasura/ndc-typescript-deno/src/mod.ts
TypeError: Cannot read properties of undefined (reading 'trim')

Weird. I thought the SDK would prevent it from getting that far in the code :/ The types are a lie.

Fatal errors seem to kill the watch command in some instances:
Watcher File change detected! Restarting!
Inferring schema with map location /Users/lyndon/temp/temp-2023-11-29_14-33-24-rtYNSYAuhX/vendor
Vendoring dependencies: /Users/lyndon/bin/binaries/deno vendor --node-modules-dir --output /Users/lyndon/temp/temp-2023-11-29_14-33-24-rtYNSYAuhX/vendor --force /Users/lyndon/temp/temp-2023-11-29_14-33-24-rtYNSYAuhX/functions/index.ts
Couldn't find import map: /Users/lyndon/temp/temp-2023-11-29_14-33-24-rtYNSYAuhX/vendor/import_map.json
There were 1 diagnostic errors.
FATAL: /Users/lyndon/temp/temp-2023-11-29_14-33-24-rtYNSYAuhX/functions/index.ts (10,30): The return type of an async function or method must be the global Promise<T> type. Did you mean to write 'Promise<string>'?
Fatal errors: 1
Previous comand exited with error status: 1

Seem to be having issue with local deps when running directly:
lyndon@spaceship 2023-11-29 16:25:39 ~/t/temp-2023-11-29_14-33-24-rtYNSYAuhX master ✓
> deno run -A --watch --check ~/hasura/ndc-typescript-deno/src/mod.ts serve --configuration config.json
Watcher Process started.
Inferring schema with map location /Users/lyndon/temp/temp-2023-11-29_14-33-24-rtYNSYAuhX/vendor
Vendoring dependencies: /Users/lyndon/bin/binaries/deno vendor --node-modules-dir --output /Users/lyndon/temp/temp-2023-11-29_14-33-24-rtYNSYAuhX/vendor --force /Users/lyndon/temp/temp-2023-11-29_14-33-24-rtYNSYAuhX/functions/index.ts
Error: Got code 1 during deno vendor operation.
stdout:
stderr: error: Module not found "file:///Users/lyndon/temp/temp-2023-11-29_14-33-24-rtYNSYAuhX/functions/lol.ts".
    at file:///Users/lyndon/temp/temp-2023-11-29_14-33-24-rtYNSYAuhX/functions/index.ts:2:22

pre_vendor failed
Previous comand exited with error status: 1

There's not much point having the two versions:

programInfo
programInfoException

after making programInfo just propagate exceptions rather than exit.


Blogs not displaying mobile site in Jekyll

check this in any browser. It is not displaying blogs other than first and second.

Submitted a PR for this, you had overflow hidden in the css which stopped all the blog posts from showing.


Client Retry Policy: Handle HTTP timeouts with request-level cross-region retry

Reference: https://github.com/Azure/azure-cosmos-dotnet-v3/issues/3467
After retries have been exhausted, the CosmosHttpClientCore component should throw a 503 with the HTTP timeout as inner exception.
The 503 should reach the ClientRetryPolicy and be handled like 503s are right now: https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos/src/ClientRetryPolicy.cs#L250
Identify which HttpTimeoutPolicy is used for each of the cases:

Data Plane
Metadata
Query Plan
Address requests

For the cases described in the referenced table (pasting here for easy access), we should convert the last timeout into a 503 with the timeout as an InnerException:

Type
Read/Write

Data plane
Read

Data plane
Write

Metadata
Read

Query Plan
Read (it is always read)

Tasks

[ ] Complete use case scenarios
[ ] Sequence diagram
[ ] Presentation and design discussion
[ ] Define deliverables, delivery date and project milestone

Use case scenarios

Data plan in Gateway mode for Reads, Request Timeout

Given a CX wants to read an existing item

When I configure my CosmosClient.ConnectionMode for Gateway mode (which uses an HTTP protocol)

- And it will send the operational request using SendHttpHelperAsync

- And the retries will throw a 408 (Request Timeout) exception

- And the retries have exhausted to its limits

Then the 408 (Request Timeout) exception is converted to a 503 (Service Unavailable) exception

Data plan in Gateway mode for Reads, Not Found

Given a CX wants to read an item that does not exist

When I configure my CosmosClient.ConnectionMode for Gateway mode (which uses an HTTP protocol)

- And it will send the operational request using SendHttpHelperAsync

Then the 404 (Not Found) exception is thrown

Data plan in Gateway mode for Writes, Request Timeout

Given a CX wants to write a new item (should test all of the following: CREATE, DELETE, REPLACE, UPSERT, PATCH, etc.)

When I configure my CosmosClient.ConnectionMode for Gateway mode (which uses an HTTP protocol)

- And it will send the operational request using SendHttpHelperAsync

- And the retries will throw a 408 (Request Timeout) exception

- And the retries have exhausted to its limits

Then the 408 (Request Timeout) exception is converted to a 503 (Service Unavailable) exception

Data plan in Gateway mode for Writes, Conflict

Given a CX wants to create an item that already exist

When I configure my CosmosClient.ConnectionMode for Gateway mode (which uses an HTTP protocol)

- And it will send the operational request using SendHttpHelperAsync

Then the 409 (Not Found) exception is thrown

Data plan in Gateway mode for Reads, Request Timeout

Given a CX wants to read an existing item

When I configure my CosmosClient.ConnectionMode for Gateway mode (which uses an HTTP protocol)

- And it will send the operational request using SendHttpHelperAsync

- And the retries will throw a 408 (Request Timeout) exception

- And the retries have exhausted to its limits

Then the 408 (Request Timeout) exception is converted to a 503 (Service Unavailable) exception

Clarifying questions

Do we need to be concerned with sub status codes?
Just need to clarify that we are not including a solution for Direct Mode.

https://github.com/Azure/azure-cosmos-dotnet-v3/issues/3467

How to exhaust all retries so that it returns a 408 (Request Timeout) exception?
What is the best means for testing a Metadata Read operation request?
What kind of exceptions, other than a 408 (Request Timeout), could occur with a Metadata Read?
What is the best means for testing a Query Plan operation request?
What kind of exceptions, other than a 408 (Request Timeout), could occur with a Query Plan?
Should we also be testing this for Change Feed iterator/estimator?

This would mean adapting the HttpTimeoutPolicy interface to know which cases should throw the 503 or which should throw the normal timeout: https://github.com/Azure/azure-cosmos-dotnet-v3/blob/1eaae654845f83faf3283ee3be53d8d2c31e5256/Microsoft.Azure.Cosmos/src/HttpClient/CosmosHttpClientCore.cs#L295

Background
Right now this is the current flow of operations for HTTP operations in gateway mode when they timeout.
sequenceDiagram
    participant A as Operation
    participant B as ClientRetryPolicy
 [Cross region retry]
    participant C as TransportHandler
    participant D as HttpTimeoutPolicy
    participant E as CosmosHttpClientCore
    A->>B: 1. Send HTTP operation
    B->>C:
    C->>D: Wraps the call around retry layer
    D->>E:
    E-->>D: OperationCanceledException
    D-->>E: Retries HTTP timeouts on same endpoint
    D-->>D: Repeats until retries exhausted
    D->>C: OperationCanceledException
    C->>B: CosmosException(408)
    B->>A:

Why we need to change this
Currently, the SDKs don't factor HTTP timeouts in their cross-regional retry policies. TCP timeouts are covered but HTTP timeouts are only retried locally and then surface to the user.
Because HTTP timeouts do not have any cross-region failover semantics, SDKs could not attempt to recover by rerouting requests to other available regions. Some HTTP operations can be performed on other regions and therefore should be retried.
Here is a list of the different HTTP operations that the SDKs preforms.

Type
Read/Write
Multi Master
Can be retried?

Data plane
Read
No
Yes

Data plane
Write
No
No

Data plane
Read
Yes
Yes

Data plane
Write
Yes
Yes

Metadata
Read
No
Yes

Metadata
Write
No
No

Metadata
Read
Yes
Yes

Metadata
Write
Yes
No

Query Plan
Read
No
Yes

Query Plan
Read
Yes
Yes

Addresses
Read
No
No

Addresses
Read
Yes
No

So, out of these operations the ones that are retriable are the following.

Type
Read/Write

Data plane
Read

Data plane
Write

Metadata
Read

Query Plan
Read (it is always read)

What needs to change?
Looking at the current flow we can see that each HTTP operation is assigned a HttpTimeoutPolicy which dictates its behavior on a timeout. So we need a way to tell the timeout policy if it is one of these types of HTTP operations that can be retried with a 503 status code it should.
If we follow the same pattern that we have in the Direct mode (retry locally, if retries are exhausted, throw 503), if the HttpTimeoutPolicy converted the HTTP timeout to 503, then it would kickoff the retry on another region. Because each HttpTimeoutPolicy is aware of the type of operation upon initiation, then it can conditionally decide whether to make the timeout a 503 or not if one were to occur.
HTTP Timeout Policies
There are 4 different types of HTTP Timeout Policies so we need to look at what they are used for and see if they need to change.
NoRetry
This policy is used for HTTP requests that should not be retried. Because it should never be retried it does not need to change.
ControlPlaneRead
Here because all control plane read requests are meta data read requests, we should make sure that all Control Plane Read timeout policies should throw a 503 on timeout.
ControlPlaneRetriableHotPath
Control Plane Retriable Hot Path policies are used in three different places.

Partition Key Requests - which should not be retried.
Address Requests - which should not be retried.
Query Plan Requests - which should be retried.

Default
All other cases use the Default HTTP Timeout Policy, including all data plane requests and some - but not all - meta data read/write requests.
So of the 4 timeout policies that exist three need to have the ability to communicate that they should throw 503 errors on timeout. ControlPlaneRead, where it should always do that, and ControlPlaneRetriableHotPath + Default where we need to be flexible on when it can throw a 503 error on timeout.
Where are the Timeout Policies called?
ControlPlaneRead

Tests
GatewyaStoreClient
GatewayAccountReader

All of these methods should retry on a timeout.
ControlPlaneRetriableHotPath

Tests
GatewayAddressCache
HttpTimeoutPolcy GetTimeoutPolicy method.

The tests the cases are mixed what ones should be retried. In the GatewayAddressCache they are address operations and therefore should not be retried. In the GetTimeoutPolicy method, only the case for the Query Plan operations should be able to retry.
Default

Tests
HttpTimeoutPolcy GetTimeoutPolicy method.

In the tests cases are mixed on which ones should be retired. In the GetTimeoutPolicy method, only meta data reads and data plane operations should be retried.
So looking at where each of these timeout polices GetTimeoutPolicy method is the main method where it is decided which timeout policy is used. Right now it is really only deciding between the Default and ControlPlaneRetriableHotPath. The other policies are specified directly in the methods where the HTTP requests is made.
Proposed Solution
Have two different instances  of each client in the construer with a flag that tells whether it should throw a 503 error on timeout or not. Then in the GetTimeoutPolicy method use the document service request to determine where it falls under the umbrella of the scenarios where we want to throw the 503 errors. By setting the flag on assignment of the policy, we can access this information anywhere in the flow of the HTTP gateway request.
So the new flow would be this:
sequenceDiagram
    participant A as Operation
    participant B as ClientRetryPolicy
 [Cross region retry]
    participant C as TransportHandler
    participant D as HttpTimeoutPolicy
    participant E as CosmosHttpClientCore
    A->>B: 1. Send HTTP operation
    B->>C:
    C->>D: Wraps the call around retry layer
    D->>E:
    E-->>D: OperationCanceledException
    D-->>E: Retries HTTP timeouts on same endpoint
    D-->>D: Repeats until retries exhausted
    D->>C: OperationCanceledException
    D-->>E: ShouldThrow503OnTimeout
    E-->>D: ServiceUnavailableException
    D->>C: ServiceUnavailableException
    C->>B: CosmosException(408)
    C->>B: CosmosException(503)
    B->>A:

Possible Downsides
Right now, if we want to add any other types of errors more flags would need to be added and more instances.


Used encoding with XmlPoke

Please review. Fix to #1889

@Marusyk,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by .NET Foundation. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot

@marcosnz your changes have been merged, thanks for your contribution 👍


fix node path.

We don't need absolute path when using nvm node package management.

Thanks. This is consistent with what we are doing on newer examples.


Allow updating the entire key value store at once

There is an issue I run into with having a typed context.
Say I have a type like:
interface Lead {
    Status: "NONEXISTANT";
} |
{
    Status: "ACTIVE",
    Details: {
    	SomeField: number,
    	SomeOtherField: string
	};
}

And my context looks like:
ctx: restate.ObjectContext<Lead>

Currently there is no typesafe way for me to update the key value store. This is annoying when trying to implement a state-machine pattern for the key value store.
Example:
ctx.set("Status", "ACTIVE"); // Context is now in non-typesafe state because Details doesn't exist
ctx.set("Details", ...) // Get a type error because Details field isn't present when Status is "NONEXISTANT"

If we were to introduce some method to update the entire key/value state at once, we can resolve this. It would be akin to a map operation.
await ctx.update((oldState) => {
	return {
		Status: "ACTIVE",
		Details: {
			SomeField: 1,
			SomeOtherField: "hello"
		}
	}
} // All fields are updated atomically and type safety is preserved

Could you not achieve the same thing with the following:
interface State {
    Lead: {
        Status: "NONEXISTANT"
    } |
    {
        Status: "ACTIVE"
        Details: {
    	    SomeField: number
    	    SomeOtherField: string
	}
    }
}

const handler = async (ctx: restate.ObjectContext<State>) => {
    const oldState = await ctx.get("Lead") // typed: Lead | null

    await ctx.set("Lead", {
        Status: "ACTIVE",
        Details: {
            SomeField: 1,
            SomeOtherField: "hello",
        },
    })
}


Dash nlp - issue #346

Issue for app: #346
App pull request

[ X] I am improving an existing app (redesigns/code "makeovers")

About

Playground deployment URL (new version): https://dash-playground.plotly.host/dash-nlp/
Current gallery app URL:

@oxananu  We are experiencing a memory issue with playground so circle-ci deployment errors you saw for https://github.com/plotly/dash-sample-apps/pull/350/commits/11c9f59b4a21950ff438edec6213f71db47922c8 and https://github.com/plotly/dash-sample-apps/pull/350/commits/8350feeb5ad31c068e0d659bef7935b5a8457aaf is not caused by app itself. I just restarted the job and it went through successfully.

@ycaokris Feel free to merge whenever is good for you. 🙏

#360


Modal not capturing focus when autoFocus and keyboard are set to true

Issue description

components: Modal
reactstrap version #5.0.0-beta
import method es
react version #16.2.0
bootstrap version #4.0.0

What is happening?
'Escape' key not working so modal cannot be closed via 'Escape'
What should be happening?
Having 'autoFocus' and 'keyboard' props set to true, I should have an ability to close modal with 'Escape' button
Code
<Modal
  isOpen={isOpen}
  fade={false}
  toggle={this.toggle}
  className={modalClassname}
>
  <ModalBody className={bodyClassName}>
    {children}
  </ModalBody>
</Modal>

My workaround
Overriding onOpened method in modal, so I'm focusing manually (and it works!):
export class ModalExtended extends Modal {
  onOpened() {
    super.onOpened();
    const { autoFocus, keyboard } = this.props;
    if (autoFocus && keyboard) {
      if (this._dialog && this._dialog.parentNode) {
        this._dialog.parentNode.focus();
      }
    }
  }
}

Additional details
When debugging this, I've found that this._dialog is null in renderIntoSubtree (renderModalDialog is called after, so ref is set later than renderIntoSubtree is called):
// check if modal should receive focus
      if (this._focus) {
        if (this._dialog // is null here.

++

Can you try reactstrap 5.0.0-beta.2 version?

@fantua Fixed with 5.0.0-beta.3, Thanks Igor!


Unable to edit the Query Text Box

A BIG Thank you for this btw. I can send queries with the msg.payload, but am unable to type them in the query textbox.

Thanks for raising and @martin-doyle for fixing the bug. It has now been pushed to npm v0.0.5


Release example bmi on PyPI

If this example implementation is available on PyPI it can more easily be used in, for example;

tests in other packages that build upon BMI*
JupyterLite notebooks (see https://github.com/csdms/bmi-python/issues/60: don't forget to publish wheels too).

*in grpc4bmi we need an extra requirements file because this gh repo cannot be refered to in the pyproject.toml file.

Sounds like a good idea to me.
Maybe we could even include the C, and C++ (and, possibly, fortran) examples in the distribution.

Maybe we could even include the C, and C++ (and, possibly, fortran) examples in the distribution on PyPI.

The downside of that would be that you need to compile that per platform & per architecture. Just publishing the Python example case is a lot simpler (and will work everywhere).

The downside of that would be that you need to compile that per platform & per architecture

For sure. We do that with some other projects and it is a bit of a pain, but not too bad. I'm not sure if it would be worth it in this case, though.

We'd build conda packages for all the language examples + a PyPI package for the Python example.


Add mandatory pit info

Split from #14.
Add mandatory pit requirement & completion per driver (i.e. DataCorePlugin.GameRawData.Graphics.missingMandatoryPits & DataCorePlugin.GameRawData.Graphics.MandatoryPitDone).

Some notes on this.
We only have MandatoryPitDone info for the player car but not for everyone else, thus we would need to figure out if a car served mandatory pit ourselves, eg we need to figure out what a car did in the pits.
There are three aspects of mandatory pitstops: pit window, driver change and tyre change. But we don't actually have any information about which of these are required at a given session. Now assuming that we would have that data there are still some issues. I think we can figure out first two parts of the mandatory pit requirements. We do have pit window time and length from the shared memory data, so we can figure that part out. Similarly we do know which driver is in the car. However if the car actually changed the tyres is more tricky.
We do know if the car is in the pitlane and if it served a penalty (to be more precise we receive a message if the penalty was cleared). From the car's speed we can figure out when it's stopped, but we don't know if it is actually stopped in it's pitbox or not (this info is only available for the player car), which is another issue. We also know that a tyre change takes 30s, so if the pitstop is shorter than than, we know there couldn't have been a tyre change but we if a car did some repairs it can have a pitstop longer than 30s without changing tyres and thus we cannot be certain.
So in summary I think this would become feasible if we get some extra data from ACC, either directly the number of served (or left to be served) mandatory pitstops or which components are required and what the car actually did in the pits.


feat: Add dialog prompt for download filename

Example output

Thanks @JacksonIsaac. Unfortunately, I have to close this since this feature needs to be discussed and agreed upon first. Then we will create a GH issue with a detailed description to make it ready to be worked on. If you would like to contribute, please pick one of our existing GH issues.

Ah, that's indeed unfortunate. Who needs to discuss, and how long does it take to become a GH Issue ? Still trying to make it within the next few days for hacktoberfest :D
But definitely will keep contributing small fixes around later.


Move subrepository to C2SM

In order to not depend on a private persons' repository, the fork https://github.com/dominichofer/spack/tree/f0fcd2734a3a0d82a1a0f69fef7b8cc301ae48a8 needs a copy in C2SM and the pointer of this repo's subrepo needs redirection.

solved in #909


PrimitiveBase

This is more like an heads up than an issue that would fit better into the proposed discussion forum than here, but I'll bring it up anyway.
With one of my students I'm ramping up a project where we will set up a BAG layout generators to use vendor specific device generators and the build kind of a version of AnalogBase around that. This simplifies  the process definition and porting the designs from one process to another. Device generators is the easiest way to provide DRC clean and relatively well characterized devices, and current process setup basically requires re-write of those generators.
Naturally there is a need for hacking possibility too, but device generator based process definition would in my opinion lower the hreshold to take BAG into use. Let me know if you share interest to  do this.

try to re-define _make_master by adding pcell will do, also parse pcell layout info in tech params

We have managed to implement this just adding primitives instead of layers in draw_mos method in tech class. Unfortunately we can not publish the result (yet) as it is not implemented with gpdk. (Ping @sporrasm ) . Anyway, this does not require changes in AnalogBase, which is good as it creates less variation in methodology.

@AmrMohamedRashed Working on it. I'm kind of stuck redefining guard rings, but I've had my plate quite full, so maybe during the summer holidays... :)

@mkosunen I wish you the best of luck.
I guess I have managed to map the BAG to work with gpdk045 if you faced any issue I might help but I still have to do some work first to be sure.
have a nice summer vacation.


passing sub_id via vestaboard.Board() is broken

Describe the bug
While attempting to initialize using just vestaboard.Board() and passing in Key, Secret, and SubID, I receive an error.
Calling the installable, and then passing that into the Board() works.
Works:
# Vestaboard
vesta_installable = vestaboard.Installable(VEST_API_KEY, VEST_API_SECRET)
vboard = vestaboard.Board(vesta_installable)

# output
>>> vesta_installable
<vestaboard.Installable object at 0x000002B57EC26970>

Breaks:
# Vestaboard
vboard = vestaboard.Board(VEST_API_KEY, VEST_API_SECRET, VEST_SUB_ID)

Exception has occurred: AttributeError
'str' object has no attribute 'apiKey'
  File "init.py", line 40, in <module>
    vboard = vestaboard.Board(VEST_API_KEY, VEST_API_SECRET, VEST_SUB_ID)

Details
Using VSCode Terminal on windows in a venv w/ Python 3

Hey thanks for this! Will take a look and see what I can spot. In a previous version we had the opposite of this problem where the line could not end with a bracketed-character code, so this is likely something similar.

Okay I dug into this a bit! The issue is related to the arguments passed in.
In your code above, you're passing in three parameters. The signature for the board class allows this, but the first argument Board() is expecting is an instance of Installable:

In your case you're passing the your API Key as the first argument. The Board constructor is trying to get access to the Installable's .apiKey property, but it's not an Installable, it's a string, thus the 'str' object has no attribute 'apiKey'.
I will add a check/warning when a string is passed in instead of an Installable instance! In the meantime, you can fix this one of two ways:

Specify the argument names, or
Since Installable is optional, you can pass in None as the first argument


CLDC-3196 Update prepare file and guidance pages for 24/25 bulk upload

This seems to all have been covered by CLDC-3207 already


Use of SECRET_KEY

I just came across this project and am wondering how the SECRET_KEY environment variable is used. I did a search for secret_key in the repo and could see it being assigned but no access to that property afterwards.
In my understanding, the keys are generated on the fly for each secret and included in the link, so there shouldn't be a need for a global secret? Except maybe for adding an additional HMAC at the end - but I couldn't come up with a scenario where this would actually help here.
Although I see that you do the decryption on the server and then render the decrypted secret, while you could do HMAC verification, my understand is that an HMAC would only make sense if an attacker can modify the encrypted data.
Incidentally I built a very similar solution a while ago: https://github.com/milgner/keedrop/ :smile:
There I decided to do all the cryptography in the browser so that the server operator never has a chance to see the encoded data. HMAC verification wouldn't work as the secret is already gone from the server at the time of decryption.
But maybe app.secret_key is only a left-over and can be removed?

app.secret_key is typically used by Flask under the hood for signing session cookies, but it doesn't look like snappass uses those. I have also seen it used by some flask extensions for generating derived keys for CSRF protection and similar functionality. From a quick skim of the dependencies I don't see an obvious use case there either (itsdangerous is the only non-flask dependency that I know uses it and I believe we are just including it to pin the version since it is also included by Flask itself). I haven't tested running snappass without the secret key, but if the server runs, it should be safe to remove.

Thank you for the explanation! :+1: I'm unfamiliar with Flask and had assumed that it has to do with the password encryption scheme :sweat_smile:


⚠ MineVN Maven & Resource Pack has degraded performance

In f77ebd9, MineVN Maven & Resource Pack (http://app.minevn.net/pack) experienced degraded performance:

HTTP code: 404
Response time: 6239 ms

Resolved: MineVN Maven & Resource Pack performance has improved in 2501bc1 after 11 minutes.


Support dropdown on page header title

The page header title should support an optional element for switching views.  For example, in this SLDS example clicking on "My Leads" should display a dropdown with additional views to switch to.
See example in GUS for this behavior as well.

Adding features are the responsibility of the consuming product team. Please follow up if any B2C UXE or SLDS decisions need to be made.


$(MAKE) variable in Makefile

The $(MAKE) variable is set at the start of the Makefile script.
On macos, this can be something like /Library/Developer/CommandLineTools/usr/bin/make
This make is not accessible from within docker, so all the make commands run in docker fail.
Specifically, make build-pdk
Suggest changing $(MAKE) to make in the build-pdk target.

Thanks. This https://github.com/efabless/openlane/commit/4203c39590191ca2befa2a0c1c02f96bed646785 should fix it.

Thanks. This https://github.com/efabless/openlane/commit/4203c39590191ca2befa2a0c1c02f96bed646785 should fix it.

@d-m-bailey: This is now merged to master, so I'm closing this issue.


COCO export not working for polygons with 2 points only

I'm unable to export annotations to COCO format on the latest master branch.  Might be a pycocotools versioning issue.
Could not dump annotations for the task 4
Error: Request failed with status code 500. "cvat.apps.engine.utils.InterpreterError: Exception at line 307: input type is not supported.\n".

And here's the relevant error in docker:
cvat          | 2019-12-13 16:00:22,023 DEBG 'rqworker_default_0' stderr output:
cvat          | 16:00:22 cvat.apps.engine.utils.InterpreterError: Exception at line 307: input type is not supported.
cvat          | Traceback (most recent call last):
cvat          |   File "/home/django/cvat/apps/engine/utils.py", line 45, in execute_python_code
cvat          |     exec(source_code, global_vars, local_vars)
cvat          |   File "<string>", line 1, in <module>
cvat          |   File "<string>", line 319, in dump
cvat          |   File "<string>", line 250, in insert_annotation_data
cvat          |   File "<string>", line 141, in polygon_area_and_bbox
cvat          |   File "pycocotools/_mask.pyx", line 307, in pycocotools._mask.frPyObjects
cvat          | Exception: input type is not supported.

@stanbiryukov , Can you reproduce the problem on develop branch as well?

My mistake - just double checked and the above error was on the develop branch.
Master is not letting me login
Could not check authorization on the server
Error: Network Error.

@stanbiryukov , master should work fine. But let's concentrate on the issue. Can it be reproduced on a task without annotations? Is it a task with video or images?

Restarted docker, switched and logged in to master and getting the same error.
I'm working on an image task and confirmed that export works fine without annotations.
The issue might be a bit more complicated since I'm working with an originally uploaded COCO dataset.  If I restart and create a new annotation project I'm able to export to COCO successfully.

Note that CVAT xml exports work just fine so this might be a pycoco related issue and not cvat.

@stanbiryukov , if you can give us some steps to reproduce the problem it will be easy for us to investigate. In any case CVAT should handle the problem gratefully.

A possible way to export  the task as datumaro project and put it here. It will not have any images. Only annotations in an internal format. Probably it will give us some ideas how to reproduce.

Thanks - here's the datumaro export.
task_imprints-2019_12_13_16_35_05-datumaro_project.zip

As far as I understand, initially you used export annotations button? Could you try using export a dataset?

Exporting as MS COCO dataset does not work either - same type error:
cvat          | Exception: input type is not supported.
cvat          | Traceback (most recent call last):
cvat          |   File "/usr/local/lib/python3.5/dist-packages/rq/worker.py", line 812, in perform_job
cvat          |     rv = job.perform()
cvat          |   File "/usr/local/lib/python3.5/dist-packages/rq/job.py", line 588, in perform
cvat          |     self._result = self._execute()
cvat          |   File "/usr/local/lib/python3.5/dist-packages/rq/job.py", line 594, in _execute
cvat          |     return self.func(*self.args, **self.kwargs)
cvat          |   File "/home/django/cvat/apps/dataset_manager/task.py", line 331, in export_project
cvat          |     server_url=server_url)
cvat          |   File "/home/django/cvat/apps/dataset_manager/task.py", line 221, in export
cvat          |     save_dir=save_dir, save_images=save_images)
cvat          |   File "/home/django/datumaro/datumaro/components/project.py", line 521, in export
cvat          |     converter(dataset, save_dir)
cvat          |   File "/home/django/datumaro/datumaro/components/converters/ms_coco.py", line 422, in __call__
cvat          |     converter.convert()
cvat          |   File "/home/django/datumaro/datumaro/components/converters/ms_coco.py", line 406, in convert
cvat          |     task_conv.save_annotations(item)
cvat          |   File "/home/django/datumaro/datumaro/components/converters/ms_coco.py", line 190, in save_annotations
cvat          |     rles = mask_utils.frPyObjects(segmentation, h, w)
cvat          |   File "pycocotools/_mask.pyx", line 307, in pycocotools._mask.frPyObjects
cvat          | Exception: input type is not supported.

As far as I can see, the issue is here, the code fails because of there is a polygon with only 2 points, and it comes first in the array. Interesting.

Perhaps the best approach is to add a warning and skip those types of segments upon export or original import? I do see the 2 point polygon in the CVAT xml file now.  The CVAT api correctly doesn't allow for creation of polygons with less than 3 points.

Looks like with the new commit and this use case, there is now an error Unable to import annotation #1113: a polygon has too few points (2)
Is it possible to skip over the problematic polygons instead of breaking the entire import?  In this case I have thousands of annotations that can no longer be imported because of a few polygon errors.

@stanbiryukov , let us understand better the problem. How did you get annotations with only 2 points for a polygon?

@nmanovic I originally used a different annotation tool with this project: https://github.com/jsbroks/coco-annotator
I suppose I can try and manually edit the JSON file, however, from a user perspective, it seems most intuitive that the import raises a warning and skips problematic polygons instead of breaking the import.

@zhiltsov-max , what do you think? What is the best way to go here from your perspective?
My thoughts:
I think datumaro can have special options (e.g. --fix or --ignore-problems or --skip-issues) to import such bad annotations and export them into a valid COCO file again. I don't think that in CVAT we should ignore errors but for the case we can give a tool (datum.py) which will "filter" bad cases for the user and produces a valid COCO file which can be uploaded.

The idea to provide means in Datumaro to deal with the issue sounds well for me. What could be the good strategy of handling this, just ignoring the wrong annotations?

@zhiltsov-max ,

What could be the good strategy of handling this, just ignoring the wrong annotations?

Let's ignore and print a warning on the console.

Hi there - any update on the raised warning strategy for converting annotations?

It will be like stated above - in CVAT will be an error, in Datumaro will be an option to ignore corrupted annotations.

Error display during import and export operations will be tracked in #1529

@zhiltsov-max Has this issue been resolved?

Wrong annotations are not exported or imported, but there is no option to ignore them, or to print a list of errors.

@zhiltsov-max , it looks like it is something on what you are working now.