Include Chinese/Japanese/Korean/more fonts in headless Chrome binary

From downstream issue in https://github.com/graphcool/chromeless/issues/43
Should include the relevant fonts for all scripts, not just kanji.
Resources:

Use fontconfig? https://groups.google.com/a/chromium.org/forum/#!searchin/headless-dev/font/headless-dev/Pmnxb1lyDBg/DyNZYxMeBgAJ

@adieuadieu we solved this for a subset of languages (CJK) in our lambda function that uses phantomjs today. We were able to do it by:

packaging a .fonts directory with the TTF for NotoSansCJK-Regular.ttc and including that in our function upload zip.
adding an environment variable in our lambda function console (on AWS) called HOME set to /var/task. This allows Qt to pick up the included font.

The liability here is that due to the 50MB size limit on Lambda function packages you have to choose which fonts to be included carefully. In our case the Noto Sans font solved our issues, but I'm sure other fonts will be needed for other purposes.
I'm relaying most of this second-hand because one of my colleagues did most of the work earlier today. I'm going to try digging in further on it to make sure I got that right but that might be a possible solution for this project as well. I can try taking a crack at it next week if that would help, but I better get acclimated with this project more before doing any work.  Nice work on this BTW!

@toddwprice oh that's great news! I hope Chrome looks for fonts in the same place.
About the 50MB limit: if you deploy your Lambda function with the deployment package in S3, the package’s size limit increases dramatically — technically 250MB (realistically more around 100MB when packaging poorly compressable data like binaries.) Forgive me for linking to myself: I recently wrote more about it in this article.

@adieuadieu wow we were early adopters of Lambda but never questioned the 5B limit. Great article!  I will see if I can include Noto Sans for starters and if that works then we could add other fonts to plug other common holes.

*50MB

@adieuadieu I'm trying to get going on the project but getting errors with some missing dependencies and files when running npm test. Let me know if you want me to post my errors here or ping you somewhere else. Thanks.

@toddwprice Jump over npm test.  Which folder are you working in? packages/lambda may be the best place to play around in. There's a pesudo-integration test for Serverless there which you can use to deploy to Lambda. Run npm run build in the packages/lambda, then create a symlink in package/lambda/integration-test for a dist folder which points to the parent directory's dist  folder (packages/lambda/dist)
My local setup (it's not so pretty..):
marco:integration-test marco$ pwd
/Users/marco/src/github/serverless-chrome/packages/lambda/integration-test
marco:integration-test marco$ ls -lhtra
total 260800
-rwxr-xr-x   1 marco  502   127M May  9 07:14 headless_shell
lrwxr-xr-x   1 marco  502     8B Jun 18 23:21 dist -> ../dist/
-rw-r--r--   1 marco  502   463B Jul 10 18:48 serverless.yml
-rw-r--r--   1 marco  502   789B Jul 10 18:48 handler.js
drwxr-xr-x  13 marco  502   442B Jul 10 18:48 ..
drwxr-xr-x   7 marco  502   238B Jul 10 18:48 .

Any chance you're available to do a quick screen share? I'd really like to help contribute to this project.  If not I totally understand and I can push ahead on my own.

@toddwprice Sure—would at 18:30 CEST work? Could you DM me on Twitter or Gitter (@adieuadieu), or email (on my GitHub profile) so we can settle on a tool/service to screen share?

Perfect. Sent you a message in gitter.
On Fri, Aug 4, 2017 at 9:53 AM, Marco Lüthy<EMAIL_ADDRESS>wrote:

@toddwprice https://github.com/toddwprice Sure—would at 18:30 CEST
work? Could you DM me on Twitter or Gitter (@adieuadieu
https://github.com/adieuadieu), or email (on my GitHub profile) so we
can settle on a tool/service to screen share?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/adieuadieu/serverless-chrome/issues/49#issuecomment-320269578,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA7M1uw0usT3G2lMbmUmoLv-Poo27U3gks5sUzBOgaJpZM4OmTeV
.

--

Todd Price
<EMAIL_ADDRESS>

@adieuadieu I'm struggling to get a good test running in Lambda without adding too many other dependencies. My current approach is to use chrome-remote-interface directly inside the test handler. See this file: handler.js.zip.
Two problems so far:

Chrome spins up fine the first time, but fails afterwards. The logic around recognizing a running instance when a container is re-used is either not working or I've configured it wrong.

Screenshots are returning a blank page. I saw this behavior in the past when testing chrome --headless with chrome-remote-interface so it's likely something I'm doing wrong there.

Any pointers you could give me to get me on track with a valid test would be appreciated.

Hi @toddwprice thanks for the update. I would not worry too much about the first problem or adding too many dependencies. I would focus on just getting fonts working correctly with blatant disregard for anything else. Once fonts work, we'd have a proof-of-concept that it's possible. We can iterate from there to make it cleaner/easier.
With that in mind.. The example handler in this repository should work for capturing a screenshot, at least on Lambda. You might need to wait for the page to load before taking the screenshot. For a simple, mostly static page without any ajax-y behavior which occurs after the DOMContentLoaded event fires, you can wait on CDPs Page.loadEventFired() Promise to resolve before doing Page.captureScreenshot().

Well this is probably no help since I don't even use AWS, but I'll share anyway. I run chromeless in a docker-container side by side with knqz/chrome-headless and to that I add:
ADD https://noto-website.storage.googleapis.com/pkgs/NotoSansCJKjp-hinted.zip /tmp
RUN unzip /tmp/NotoSansCJKjp-hinted.zip && \
    mkdir -p /usr/share/fonts/noto && \
    cp *.otf /usr/share/fonts/noto && \
    chmod 644 -R /usr/share/fonts/noto/ && \
    fc-cache -fv

(All of noto is 120MB and only regular is 15MB) after that at least japanese works fine. You probably have all that figured already, so sorry for being noisy!

Thank you for the tip, @kumorig!

Please use it as a reference
http://fd0.hatenablog.jp/entry/2017/09/10/223042 (sorry, written in Japanese)

use custom fontconfig
use small size font
strip chrome binary

thanks @fd00, I tried exactly as your guide in the blog but it seems not working for me, I still get a lot tofu after deployed to lambda. Not sure if I missed anything :(.

Updated: OTF fonts from google does not work for me, but TTC does work well with fontconfig following the guide of @fd00 thanks a lot mate :).

another update<EMAIL_ADDRESS>with headless_shell changing to headless-chromium, the method of @fd00 stopped working and the tofus are now coming back :(

@nmqanh you might just need to change the name/paths in a few steps from headless_shell to headless-chrome. For example, in the article, in the Deploy section, there is reference to CHROME_PATH pointing at headless_shell. Change this to headless-chrome.
I have an implementation of font support in progress that I'll finish sometime over the next week or two which will close this Issue.

Thanks for your advise @adieuadieu , tried to assign
CHROME_PATH to /var/task/node_modules/@serverless-chrome/lambda/dist/headless-chromium on lambda but it's still showing tofus so I guess it can be something else. In the worst case, I would love to wait for your next release though. Feel free to let me know if I can help anything :) or I can even help you test.

Just a quick update: I tried updating the CHROME_PATH and also re-built the font cache from step 0 as guided in the article and it does not work, tofus are still coming back with<EMAIL_ADDRESS>and later. Thanks for the good news that new release gonna support CJK fonts by default in 1-2 weeks :). Would love to try it soon. Please let me know if there is anything I can help.

@adieuadieu We have tried several times to add support for this, but still a dead end. Have you manage to figure it out? Can we assist with something?
Thank you for your great work.

@nat-n I tried to follow your note, but I'm missing knowledge from "Configuring fontconfig" to the end. Could you please detail more on how to do it or give links to learn what I'm missing. Thanks

@luminous8 I can try to help, but I'm not sure what you're missing. The general idea is that the fontcofig build inside the container also exists under /tmp outside the container, and so you can make some requires changes to it there. I've just fixed a formatting issue that might have made a part of if less clear but I'm afraid I can't make the instructions too concrete without making them to specific to a particular setup (which may be different from your own).

I followed @toddwprice's suggestion and it works locally but not in lambda. Then I tried *.otf file instead of *.tcc file, it works both locally and lambda. The font I used is NotoSansCJKtc-Black.otf
So you may try both font files and see if it works.

In case this saves someone else some work.
I got fonts working by putting them in .fonts and setting HOME=/var/task, but you need to make sure that the font files have permission 644 (-rw-r--r--) otherwise they won't work.

I'm still unable to get these characters to render. Can somebody clarify this for me please - where does this .fonts directory go?
I added the same fonts as listed in @abargnesi 's comment, changed their perms to 644, and tried:

adding this directory to `chrome/chrome-headless-lambda-linux-x64.tar.gz' archive
adding this directory at top level of serverless-chrome git repo root
serverless deploy, but my characters are rendered as empty space (not even squares).

Any ideas?
Thank you .

Bump

@kirilledelman Any updates? I'm not able to render special characters too.

@NickBlow @abargnesi I am copying the fonts to .fonts folder and also set the HOME to /var/task. But I noticed the otf fonts are NOT working using this method. They work locally but not through lambda.
On the other hand, ttf fonts work both locally and through lambda. Any pointers on what could be going wrong with otf fonts?

I've managed to make it working with layers. @NickBlow comment is not visible enough, got a lot of troubles due to permissions...

In case this saves someone else some work.
I got fonts working by putting them in .fonts and setting HOME=/var/task, however it didn't work for me until I made the font files have permission 644 (-rw-r--r--).

I wanted to provide font for every possible using Noto. I download all unhinted versions (Using download all fonts).
Then I run the following command to extract and pick only variants I want to keep layer size as small as possible.
mkdir -p fonts/.fonts
unzip Noto-unhinted.zip
cp NotoSans*-Regular.* fonts/.fonts
cp NotoColorEmoji.ttf fonts/.fonts
chmod 644 fonts/.fonts/*
zip fonts.zip -r fonts

Then I  store the file in my project under layers/fonts.zip and I use the following serverless config:
service: chrome-lambda
provider:
  name: aws
  runtime: nodejs10.x
  timeout: 30

package:
  exclude:
    - node_modules/puppeteer/.local-chromium/**
    - layers/**

functions:
  headless-chrome:
    handler: src/index.handler
    memorySize: 1600MB
    timeout: 30
    environment:
      HOME: /opt/fonts
    layers:
      - { Ref: FontsLambdaLayer }
    events:
      - http:
          path: /
          method: post

layers:
  fonts:
    package:
      artifact: layers/fonts.zip

custom:
  chrome:
    flags:
        - --hide-scrollbars
        - --ignore-certificate-errors
    functions:
        - headless-chrome

Note: Make sure to use the hinted version of Noto font!

Note: Make sure to use the hinted version of Noto font!

But you say earlier:

I download all unhinted versions (Using download all fonts).
and run:
unzip Noto-unhinted.zip
Am I missing something here, or did you meant to say unhinted or hinted everywhere.


Validate ballot size before unmarshaling ballot

This check used to exist, but it was broken causing the unmarshaling to completely misbehave.
It was removed in this PR https://github.com/c4dt/d-voting/pull/14
and now needs to be reviewed, and the check needs to be added again provided we add:

Good test or two to validate the check is working as expected (both positive and negative cases)
Test this in integration mode for this specific kind of question: select

Linked with #31


Multicluster app - Allow admin to edit all multicluster apps

rancher/rancher:master-38fd13a8 3/7
HA installation

As admin create cluster
As admin create projects
As admin add User2 as cluster owner
As User2 create multicluster app
As admin edit multicluster app roles/members/target projects

Expected:

Admin is able to make edits to the multicluster app

Actual:

HTTP 500 - user user-pz84w is not in members list

@bmdepesa Wait for the fix for issues - https://github.com/rancher/rancher/issues/18774 , before you validate this issue .

rancher/rancher:master 3/12

Created two clusters
Created additional user
Added second user as cluster owner to one of the clusters
Login as second user and create a multicluster app in the cluster where user has access
Login as admin (without admin being added as a member to MCA) and add a second target project
Target projects / MCA updated successfully.


Warn when step size falls below a given threshold in adaptation?

Description:
Currently adaption makes no attempt to warn the user when the step size falls to extremely small values.  Technically such small values are okay and indicate an intermediate state where the sampler is trying to adapt to a posterior with a poorly-posed covariance.  Because those models are not ideal, however, it may be useful to warn the user.  Additionally, inaccurate gradients also force the adaptation to extremely small step sizes.
A somewhat-informative warning can be displayed to users, but when should it be displayed?

Anytime that the step size adaptation falls below a threshold?  This has the advantage of warning the user quickly, but also will dump a ton of output.
Anytime the step size adaptation transitions from above the threshold to below?  This reduces the output but also doesn't display any warnings when adaptation terminates.
At the end of each adaptation window?  This doesn't warn the user early but it does remind them towards to the end of adaptation.

Current Version:
v2.14.0

If we allow users to set the threshold reporting when it falls below the threshold at each iteration seems like the best. Almost an aside: could we detect and distinguish the two sources of low step size? I think the answer is yes and it would be useful to report but I doubt we could do it reliably.

Making it configurable as suggested is certainly nice. Among the three options you present I would opt for the second one; reporting when it transitions across the threshold.


"Every operation must have unique "operationId""?

If I have a reference to another path, why am I requested to provide a different id?
I am getting Every operation must have unique "operationId" error
  Message :   Every operation must have unique "operationId".
  Rule    :   operation-operationId-unique
  Path    :   paths./municipios/{municipio}.get.operationId
  Line    :   209

I am using v 0.96.2 and this is the YAML
  /municipios/{municipio}:
    get:
      operationId: getMunicipality
      summary: Obter detalhes sobre Município
      parameters:
        - in: path
          name: municipio
          required: true
          schema:
            type: string
            example: évora
          allowReserved: true
          description: Município
        - $ref: '#/components/parameters/json'
      responses:
        '200':
          description: Detalhes do Município
          content:
            application/json:
              schema:
                type: string
        '404':
          description: Município não encontrado
          content:
            application/json:
              schema:
                type: string

  /municipio/{municipio}: # when I remove this path and $ref, it works
    $ref: '#/paths/~1municipios~1%7Bmunicipio%7D'

Each operation should have a unique identifier, regardless of what path they live under


Test file must be placed in folder tests only

Please do not post any internal, closed source snippets on this public issue tracker!
Description
Now test file must be placed in folder sysl/tests only, otherwise Go-Windows / Build Windows and test can't work. Is it expected?

 It does some limitation on test file location and doesn't impact Sysl itself, so it can be set as low priority.


Raised field on crossing presets triggers geometry type warning

As reported in OSMUS Slack, checking the Raised field on the line-based Pedestrian Crossing preset produces a warning about using traffic_calming=table on a line:

This regression of #733 seems to be related to https://github.com/openstreetmap/id-tagging-schema/pull/1076#issuecomment-1896129765 and might require a fix in iD proper.

Not sure if this also helps, but the warning shows up on crossings for all values of traffic_calming= except for:

bump
mini_bumps
hump
table
cushion
rumble_strip
dip
chicane
choker
island

Misspellings (and other invalid values) also produce the warning.

@1ec5 I've closed #1279 as a duplicate. From a quick check with overpass, it appears that nobody in the UK has followed this suggestion yet. All the instances I've found predate this issue and will be checked and fixed if necessary in the next few days.


openstack-ardana: set default project for heat_stack role

As the heat_stack role is also used in SES and it does not use the same inventory as ardana, it cannot get the default value for os_project_name which is defined on the ardana group vars.
Defining the os_project_name in the defaults values for the heat_stack role ensures that we have the default value available for anyone who uses the role.

This is needed to fix the manual Crowbar/Ardana builds.


Geojson and fill color

I am using react-native-maps with geojson. My json have 14 state (region). It is show on map but I would like to show different color for each state. My geojson already had a fill color and stroke color but it is not show on map. May I know how can I add the color for each region.
Thanks.

Ok, I can managed to solved it. added the fill color in <MapView.Polygon />

Hi @alexaung, I have the same issue. May you provide your code sample so that I could know how to put it in my code. Thank you!

I have the same problem and tried to use Polygon but they are showing in the middle of the ocean. When I was using  everything was ok.


Multibyte special character messes up initials

We parsed the author name from a page, where the input name is © Pavel Voinau.
Expected result
not return an initials, return null as initials
Actual result
When dumping the value, name-parser exports "Â ©"

How to fix

make initial parser multibyte safe
do not parse copyright sign

see draft https://github.com/theiconic/name-parser/pull/37


Nightly Shepherd indexing stopped?

Fetch and MIME Sniffing do not appear to be recently indexed. Blocks https://github.com/whatwg/xhr/pull/130.

Cc @plinss

Stuck process on server, kicked, running now... (will take a few minutes to update backlogged tasks)

Now bikeshed update seemingly hangs on "Downloading anchor data...".

Seemed to have eventually succeeded. Closing this. Thanks all!


SageMath support

SageMath is an extension of Python that allows to use math objects such as matrices, groups, fields, polynomials, etc.
I was wondering if we could have an official tutorial how to use SageMath using this extension. Particularly on Windows. There are some hacks, but official support would be great.
Elad

Thanks for filing this issue.
We don't have an official tutorial as such fo SageMath, as this question is related to general python programming & not specific to this extension, I would suggest you raise this issue on Stack overflow.


Update to latest com.android.tools 26.2.1

I updated the libs, but I have a big problem:
Google has changed the version of Guava to the latest, I have it in project dependencies, but com.android.tools loads it from Netbeans module com-google-guava and there is version 19.0
I created a netbeans patch and added the missing method signature, but it still uses the original version.
java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument (ZLjava / lang / String; Ljava / lang / Object;) V

When it does not work this way, then the other side:
I made a modified version of BuildToolInfo, with the problematic call removed. (https://github.com/NBANDROIDTEAM/NBANDROID-V2-guava-version-patch)
I added it as Netbeans module patch to NBANDROID-V2.
.


Update README.md

Description
I've only adjusted the README.md. Before when you clicked on anything it would say File not Found. I have solved that in this fork.
Checklist for all submissions

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?
[x] Have you tested and reviewed your changes with confidence that everything works?
[x] * [x] Did you branch your changes and PR from that branch and not from your master branch?

If not, why?: No need to as it only is a small README.md change.

No problem.


error: cannot find symbol String while updating to RN 0.73.4

Hello,
I get an error after updating our app to the latest RN.
error Failed to install the app. Command failed with exit code 1: ./gradlew app:installDebug -PreactNativeDevServerPort=8081 .../node_modules/blueshift-react-native/android/src/main/java/com/blueshift/reactnative/BlueshiftReactNativeModule.java:440: error: cannot find symbol String errorMessage = getReactApplicationContext().getString(R.string.bsft_inbox_delete_failure_message); ^ symbol: variable string location: class R Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. Note: .../node_modules/blueshift-react-native/android/src/main/java/com/blueshift/reactnative/BlueshiftReactNativeEventHandler.java uses unchecked or unsafe operations.
Java version: 17
gradle version: 8.3
blueshift sdk version: 3.4.4

Hi @ayobami303. The issues reported on GitHub aren't closely monitored.
The primary -- and the correct -- way to report an issue to us is via the customer support channel. Send us an email at<EMAIL_ADDRESS>from your work email with all the necessary details that contain step-by-step instructions on how to reproduce the issue.
In addition, include the following details based on the nature of the issue.

Crash Log (in case of app crash)
Screenshots/Screen Recordings
Device details (Make, OS version etc)
Blueshift Android/iOS SDK version being used
Campaign Name/Link (if available)
Template Name/Link (if available)
Any extra details that you think would help in repro the issue

Thanks,
Rahul


Add ar-EG

Adds Arabic locale that suits most of Arabic countries

Coverage increased (+0.07%) to 98.204% when pulling 2ab9400543b7b6ae8342dc98bfcb9e4d2809e7ed on enghelewa:patch-1 into 20a1cd6fcaa72d4672dc2072b3064a7d804239d3 on wxsms:master.


Adjust memberOf casing to fix group membership

This should fix group membership checks, our ldap server uses the default memberOf instead of memberof

Looks like tests are failing

Do you have an idea what it might be? I've skimmed over them and it doesn't look related. Possibly it is not using the secrets because I am not a repo owner?

Do you have an idea what it might be? I've skimmed over them and it doesn't look related. Possibly it is not using the secrets because I am not a repo owner?

Yes it looks like this is a secrets issue. I'll merge this into a separate branch and then we can run the tests again


[Bug]: [Windows] ChromaDB crashes when querying a collection with more than 99 records after running normally for two months!

What happened?
ChromaDB 0.5.0 suddenly crashed one morning due to a query after running normally for two months.
The last few logs are (The last one is "Starting component PersistentLocalHnswSegment" everytime when it crashes):
[32mINFO[0m:     [04-11-2024 12:19:31] myIP:61952 - "GET /api/v1/collections/redmine?tenant=default_tenant&database=default_database HTTP/1.1" 200
[32mINFO[0m:     [04-11-2024 12:21:43] myIP:62055 - "GET /api/v1/pre-flight-checks HTTP/1.1" 200
[36mDEBUG[0m:    [04-11-2024 12:21:43] Starting component PersistentLocalHnswSegment

I eventually discovered that the reason was that the collection I was querying had more than 99 records, while another collection with only 83 records didn't have this issue. However, this doesn't make sense because this collection had been used normally with over a thousand records for two whole months!
Yes, there are some similar problems reported (such as https://github.com/chroma-core/chroma/issues/2513).
However, this time the situation is somewhat special. Two months ago, I installed and used ChromaDB, and after using it for a week, I encountered a similar issue. At that time, the database I created using chromadb.PersistentClient() would silently crash without any warning when inserting more than 99 records.
So, I followed the solution from the aforementioned issue(https://github.com/chroma-core/chroma/issues/2513) and downgraded ChromaDB to version 0.5.0 and hnswlib to 0.7.3, but it was of no use. Later, I accidentally discovered that when I switched to using chromadb.HttpClient() to start the database, everything returned to normal. Although the issue wasn't completely resolved, I felt that as long as the program could run, it was fine.
Therefore, ChromaDB worked normally for two months, then suddenly crashed during a query last Friday. Moreover, I can no longer insert any data into the database if the target collection already has more than 99 records or if my insertion would exceed 99 records.
People often say that when God closes a door, He opens a window. Yes, I once had that window, which was chromadb.HttpClient(). Now, this last window has been mercilessly closed. It was still working normally on Thursday night—it ran smoothly for two months, and the next day when I returned, it crashed?  XD
And you know what? The exact same code and the exact same ChromaDB now work perfectly in a Linux virtual machine on my Windows system. Although it can work in a Linux virtual machine's Docker, this still causes unnecessary trouble for me when testing the program on the Windows system. Anyway, I hope this issue gets resolved soon (maybe when I come back tomorrow morning, it will suddenly work again, since if it can suddenly stop working, it might just as suddenly start working again).
Versions
Chroma 0.5.0
hnswlib 0.7.3
python 3.12.6
Windows10 Corp 22H2
CPU: Intel(R) Core(TM) i7-9700F CPU @ 3.00GHz
RAM: 32.0 GB
Relevant log output
[32mINFO[0m:     [04-11-2024 12:19:03] Uvicorn running on http://<IP_ADDRESS>:8000 (Press CTRL+C to quit)
[32mINFO[0m:     [04-11-2024 12:19:29] myIP:61951 - "GET /api/v1/tenants/default_tenant HTTP/1.1" 200
[32mINFO[0m:     [04-11-2024 12:19:29] myIP:61951 - "GET /api/v1/databases/default_database?tenant=default_tenant HTTP/1.1" 200
[32mINFO[0m:     [04-11-2024 12:19:30] myIP:61952 - "POST /api/v1/collections?tenant=default_tenant&database=default_database HTTP/1.1" 200
[32mINFO[0m:     [04-11-2024 12:19:30] myIP:61952 - "POST /api/v1/collections?tenant=default_tenant&database=default_database HTTP/1.1" 200
[32mINFO[0m:     [04-11-2024 12:19:30] myIP:61952 - "POST /api/v1/collections?tenant=default_tenant&database=default_database HTTP/1.1" 200
[32mINFO[0m:     [04-11-2024 12:19:31] myIP:61952 - "POST /api/v1/collections?tenant=default_tenant&database=default_database HTTP/1.1" 200
[32mINFO[0m:     [04-11-2024 12:19:31] myIP:61952 - "GET /api/v1/collections/redmine?tenant=default_tenant&database=default_database HTTP/1.1" 200
[32mINFO[0m:     [04-11-2024 12:21:43] myIP:62055 - "GET /api/v1/pre-flight-checks HTTP/1.1" 200
[36mDEBUG[0m:    [04-11-2024 12:21:43] Starting component PersistentLocalHnswSegment

database = chromadb.HttpClient(DB_HOST, DB_PORT)
print(database.get_collection(COLLECTION_NAME).query(query_embeddings=embedding_text("some_text"), n_results=5))

Simple code, Big problem.
It has been running for two months, and now collapses at the slightest touch.

@SchwarzeFahne, sorry to hear about your troubles. These kind of issues are extremely frustrating. The 100+ record thing is related to default BruteForce (buffer) index in Chroma which holds up to 100 uncommitted embeddings in memory and performs KNN search by going thru all the vectors. The issue arises from HNSW index which is used when you have more than the default buffer size.
The problem with the HNSW running on Windows machines is pretty annoying and does not entirely depend on Chroma hnswlib version. Over time various windows configuration have managed to run Chroma whereas other has failed (like in your case). Changing python version and applying the latest Windows patches worked for some (if you haven't give it a try, python 3.11 + latest patches).
On a related note, I think that Linux is the only proper way to run Chroma in production or at least more serious workloads, but of experimentation Windows is a perfectly valid choice too.
That said there is a workaround you can use to fix the problem - bump your hnsw:batch_size to a figure that is well above expected number of embeddings. Yes the queries might be a bit slower (especially on large datasets), but it will work just fine for experimentation on Windows machines.
import chromadb

client = chromadb.PersistentClient("path/to/db")

collection = client.get_or_create_collection("my_collection",metadata={"hnsw:batch_size":10000})
# add data to the collection and query it

More info here.

@SchwarzeFahne, you can also rebuild the HNSW lib on your system. Are you using WLS or PS for running your apps?
For WLS:
sudo apt update && apt install build-essential cmake autoconf
python -m venv venv
source venv/bin/activate
pip install --no-binary :all: chroma-hnswlib

@SchwarzeFahne, sorry to hear about your troubles. These kind of issues are extremely frustrating. The 100+ record thing is related to default BruteForce (buffer) index in Chroma which holds up to 100 uncommitted embeddings in memory and performs KNN search by going thru all the vectors. The issue arises from HNSW index which is used when you have more than the default buffer size.
The problem with the HNSW running on Windows machines is pretty annoying and does not entirely depend on Chroma hnswlib version. Over time various windows configuration have managed to run Chroma whereas other has failed (like in your case). Changing python version and applying the latest Windows patches worked for some (if you haven't give it a try, python 3.11 + latest patches).
On a related note, I think that Linux is the only proper way to run Chroma in production or at least more serious workloads, but of experimentation Windows is a perfectly valid choice too.
That said there is a workaround you can use to fix the problem - bump your to a figure that is well above expected number of embeddings. Yes the queries might be a bit slower (especially on large datasets), but it will work just fine for experimentation on Windows machines.hnsw:batch_size
import chromadb

client = chromadb.PersistentClient("path/to/db")

collection = client.get_or_create_collection("my_collection",metadata={"hnsw:batch_size":10000})
# add data to the collection and query it

More info here.

Thank you for your help. Since I can't upgrade the Windows patches at the moment, I tried adjusting the hnsw:batch_size method. Indeed, after adjusting the buffer size, ChromaDB is working again on my Windows system just like it did in the past two months. Thanks again!
However, I am puzzled because over the past few dozen days, ChromaDB never crashed because of this issue until it suddenly happened last Friday, like a sudden acute illness. Before that, even with thousands of records in the collection and sometimes inserting thousands of records at once, ChromaDB was functioning normally. How could it suddenly crash one day?
Moreover, two months ago, I only encountered crashes when inserting more than 99 records while using the PersistentClient() method to access ChromaDB, whereas there were no such issues with the HttpClient(). It's only recently that this problem has manifested, which is quite perplexing.
My current concern is that my ChromaDB used in production is indeed Linux. But if users access it with clients on Windows systems, could it also suddenly crash one day?

@tazarov
Sorry to bother you again. I have another small issue which, although currently minor, might be a bug. When I start ChromaDB on a Windows system and connect using the HttpClient() method, the list_collections function works fine. However, when I start ChromaDB on a Linux system and connect from a Windows system using the HttpClient() method, calling list_collections gives me this message in the terminal. What is causing the program to terminate unexpectedly?
Traceback (most recent call last):
  File "d:\Workshop\projects\ai_assistant\DatabaseManager.py", line 486, in <module>
    manager.list_collections()
  File "d:\Workshop\projects\ai_assistant\DatabaseManager.py", line 67, in list_collections
    collections = self.__database.list_collections()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\AppData\Local\Programs\Python\Python312\Lib\site-packages\chromadb\api\client.py", line 177, in list_collections
    return self._server.list_collections(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\AppData\Local\Programs\Python\Python312\Lib\site-packages\chromadb\telemetry\opentelemetry\__init__.py", line 143, in wrapper
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "C:\Users\me\AppData\Local\Programs\Python\Python312\Lib\site-packages\chromadb\api\fastapi.py", line 212, in list_collections
    collections.append(Collection(self, **json_collection))
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Collection.__init__() got an unexpected keyword argument 'configuration_json'

To clarify, I am currently using ChromaDB 0.5.0. If this issue is due to the fact that I am using a somewhat older version, please let me know, and I will upgrade to see if it resolves the problem.

@SchwarzeFahne, this seems like a client/server mismatch. Can you check you server and client and make sure they are on the same version.
Chroma moves pretty fast and these kind of incompatibilities are expected all the time until 1.0.

@SchwarzeFahne, this seems like a client/server mismatch. Can you check you server and client and make sure they are on the same version.
Chroma moves pretty fast and these kind of incompatibilities are expected all the time until .1.0

Thanks for your help. I noticed that the Server's Chroma version is indeed a bit higher, so I updated the Client's version. Everything is running smoothly now, and it seems these issues won't trouble me for the time being.


🍒 Cherry pick PR #7277 to staging 🍒

🍒 Cherry pick https://github.com/Expensify/App/pull/7277 to staging 🍒

🚀 Cherry-picked to staging by @OSBotify in version: 1.1.30-1 🚀

platform
result

🤖 android 🤖
failure ❌

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

@Expensify/applauseleads please QA this PR and check it off on the deploy checklist if it passes.

🚀 Deployed to production by @roryabraham in version: 1.1.30-3 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


fix: update x/net for CVE-2022-41721

A CVE was discovered in x/net. This is currently blocking everything. This manual upgrade resolves the alerts (hopefully).

Codecov Report

Merging #301 (83ab94e) into main (152bf59) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head 83ab94e differs from pull request most recent head 8e7b402. Consider uploading reports for the commit 8e7b402 to get more accurate results

@@           Coverage Diff           @@
##             main     #301   +/-   ##
=======================================
  Coverage   51.48%   51.48%
=======================================
  Files           4        4
  Lines         540      540
=======================================
  Hits          278      278
  Misses        241      241
  Partials       21       21

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

@beeme1mr @AlexsJones
This fixes an 8.2 CVE but is blocked by a FOSSA false positive I cannot dismiss (it's reading encoded test files falsely as licenses).


Kodi Netflix log in on andoid tab

Hello,
I have became a member of this forum recently, so I dont know is there any word about log in kodi netflix addon on android tablet. I cant watch Netflix on android app, because my Lenovo tab M8 is less and less supportive to a new uptades on Netflix. For Desney+ and other simmilar apps reproduction is much better on Kodi. So, I am wondering is it possible to log in Kodi Netflix on android:

I can`t find any NFAKey for Android. Is there any?
I was trying with linux key and android python. It `s inpossible.
Is there any tutorial how to login Kodi Netflix on android?

Thanks in advance.
Srdjan

You just need to get one on PC and then transfer it to your android device and import it from there (for example with a USB Stick and USB OTG). It uses browser automation, so you can't really do it on Android directly.


cds-pg 'and' operator not working

hello,
/Things?$top=50&$filter=_id eq ' 327036AE27D94E9B9B85FC91A3ED0BC0' and _thingType eq 'ABC2Series'
'and' operator for filter is working in locally sqlite but after deploying to a postgresql server it is not working.


Improve logo

LODSPKeaKr logo is fine but it can be better
Options

Include at least the initials (LS, LDSPK)
Design a brand new logo

It includes 'LODSPeaKr' in http://lodspeakr.org/img/lodspeakr_logotype.png


Added the Acrobot Env, without the rendering

Will be reintroducing the Acrobot environment along with the remaining classic control environments. Closing the PR now.


Wrong structure name

Why is this page named for the structure tag (_HUB_DEVICE_CONFIG_INFO_V1) instead of the structure type (HUB_DEVICE_CONFIG_INFO)? None of the docs or examples ever refer to this structure using the tag

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 1b88c77d-d9a8-9cfe-7cea-d2a502fb9248
Version Independent ID: 7dbc8a6f-c34d-ba6b-dfc0-b4e12472e402
Content: _HUB_DEVICE_CONFIG_INFO_V1
Content Source: wdk-ddi-src/content/usbioctl/ns-usbioctl-_hub_device_config_info_v1.md
Product: windows-hardware
GitHub Login: @windows-driver-content
Microsoft Alias: windowsdriverdev

@PageWriter-MSFT this seems to be a dupe of Issue #197. Can you verify and close, if so?

The new documentation system uses automation to assemble the reference documentation based on the source files. We are using Visual Studio’s intellisense engine for API discovery. This has the advantage of making our docs consistent with what users see in Visual Studio. Unfortunately, that means that we moved to the actual API names from the more appealing typedef aliases.
Page Writer

Then your automation should be fixed!
Other structures that reference or incorporate structures refer to them by structure name, not by the tag, so people are going to search for the structure name.

About searching: All aliases are added to the SEO list for each topic. Searching for any related term should bring up the correct topic.
Page Writer

That doesn't help people who are browsing the table of contents. Bottom line: From a usability perspective, there is no good reason for using just the tag as over the structure name. If your automation prevents that, then fix your automation! Clearly, you have the structure names if you're adding them to your SEO list, so there is no reason not to use the structure name in the docs. A good argument could be made to list both the structure name and the tag name in the docs


[Bug Report] Player jumping around when moving channels

Describe the bug
After todays update when moving the bot between channels it disconnects and reconnects directly after moving it.
To Reproduce
Steps to reproduce the behavior:

Have music playing
Move bot
See bot jumping in and out after move

Expected behavior
It should move, simply, as before.
Screenshots
https://gfycat.com/DiligentGlassIberianchiffchaff
Debug Info
debug_information.txt
Additional context
Only occured after todays update, restarting does nothing.

Closing as there have been several changes to both the dependencies of the bot and Discord's voice servers since this was opened. If this can still be reproduced, feel free to reopen this issue or make a new one with updated info.


@ModifyExpressionValue may have some problems

if (!this.level.getBlockState(this.worldPosition.below((int)Math.ceil((double)newOffset))).canBeReplaced()) {
In this code, even though canBeReplaced returns true, the original obtained through @ModifyExpressionValue is also true
Not only that.
return original&&canThrough
canThrough is false, which should theoretically return false, but is actually true

I'm not exactly sure what you mean but i sincerely doubt it's a bug. Export the target class and play around with the debugger and join the discord if you're confused by something.


How to use S3 for OAuth and installation state store?

Currently I am using FileOAuthStateStore for storing OAuth and installation states, however I want to store these states in S3 instead. What object would I need to use instead of FileOAuthStateStore, and how would I go about setting this up? This is how my OAuth object looks currently (just followed docs)
# Define state store outside of oauth settings to be used in oauth_redirect
state_store = FileOAuthStateStore(expiration_seconds=600, base_dir="./data")

# Create Bolt oauth settings object
oauth_settings = OAuthSettings(
    client_id=os.environ["SLACK_CLIENT_ID"],
    client_secret=os.environ["SLACK_CLIENT_SECRET"],
    scopes=[...],
    installation_store=FileInstallationStore(base_dir="./data"),
    state_store=state_store,
    callback_options=CallbackOptions(success=success, failure=failure),
)

Requirements
For general questions/issues about Slack API platform or its server-side, could you submit questions at https://my.slack.com/help/requests/new instead. :bow:
Please read the Contributing guidelines and Code of Conduct before creating this issue or pull request. By submitting, you are agreeing to those rules.

Hi @JasonDykstraSelectStar, thanks for asking the question! I'm happy to know that you are going to use the S3 store!
You can do the same with this one: https://github.com/slackapi/bolt-python/blob/v1.7.0/slack_bolt/adapter/aws_lambda/lambda_s3_oauth_flow.py#L29-L58 but a simpler example would be:
import os

# pip install boto3
import boto3

# pip install slack-sdk
from slack_sdk.oauth.installation_store.amazon_s3 import AmazonS3InstallationStore
from slack_sdk.oauth.state_store.amazon_s3 import AmazonS3OAuthStateStore

# pip install slack-bolt
from slack_bolt import App
from slack_bolt.oauth.oauth_settings import OAuthSettings

# The Python AWS SDK
s3_client = boto3.client("s3")

app = App(
    signing_secret=os.environ["SLACK_SIGNING_SECRET"],
    oauth_settings=OAuthSettings(
        client_id=os.environ["SLACK_CLIENT_ID"],
        client_secret=os.environ["SLACK_CLIENT_SECRET"],
        scopes=["commands", "chat:write"],
        installation_store=AmazonS3InstallationStore(
            s3_client=s3_client,
            bucket_name="your-own-s3-bucket-name-for-states",
            client_id=os.environ["SLACK_CLIENT_ID"],
        ),
        state_store=AmazonS3OAuthStateStore(
            s3_client=s3_client,
            bucket_name="your-own-s3-bucket-name-for-installations",
            expiration_seconds=600,  # the default value: 10 minutes
        ),
    )
)

I hope this was helpful to you. Do you have anything further to ask here?

I think that should cover it, thanks so much for the quick response! :)

Sorry for the re-opening, but I was wondering what settings would be required in the settings.py file of a django application. I see online that variables such as AWS keys and changing the static url are required, but am not sure if what bolt does is considered using static files?

@JasonDykstraSelectStar
For connecting to S3 to fetch/store installation / OAuth state data, no additional configuration in settings.py is needed. As for the AWS credentials, the common way is to load the value from env variables.
I'm not sure about what you mean by static files but if you are wondering if Bolt may treat installation / state value as static files first and the synchronize them to S3, that's not our approach. The S3 installation store directly communicates with Amazon S3 APIs to fetch/upload/delete the stored data.


Seconds to HH:MM:SS Bug

Bug in package eu.hansolo.toolbox.Helper
public static final long[] secondsToHHMMSS(final long seconds)
should be
public static final long[] secondsToHHMMSS(final long seconds) {
        long secs    = seconds % 60;
        long minutes = (seconds / 60) % 60;
        long hours   = (seconds / (60 * 60)) % 24;
        return new long[] { hours, minutes, secs };
 }

I would also rename secondsToHHMMString to secondsToHHMMSSString

That’s funny, I‘ve stumbled upon this issue 3 days ago in one of my Swift apps and fixed it there. Thy for the heads up, will fix it.👍🏻

Fixed with release 21.0.17


Quickfix jsdoc in Typescript files

Add a codefix for the "JSDoc types can only be used inside documentation comments" error in .ts files. The codefix offers to convert the JSDoc-syntax type into the equivalent Typescript-syntax type.
Notes:

Because this error is not issued in Javascript files, it is not applicable to (for example) flow files that have the .js extension. Those files will continue to get the "types can only be used .ts files" error.
Because the codefix uses getTypeFromTypeNode and typeToString, you will only get T | null from ?T for strictNullChecks: true. This is technically correct, but loses the intent of the JSDoc.
The codefix has a special case for ?T since it is valid flow syntax as well as valid JSDoc/Closure syntax. The special case converts ?T to T | null | undefined. Because of other flow and typescript differences, this is not likely to be a successful Flow → Typescript converter!

Good idea. I added some documentation.


Fix koji config error message

I hit this error msg issue when running the latest img today.

very simple fix. merge directly.


Fix linting

Codecov Report

Merging #708 (a10add4) into main (ad0abc6) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #708   +/-   ##
=======================================
  Coverage   85.83%   85.83%
=======================================
  Files         308      308
  Lines       28719    28719
=======================================
  Hits        24650    24650
  Misses       4069     4069

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ad0abc6...a10add4. Read the comment docs.


[v2] Adopt the spring boot framework

Work on adopting spring boot framework as part of topology.

@KetanA as discussed let's focus on this as next step.

Make this as P0 please.


Suggestion: New operator 'switchb' - "Auto-breaking switch statement"

Problem Statement
It is generally considered bad form to use fall throughs in switch statements. We've even added a compiler option to prevent it completely: noFallthroughCasesInSwitch.
With this option enabled, we've declared that break is always and forever required in each and every case statement. Which makes the 'break' statement completely redundant.
Changing the semantics of the switch operator to automatically break would be shockingly counter to the principle of least surprise. Hence, I propose the addition of a new operator ('Decision-making construct'): the auto-breaking switch statement.
Similar Functionality
This behavior can be easily coded manually using either if/else blocks or even just living with the current behavior of case "x": doThing(); break;. This proposal is for convenience only.
Proposed Changes

Choose a name for the statement. switchb, bswitch, whatever, etc.
Because Typescript does not directly control the list of reserved words in ES, the feature would need to be enabled via a new compiler option, such as enableAutoBreakingSwitch
Perhaps propose that the name chosen be nominated for reserved-word status in ESNext+1. But that seems highly optimistic.

Specific behavior
The auto-breaking switch will compile exactly as if the developer had added 'break;' as the last statement in each case block.

TypeScript design goals:

Avoid adding expression-level syntax.

While this might be a handy feature, the right way to champion it in ECMAScript.  Especially when it comes to things that don't have anything to do with types, the TypeScript team has been very reluctant to introduce new expression level syntax.


DeprecationWarning: node --inspect --debug-brk is deprecated

Since VSCode version 1.15.0 F5 (starting debug) prints out the following message:
(node:31245) [DEP0062] DeprecationWarning: `node --inspect --debug-brk` is deprecated. Please use `node --inspect-brk` instead.

Any ideas how to prevent this warning?

Project/System details:

VSCode Version: 1.15.0 (8b95971d8cccd3afd86b35d4a0e098c189294ff2)
OS Version: Ubuntu 16.04.3 LTS
node Version: v8.3.0

Steps to Reproduce:

see https://github.com/ManfredSteiner/typescript/tree/master/console
start in debugging mode (F5)

Reproduces without extensions: Yes

duplicate of https://github.com/Microsoft/vscode/issues/28223

Nice to see that it's closed...
But what's the solution to prevent printing this warning?
Maybe I am blind, but I cannot see any hint in the referenced issue.

Let me try to summarise this for you:
Currently there is no way to prevent this.
The reason why we use the combination of "--inspect --debug-brk" is clearly explained in the (indirectly) linked issue https://github.com/Microsoft/vscode/issues/27731:

This combination of args is the only way to enter debug mode across all node versions. At some point I'll switch to inspect-brk if we don't want to support node 6.x anymore, or will do version detection for it and do something for runtimeExecutable scenarios.

@weinand, thank you for summarizing. Maybe it is possible to check node version and to use the right combination, or to add an additional config option in launch.json.

As advice to googlers coming here through google on this deprecation warning (this page ranks 1st or 2nd...). This feature now exists!

ensure your Node Version is ≥ 8.0

node --version

change in your launch.json from

< "protocol": "legacy",
> "protocol": "auto",

Problem solved (=feature activated). On your next run...
Debugging with inspector protocol because Node.js v8.4.0 was detected...

Since "protocol": "auto" is the default anyway, there is no need to use an explicit protocol attribute.


Release

Coverage decreased (-0.5%) to 94.834% when pulling 84751a20d679f9f49fe75c95c44c72bd19f15a9d on develop into 3efab6866a17bbfe46b1dae5720e2b405d59c743 on master.

:tada: This PR is included in version 1.1.0 :tada:
The release is available on:

GitHub release
npm package (@latest dist-tag)

Your semantic-release bot :package::rocket:


Add sass dependency and handlers

Add sass dependency only

need to investigate what this is changing with types. Seeing a warning thrown with the migrations history table


[User Story] As a system developer, I want the user to be able to use their mobile phone or tablet to access and interact with every functional component of the web application.

Is your user story related to a problem? Please describe.
I'm frustrated when I can't use my web application on my mobile phone or tablet.
Describe the solution you'd like
I want my web application to be able to access every page and interact with every functional component through a mobile phone or tablet.
Describe alternatives you've considered
There is no suitable alternative to this.
Additional context
No further context is necessary.

Mobile functionality is continuous issue. All current features work, all future features should work as well.


updated the profile of jourdan-templeton

Docs Build status updates of commit a3d93a9:
:clock10: Pending: waiting for processors (14 builds ahead of you)
:warning: Docs Build is busy, currently there are 14 builds ahead of this one, for more information you can view the Build queue graph on the Docs Portal.

Docs Build status updates of commit a3d93a9:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

rd/jourdan-templeton.yml
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Jurst incorrectly flags unused variables when nulling objects

As far as I can guess, variables still need to be nulled in Jurst, but Jurst gives a warning "The assignment to local variable t is never used."

No, Jurst basically has the same semantics as Wurst. It is just a different frontend which tries to look like vjass.

So nulling variables isn't necessary, thanks.


Linker cannot find spdlog

Issue
Linker cannot find spdlog
Output
[build]   spdlog.vcxproj -> C:\Dev\Iridescent\build\Debug\spdlogd.lib
[build]   Core.vcxproj -> C:\Dev\Iridescent\build\Debug\Core.dll
[build]      Creating library C:/Dev/Iridescent/build/Debug/Sandbox.lib and object C:/Dev/Iridescent/build/Debug/Sandbox.exp
[build] main.obj : error LNK2001: unresolved external symbol "private: static class std::shared_ptr<class spdlog::logger> Iridescent::Log::s_CoreLogger" (?s_CoreLogger@Log@Iridescent@@0V?$shared_ptr@Vlogger@spdlog@@@std@@A) [C:\Dev\Iridescent\build\Sandbox\Sandbox.vcxproj]
[build] main.obj : error LNK2001: unresolved external symbol "private: static class std::shared_ptr<class spdlog::logger> Iridescent::Log::s_ClientLogger" (?s_ClientLogger@Log@Iridescent@@0V?$shared_ptr@Vlogger@spdlog@@@std@@A) [C:\Dev\Iridescent\build\Sandbox\Sandbox.vcxproj]
[build] C:\Dev\Iridescent\build\Debug\Sandbox.exe : fatal error LNK1120: 2 unresolved externals [C:\Dev\Iridescent\build\Sandbox\Sandbox.vcxproj]

Possible fixes

Spdlog library is a header only library, meaning it is not 'compiled' into the DLL itself. For this reason, the Sandbox project should include the spdlog library as an include path.

Conclusion
Figure out how to fix the linking in cmake!

Core now compiles as static, which fixes the issue temporarily


error: Liquid syntax error (line 3): Unknown tag 'bibliography' on github.io

Hello,
This theme attracts me a lot. But when I want to use it by setting it as my github page, it reports this error. I have no idea to solve this problem.
Could you help me? I will really appreciate your help!

Please use the script for deployment: bin/deploy. Also, see discussion in #6.


refs best practises

I think there should be a way to work with refs somehow. For sure, refs are problematic in stateless components, but let give a try. For now I have a following ~antipattern~ workaround:
const Stateless = ({ refs }) => (

    <Foo ref={r => refs.store('myFoo', r)} />

 refs.myFoo.doSomeStuff()}>
      doSomeStuff

)

class RefsStore {
  store(name, value) {
    this[name] = value;
  }
}

const enhancer = compose(
  withProps({ refs: new RefsStore() }),
)(Stateless);

With this we create a wrapper, to store refs. Something similar to withState, but it doesn't trigger component re-render. This code work, but I'm not sure, if it's really ok, from idiomatic point of view. If it ok, we could probably create a new enhancer withRefs. If it's not ok, how should we work with this?

Hi, here I wrote IMO about refs https://github.com/acdlite/recompose/pull/272

Related to #319. We can add this part in the cookbook section.


Allow invoke helper to work with tgz files.

A zip or tgz (more generally archive format) is specific to a runtime. Allow this script to work with zip or tgz files.
See https://github.com/apache/incubator-openwhisk-runtime-nodejs/pull/16 for example.

We should add a test "should invoke python package with tgz"

That will require changes in the runtimes and should be tested in those repos. This just makes the helper script work with zip or tgz.

Ah missed that it was the helper/dev script


Updated Leaderboard API controllers2

Added missing API calls in controllers.

PR is merging into master 😬

@NitronBiohazard pinging for visibility

Changed to be into development instead of master

LeaderboardApiController is not calling the LeaderboardService @NitronBiohazard

Fixed to call the LeaderboardService from the LeaderboardApiController.


Nested pipes

📖 Documentation
For peace of mind, I would love to know whether nested pipes are ok or not.
export const deleteContent: Endomorphism<StateWithSelection> = state =>
  pipe(
    state,
    // state.element
    elementLens.modify(
      pipe(
        state.selection,
        mapSelectionToRange,
        deleteContentElement,
      ),
    ),
    select(collapseSelectionToStart(state.selection)),
  );

This looks weird as well. It is idiomatic fp-ts code or not?
export const select = (selection: Selection): Endomorphism<Value> => value =>
  pipe(
    value.selection,
    exists(s => eqSelection.equals(s, selection)),
  )
    ? value
    : { ...value, selection: some(selection) };

I am writing this question here in order to improve docs. I will send PR.

I am pretty sure I am doing something wrong. But what?
export const setText = (
  text: string,
  selection?: Selection,
): Endomorphism<Value> => value =>
  pipe(
    fromNullable(selection),
    chain(() => value.selection),
    fold(
      () => value,
      selection =>
        pipe(
          value,
          elementLens.modify(setTextElement(text, selection)),
        ),
    ),
  );

How to kill this pyramid of doom?
export const deleteContent: Endomorphism<Value> = value =>
  pipe(
    value.selection,
    fold(
      () => value,
      selection =>
        pipe(
          value,
          elementLens.modify(
            pipe(
              selection,
              mapSelectionToRange,
              deleteContentElement,
            ),
          ),
          select(collapseSelectionToStart(selection)),
        ),
    ),
  );

If I'm not mistaken, setText and deleteContent can be refactored by removing the fold and replacing it with a map and then getOrElse
export const setText = (
  text: string,
  selection?: Selection,
): Endomorphism<Value> => value =>
  pipe(
    fromNullable(selection),
    chain(() => value.selection),
    map(selection =>
      pipe(
        value,
        elementLens.modify(setTextElement(text, selection)),
      )),
    getOrElse(() => value)
  );

Anyway, you still need the other pipes. In order to improve the readability and expressiveness of the code you can extract some of the "nested" logic into functions.
const modifyElementText = (text: string, value: Value) => (s: Selection): Selection =>
  pipe(
    value,
    elementLens.modify(setTextElement(text, selection)),
  )

export const setText = (
  text: string,
  selection?: Selection,
): Endomorphism<Value> => value =>
  pipe(
    fromNullable(selection),
    chain(() => value.selection),
    map(modifyElementText(text, value)),
    getOrElse(() => value)
  );

Note that:
a => pipe(a, ...)

is the same as:
flow(...)

The rule of thumb is that if a can be inferred, you can probably substitute pipe for flow.

@pfgray Well, I wouldn't agree with that rule of thumb. IMO it bloats API and increases maintenance complexity. If you ever need to get access to hidden argument (for example to parametrise some operator in the flow) you end either adding extra lambdas or rewriting flow to pipe. This in turn leads to a messy code filled with a mix of flow and pipe.
On the other hand after migration to classless/pipeable an idiomatic and universal way to work with fp-ts is to use pipe.
We've been battle-testing fp-ts since 0.x with different teams on different projects and API surface has always been a problem for us in terms of documentation, best practices, idiomatic approaches etc. Therefore I would recommend to always stick to pipe because it's universal and to use flow if and only if you know what's you are doing and have strong pros for it besides point-free and code size.

I got it. Thank you for answers.


Emacs 29 breaks many packages

Describe the bug
The title is self-explicative. When the default Emacs is set to 29, some packages don't build.
Steps To Reproduce

set default Emacs handles to their 29 versions
try to build neomutt
look and admire

Expected behavior
A seamless installation.
Additional context
Reports by @\FlorianFranzen [^1] and @\miangraham [^2]
[^1] https://github.com/NixOS/nixpkgs/pull/246203#issuecomment-1666598095
[^2] https://github.com/NixOS/nixpkgs/pull/247357#issuecomment-1666805334
Notify maintainers

Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
output here

For packages inside elisp-packages, it will be harder because it will require manual overriding.


Update example for new turf-hex-grid api

Ref #11

@jseppi fyi, docs are now updated in the jsdocs at the top of turf modules. To regenerate the readme.md file, run npm run doc. This allows us to generate the website and the readmes from 1 definitive source.

Oh right, whoops!


Is share_textvqa is the same as textvqa in llava dataset?

https://tinyllava-factory.readthedocs.io/en/latest/Prepare Datasets.html#organize-data
from the description here :

TextVQA: Already download as “TextVQA” in LLaVA’s SFT Images.

No. they are not the same. share_textvqa can be get from here https://drive.google.com/drive/folders/1tCUQ-sq6vdshZVkF0ZeF3K4eztkXJgax according to our Doc.


Showing Toast After ActionSheet

The toast does not seem to appear after dismissing an ActionSheet.  Latest iOS (10.3.1), iPhone.  Are you seeing the same behavior?

Hi czuvich, i have not run into this yet. Can you add a minimalistic reproduction sample?


Changes in deployment file and sa

Coverage remained the same at 86.16% when pulling f552cc310b7a040a7be21b3b3c91490da11bf2a2 on ambiknai:deptest into 5031bf73af43e4c9d20299b8a366d5b593818df8 on kubernetes-sigs:master.


PERF: Get rid of unnececssary allocation in VarReader5.cread_fieldnames

This pull request is like #12519 but for the C memory allocation functions.

On Linux calloc is almost free until you write to it.
In [4]: %timeit numpy.zeros(10**8)
7.28 µs ± 6.05 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

In [5]: %timeit numpy.ones(10**8)
68.2 ms ± 137 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

The OS uses virtual pages that are pre-zeroed.

CI is green and core dev approval--the change may not provide much performance improvement (as Chuck notes) but it does seem to simplify the code a bit in any case.
Thanks @alexhenrie & reviewers


Apache Kafka integration

Any ideas how to integrate with Apache Kafka topics.

@polya20 we haven't added the support for Kafka topic yet. But per my understanding with Kafka, I think if you have schema registry deployed on your end, you could topic metadata info from there and persisted in Amundsen. But you also need to build UI for that.

Not sure if schema-registry will be a valid source as in latest versions schemas are not anymore uniquely linked to one topic https://docs.confluent.io/current/schema-registry/serializer-formatter.html#sr-avro-subject-name-strategy.
I'd prefer a kafka-topic-metadata-extractor that uses Admin client to gather topic metadata.
Would this work with current data model, mapping a kafka topic as a table? or is a stream entity support coming in later versions?

@jeqo I think it needs a new model for kafka topic .

How would the Kafka topic model look like?
Here is a list of metadata that pops into my heads.

topics
schemas
offset info
partitions
consumer & producer info

And here are some ideas for the frontend and databuilder.
Frontend

Instead of date range used for hive partitions, maybe frontend can show offset range and such for Kafka topics.
Kafka consumers as table-users and producers as table-owners?

Databuilder
Databuilder will periodically gather data via Kafka Admin client and gather metadata. If we don't use schema-registry, the topics' schema will have to be inferred from the latest offset. If the data structured like Protobuf or Avro, inferring schema won't be too hard. But if some Kafka topics use unstructured data like text or CSV, we might have to pull data from schema-registry.

@jacobhjkim are you suggesting to reuse the table dataset support for Kafka topic?

also cc @lukelowery as the team from reddit expresses interests in supporting Kafka topic in Amundsen.

Like this http://schema-registry-ui.demo.lenses.io/ or https://www.apicur.io/registry, it would be superpowerful for event-driven schema register. And all the database schema can be express in schema register..

Hi guys, any update on the progress of this feature ?

I don't think it is just a matter of schema registry. There are some caveats that need to be taken into consideration like:

What is the source of the topic? A JDBC table?
Who uses the topic? Snowflake? Neo4j?
and others...
Because this could also help to reconcile data lineage for those workflows that use Kafka in order to move data between data sources;
I guess that a good approach would be managing not only the Schema-Registry, but also the whole Kafka Connect platform that basically allows to:
push the data to a Kafka Topic from any arbitrary data source via a module called Source
ingest the data from a topic into any data source via module called Sink
manipulate each topic message via Transforms

Why was this issue closed??
+1

We have changed the way we manage issues, you can read more here: https://amundsenworkspace.slack.com/archives/CGGHH0XSB/p1671130550809429
If you are interested on this issue being tackle, please react with 👍 in the original issue post!


Where do I find the jar files

I would want to use the drivers/jars/packages to read excel files. Where can I find them to download?

If you look at the releases section the jar files are attached with each version.
I'll add something to the README to make that more apparent 🙂

I've updated the README to include the download instructions as described previously. Let me know if you're all good and I'll close the issue.

I see the jar files and thank you very much. I'm new to all these. Is there a sample pyspark code to that I can try?

 It needs to be in the classpath so that Spark can find it, how you do this depends on which platform you're using.
If you're using Databricks then have at their library docs. If it's Azure Synapse then their docs cover how to install a new library.
If you're running locally and using pyspark then you can do the following:
$ pyspark --jars file1.jar,file2.jar

Or you can use the spark configuration (see spark.jars)

daz - Thank you very much. I was able to test it successfully in Azure Databricks as well as SQLServer 2019 Big Data Cluster.
In Databricks, I was able to load the FAT Jar via the library module and use it directly
On SQLServer BDC, I had to download the FAT Jar into the HDFS drive and use CONFIG to refer to the JAR.
%%configure -f
{"conf": {"spark.jars": "hdfs:///modules/jar/elastacloud/spark-excel-3.1.2_0.1.7.jar"}
}

df = spark.read.format("com.elastacloud.spark.excel").load("/test/users.xls")

df.show()

Glad you got it working. Also nice to hear it working on a SQL 2019 Big Data Cluster as it's not an environment I've tried it out on.
Hopefully you can get that data out of Excel successfully 🙂


Security issue on Android < 4.4

See http://android-developers.blogspot.com/2013/08/some-securerandom-thoughts.html
Since this library appears to be directly using numbers from a default-initialized SecureRandom, it should not be used on Android versions below 4.4 without a separate intialization step.
Either the fix should be applied in the code here where applicable (via reflection, probably, to avoid borking non-Android environments), or there should be a big disclaimer and a minimum Android recommendation of 4.4 rather than 2.3.3 for using this library.

Thank you for bringing this to my attention. I agree this affects JNCryptor and I'll immediately look into mitigation.

Has any work done on this? I'd love to use this library in LDLN's Android client without a disclaimer on it that devices have to be 4.4+. Perhaps we could help look into nice ways of implementing this fix if it's not been started yet...

Looking at the linked blog post, there is what seems to be a reasonable mitigation available by seeding the PRNG. Is there a problem with the code in that blog post?

I've decided not to address Android issues within this project. I don't have the means to test fixes, so this remains a pure Java implementation.


Pre-req work for bringing SegmentAssignmentStrategy back, issue #8585

Instructions:

The PR has to be tagged with at least one of the following labels (*):

feature
bugfix
performance
ui
backward-incompat
release-notes (**)

Remove these instructions before publishing the PR.

(*) Other labels to consider:

testing
dependencies
docker
kubernetes
observability
security
code-style
extension-point
refactor
cleanup

(**) Use release-notes label for scenarios like:

New configuration options
Deprecation of configurations
Signature changes to public methods/interfaces
New plugins added or old plugins removed

Table config now has the following:
  private Map<InstancePartitionsType, InstanceAssignmentConfig> _instanceAssignmentConfigMap; // existing
  private Map<InstancePartitionsType, SegmentAssignmentConfig> _segmentAssignmentConfigMap; // new

Since SegmentAssignmentConfig contains only the name of the strategy, can't we add it to InstanceAssignmentConfig?

Also SegmentAssignment and SegmentAssignmentStrategy interfaces are pretty similar. It makes the code a bit more complicated than what it is now.
Seunghyun mentioned that Jackie has a design doc for it. If you have that, could you please share it?

one pager on OSS issue for this change
https://docs.google.com/document/d/16sGvC8UPHcaYJvUXHB2wODsaeNYkXj4lAhaJiSk9Jiw/edit

instance assignment
https://docs.google.com/document/d/13nZPPOzmPJPoxmv9XQ7Z9evqGP4DWHIL49N96A6sv7A/edit

Instance Assignment: when creating the table, assign servers to serve it based on the instance assignment strategy. All the segments for this table will only be served by the assigned servers.

Segment Assignment: when adding a new segment, pick some servers from the assigned servers for the table to assign the new segment based on the segment assignment strategy.

I now recall the discussion with Jackie last time. Based on the design, now, the segment assignment now happens in 2 steps after we introduced the pool concept.


Disallow picking out of range month or year in date input dropdown

Proposed Changes:

Fixes #9030
Restrict month/year selection to only valid months and years, preventing out-of-range selections.
Ensure that the navigation buttons are always functional, even when an invalid month/year has been selected.
Add validation to ensure that invalid months or years cannot be selected via the month/year selector.

Merge Checklist:

[ ] Add specs that demonstrate bug and test the new feature.
[ ] Update product documentation.
[ ] Ensure that UI text is kept in I18n files.
[ ] Prepare a screenshot or demo video for the changelog entry, and attach it to the issue.
[ ] Request for Peer Reviews.
[ ] Completion of QA.

Summary by CodeRabbit

New Features

Enhanced date selection logic with improved constraints for year, month, and date.
Added error notifications for invalid selections.

Bug Fixes

Fixed issues with month and year selection to prevent out-of-bounds values.

Improvements

Conditional rendering for date adjustment buttons to improve user experience.

@Rishith25 hoding this PR test , a related issue need to be solved and merge before we test this : https://github.com/ohcnetwork/care_fe/issues/9101

@Rishith25

I want to close this PR because some conflicts have been araised.
I will immediately open a new PR. related to this issue.


Build testimonials page and make minor edits to landing page

Related to #24 and #28

@des-des please hold on before merging/reviewing as we need to move the index.html file outside the html folder so that it is deployable on github pages. I'm doing that now!

@marisid re-assign me when ready

New commits relate to #37


decouple scan from observable

Decoupled scan from observable,
please review.
Thank you,
Grigoriy

Thank you!


Action creation with .jar fails with "The request content was malformed"

1: Create a Java file called Hello.Java from here
import com.google.gson.JsonObject;
public class Hello {
    public static JsonObject main(JsonObject args) {
        String name = "stranger";
        if (args.has("name"))
            name = args.getAsJsonPrimitive("name").getAsString();
        JsonObject response = new JsonObject();
        response.addProperty("greeting", "Hello " + name + "!");
        return response;
    }
}

2: Compile Hello.java into a JAR file hello.jar as follows:
javac Hello.java
jar cvf hello.jar Hello.class

3: Create manifest.yaml with:
package:
    name: helloworldjar
    actions:
        helloworld:
            location: src/hello.jar
            runtime: java
            main: Hello

4: Deployment fails with:
Response body received:
{
  "error": "The request content was malformed:\n'main' must be a string defined in 'exec' for 'java' actions",
  "code": 4001042
}
2017/06/01 12:48:50 Got error creating action with error message: The request content was malformed:
'main' must be a string defined in 'exec' for 'java' actions (code 4001042) and error code: 144.

As part of fixing this issue, please add integration test for JAR.

The same failure is seen when an action is created without --main:
wsk action create helloJava src/hello.jar
error: Unable to create action 'helloJava': The request content was malformed:
'main' must be a string defined in 'exec' for 'java:default' actions (code 6005960)
Run 'wsk --help' for usage.

v/s
wsk action create helloJava src/hello.jar --main Hello
ok: created action helloJava

See PR: https://github.com/apache/incubator-openwhisk-wskdeploy/pull/278


VM/VMSS: Referencing VMSS during VM creation

Resource Provider
What is the Azure resource provider your feature is part of?

Compute

Description of Feature or Work Requested
Provide a brief description of the feature or work requested. A link to conceptual documentation may be helpful too.

Converged API VMSS now allows usage of VMSS in a way similar to availability Sets. Therefore we now need to reference VMSS instead of av-set during VM creation for clients who want to use this. Find internal API docs here: https://microsoft-my.sharepoint.com/%3Aw%3A/g/personal/shpa_microsoft_com1/Ef68QZMoy9BIpvnmq8RDeB8BoSvADf3afeUadvvs49MlBQ?e=gmWhky
https://microsoft-my.sharepoint.com/%3Aw%3A/g/personal/shpa_microsoft_com1/Ea1j9UXmUA5OnMul_N4fFrABjyLHBNjayNArtj3vRgSvCA?e=Xwrx0d

Minimum API Version Required
What is the minimum API version of your service required to implement your feature?

2019-03-01

Swagger Link
Provide a link to the location of your feature(s) in the REST API specs repo. This should be on the master branch of the REST API specs repo.

https://github.com/Azure/azure-rest-api-specs/pull/6836 (needs to be merged yet)

Target Date
If you have a target date for release of this feature/work, please provide it. While we can't guarantee these dates,
it will help us prioritize your request against other requests.

We have received you feature request.

Need traction on this. Tried reaching out to whoever I could find. Perhaps posting here will do the job. @qwordy , @Juliehzl , @yonzhan Please help. We need this for Ignite too and it's been open for a very long time. Thank you! :)

Need traction on this. Tried reaching out to whoever I could find. Perhaps posting here will do the job. @qwordy , @Juliehzl , @yonzhan Please help. We need this for Ignite too and it's been open for a very long time. Thank you! :)

OK. We will prioritize this issue. I have opened a PR, but it has problems. Do you know how to create a valid vmss in Azure CLI to be referenced in vm create? vmss of new type?

Yes, when you create a VMSS, please do not provide any VM profile in the template and that should be acceptable for this one. More details in the email thread I started.

@qwordy are you blocked by some problem ?

I think I need add a new parameter for vmss create to create a such kind of vmss. Now, all control flows in vmss create will define virtualMachineProfile.

Made review comments on that PR. Please reach out to me directly for any confusion.

This request is already planned for Ignite. The PR is https://github.com/Azure/azure-cli/pull/10646, currently reviewed by Jun Wang (AZURE) from Compute team.

Made review comments on that PR. Please reach out to me directly for any confusion.

OK. Thanks!

This feature will be available in Azure CLI 2.0.76. Official Release: 11/04/2019


add rush option on online order form

Policy is
April 1 – December 10

Time of Request
1 Business Day Turnaround
2 Business Day Turnaround
3 Business Day Turnaround

Before Film Arrives in Lab
N/A
Add 100% to order
Add 50% to order

After Film Arrives in Lab
N/A
Add 100% to order
Add 100% to order

December 11 – March 31
During this period, our turnaround is typically 1-2 business days. If this does not meet your urgent time frame, please give us a call, we will do all we can to help. [{{ site.company-info.display-phone }}](tel:{{ site.company-info.phone }})
Em + Ash's ideas for form:
Based on if they access the form between April 1 and December 10 (PST) have the above options show up. And if they access the form between Dec 11-march 31 (PST) take the options away.

Between Apr. 1 and Dec. 10, rush options appear. Otherwise, our "During this period, our service time is typically 1-2 business days. If this does not meet your urgent time frame..." verbiage appears.

Change now merged into master branch (commit b781857)


Local storage lost after updating from 4.1.1 to 5.0.0

Hi, after updating your plugin to version 5.0.0 from 4.1.1 all local storage is lost on Android and iOS
I couldn't find any documentation or existing issue on this topic. What can be done about this?

If the data you stored is important, you should consider another way of saving this data.

iOS stores localStorage data in a location that may be cleaned out by the OS when space is required.

https://cordova.apache.org/docs/en/latest/cordova/storage/storage.html#localstorage


Fix transformer build

I think transformer build is broken on master because of an error in our makefile. FIxing it but firste I'm triggering a rebuild of the transformer to make sure

Oops.
🙈

(This is why I shouldn't edit code on my phone!)

... during leave! We can see you!

I was still in the building when I made that code change yesterday, totes doesn't count


Packaging fix

What was done
Wootric can be used via SPM now.

I was importing Wootric through spm and it wasn't working. Seems this is the fix to make it work!

any progress on this? @diegoserranoa

Bump @diegoserranoa

@diegoserranoa would be great to have this PR reviewed (and merged 🤞) 🚀
thank you

Hello everyone! I'm really sorry that I've been slow to respond. I've been working on other projects and just last week I was able to get back into iOS world. I appreciate your collaboration with this. I create this https://github.com/Wootric/WootricSDK-iOS/pull/102 which includes the changes from this one and other fixes/updates.
Please check it and let me know if I'm missing anything there. Thanks!

Hello everyone! I'm really sorry that I've been slow to respond. I've been working on other projects and just last week I was able to get back into iOS world. I appreciate your collaboration with this. I create this PR #102 which includes the changes from this one and other fixes/updates.
Please check it and let me know if I'm missing anything there. Thanks!

Looks great, thanks!

The changes from this PR were merged here: https://github.com/Wootric/WootricSDK-iOS/pull/102
Closing this PR. Thanks everyone!


PushNotificationIOS Cannot read property 'requestPermissions' of undefined

What is the reason? thanks....

class indexContent extends React.Component {
  constructor(props) {
    super(props);
    this.state = {
      tabIndex: 1,
    };
  }

  componentWillMount() {
    PushNotificationIOS.addEventListener('notification', this._onNotification);
  }

  componentWillUnmount() {
    PushNotificationIOS.removeEventListener('notification', this._onNotification);
  }

  render() {
    PushNotificationIOS.requestPermissions();
    return (
      <TabBarIOS>
        <TabBarIOS.Item
    .......

Resolved. . .
look please, The document did not write

thanks .... this should be added into document ...........

@qingfeng your answer is not clear, what should be added and from where should I add it ?
thanks

Building with Xcode rather than running the command react-native run-ios solved it for me.


Changing file permissions for securityconfig and tools

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Cherry picked from 5f28e80

@vrozov added cherry pick to commit message.


fix: add PUnit/PEmpty to additive dict

This should fix the problems in #1319

We have the same attributes in mathlib3.
bors r+


Hallucinogen-Persisting Perception Disorder

Hallucinogen-Persisting Perception Disorder
According to DSM-5, Hallucinogen Persisting Perception Disorder is caused by intoxication with hallucinogen drugs, especially LSD.  It is defined as the recurrence of perceptive disturbances that firstly develop during intoxication. The contents of the perception and visual imagery range extensively. DSM-5 and previous DSM editions report a list of the most common symptoms experienced by HPPD patients, but only a few symptoms have been described in the professional literature. The main group of symptoms reported by Criterion A of the DSM-5 are visual disturbances. In fact, as in the vast majority of induced psychoses, visual hallucinations are notably more common than auditory (PMID: 29547576)
PMID: 27822679
ICD-10-CM code: F16.183
MSD-5 code 292.89  F16.983
Parent term (Substance use related  disorders  -   Hallucinogen abuse]

I added this as a child of 'perceptual disorders', as the def of Hallucinogen abuse says: A substance abuse that involves the recurring use of hallucinogenic drugs despite negative consequences. -> I think this could occur even after one use of a hallucinogenic drug, it doesn't have to be recurring abuse.

Thank you Nicole!
From: Nicole Vasilevsky<EMAIL_ADDRESS>Sent: Thursday, May 14, 2020 4:48 PM
To: monarch-initiative/mondo<EMAIL_ADDRESS>Cc: Alyea, Gioconda (NIH/NCATS) [C]<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [monarch-initiative/mondo] Hallucinogen-Persisting Perception Disorder (#1508)
I added this as a child of 'perceptual disorders', as the def of Hallucinogen abuse says: A substance abuse that involves the recurring use of hallucinogenic drugs despite negative consequences. -> I think this could occur even after one use of a hallucinogenic drug, it doesn't have to be recurring abuse.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/monarch-initiative/mondo/issues/1508#issuecomment-628878434, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APG3MG7C3MQQBTTF6S6EI2DRRRKH7ANCNFSM4NAJEIRA.


⚠ GitLab Server has degraded performance

In 158b337, GitLab Server (https://gitlab01.its-telekom.eu) experienced degraded performance:

HTTP code: 200
Response time: 6730 ms

Resolved: GitLab Server performance has improved in f6cc95d.


kubernetes dns deploy fails when calico policy_enabled: true

Hey all,
I've deployed k8s with kismatic successfully and wanted to try out enabling calico network policies. I redeployed my bare-metal machines and ran the kismatic installer with cluster:networking:policy_enabled: true. The deployment completes up to deploying the dns pods, but fails when waiting for them to be ready.
Checking the logs on the pods/containers, nothing is remarkable other than in the kubedns container, <IP_ADDRESS> being the master's ip, which is strange as i can reach the api from my browser:
I0405 17:16:59.956279       1 dns.go:174] DNS server not ready, retry in 500 milliseconds
I0405 17:17:00.456228       1 dns.go:174] DNS server not ready, retry in 500 milliseconds
E0405 17:17:00.456401       1 reflector.go:199] k8s.io/dns/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1.Service: Get https://<IP_ADDRESS>:6443/api/v1/services?resourceVersion=0: dial tcp <IP_ADDRESS>:6443: i/o timeout
E0405 17:17:00.456439       1 reflector.go:199] k8s.io/dns/vendor/k8s.io/client-go/tools/cache/reflector.go:94: Failed to list *v1.Endpoints: Get https://<IP_ADDRESS>:6443/api/v1/endpoints?resourceVersion=0: dial tcp <IP_ADDRESS>:6443: i/o timeout
I0405 17:17:00.956070       1 dns.go:174] DNS server not ready, retry in 500 milliseconds
I0405 17:17:01.456136       1 dns.go:174] DNS server not ready, retry in 500 milliseconds

The calico-node container running on the machine trying to run kubedns shows in it's logs:
2017-04-05 17:14:09.790 [INFO][123] calc_graph.go 244: Local endpoint updated id=WorkloadEndpoint(node=daralis, orchestrator=k8s, workload=kube-system.kube-dns-3895116667-jz4hz, name=eth0)
2017-04-05 17:14:09.790 [WARNING][123] active_rules_calculator.go 260: Profile not known or invalid, generating dummy profile that drops all traffic. profileID="k8s_ns.kube-system"
2017-04-05 17:14:09.791 [INFO][123] int_dataplane.go 466: Received *proto.ActiveProfileUpdate update from calculation graph msg=id:<name:"k8s_ns.kube-system" > profile:<inbound_rules:<action:"deny" rule_id:"ZQQ7Duk5sGpQ5vQi" > outbound_rules:<action:"deny" rule_id:"hmRRFH_D_5H_-aEQ" > >

but then shortly after it looks like it's creating/opening the route:
2017-04-05 17:14:09.793 [INFO][123] endpoint_mgr.go 393: Updating endpoint routes. id=proto.WorkloadEndpointID{OrchestratorId:"k8s", WorkloadId:"kube-system.kube-dns-3895116667-jz4hz", EndpointId:"eth0"}
2017-04-05 17:14:09.794 [INFO][123] endpoint_mgr.go 466: Workloads changed, updating dispatch chains.
2017-04-05 17:14:09.794 [INFO][123] table.go 400: Queing deletion of chain. chainName="cali-from-wl-dispatch" ipVersion=0x6 table="filter"
2017-04-05 17:14:09.794 [INFO][123] table.go 627: Invalidating dataplane cache ipVersion=0x6 reason="chain removal" table="filter"
2017-04-05 17:14:09.794 [INFO][123] table.go 400: Queing deletion of chain. chainName="cali-to-wl-dispatch" ipVersion=0x6 table="filter"
2017-04-05 17:14:09.794 [INFO][123] table.go 627: Invalidating dataplane cache ipVersion=0x6 reason="chain removal" table="filter"
2017-04-05 17:14:09.794 [INFO][123] table.go 376: Queueing update of chain. chainName="cali-from-wl-dispatch" ipVersion=0x6 table="filter"
2017-04-05 17:14:09.794 [INFO][123] table.go 627: Invalidating dataplane cache ipVersion=0x6 reason="chain update" table="filter"
2017-04-05 17:14:09.794 [INFO][123] table.go 376: Queueing update of chain. chainName="cali-to-wl-dispatch" ipVersion=0x6 table="filter"
2017-04-05 17:14:09.794 [INFO][123] table.go 627: Invalidating dataplane cache ipVersion=0x6 reason="chain update" table="filter"
2017-04-05 17:14:09.794 [INFO][123] endpoint_mgr.go 692: Applying /proc/sys configuration to interface. ifaceName="cali94f4eef6e0e"
2017-04-05 17:14:09.794 [INFO][123] endpoint_mgr.go 313: Re-evaluated workload endpoint status adminUp=true failed=false known=true operUp=true status="up" workloadEndpointID=proto.WorkloadEndpointID{OrchestratorId:"k8s", WorkloadId:"kube-system.kube-dns-3895116667-jz4hz", EndpointId:"eth0"}
2017-04-05 17:14:09.794 [INFO][123] status_combiner.go 58: Storing endpoint status update ipVersion=0x6 status="up" workload=proto.WorkloadEndpointID{OrchestratorId:"k8s", WorkloadId:"kube-system.kube-dns-3895116667-jz4hz", EndpointId:"eth0"}
2017-04-05 17:14:09.794 [INFO][123] table.go 416: Loading current iptables state and checking it is correct. ipVersion=0x4 table="filter"
2017-04-05 17:14:09.800 [INFO][123] table.go 416: Loading current iptables state and checking it is correct. ipVersion=0x6 table="filter"
2017-04-05 17:14:09.807 [INFO][123] route_table.go 293: Syncing routes: adding new route. ifaceName="cali94f4eef6e0e" ipVersion=0x4 targetCIDR=<IP_ADDRESS>/32
2017-04-05 17:14:09.808 [INFO][123] status_combiner.go 81: Endpoint up for at least one IP version id=proto.WorkloadEndpointID{OrchestratorId:"k8s", WorkloadId:"kube-system.kube-dns-3895116667-jz4hz", EndpointId:"eth0"} ipVersion=0x4 status="up"
2017-04-05 17:14:09.808 [INFO][123] status_combiner.go 98: Reporting combined status. id=proto.WorkloadEndpointID{OrchestratorId:"k8s", WorkloadId:"kube-system.kube-dns-3895116667-jz4hz", EndpointId:"eth0"} status="up"

Do I need some other configuration or am I misunderstanding how to set this up?

We are aware this and are working on a fix. There is an issue with a one of our dependencies described here.

ahh thanks @dkoshkin i'll be tracking it!

Is there a workaround to solve this prolem?

When setting cluster.networking.policy_enabled: false to deploy a node, I encounter the same problem.

@flyer103 it sounds like you are seeing a different issue, do you mind sharing some more details: the kismatic version, OS, where you are running this and the plan file you are using.

OK, I will open a new issue to describe the problem I have encountered :D


Private feeds Part 2

Basics for token based auth.
Changes are additive, not changing current code and never called at this moment (calls should coming with part 3 or 4...)
review: namespaces and locations

Hey thanks for figuring all this out, it's looking good so far! :)
Could you explain the purpose of NugetBehaviorMiddleware? What's the benefit of translating basic auth to JWT?

NuGet Client implementation is NOT following the typical workflow/RFC for Token based security.
NuGet Client has no  (March 2019) support for tokens (a header with AUTH=Bearer).
https://github.com/NuGet/Home/issues/7841
The most common workaround(hack) that i have found is: abuse the basic authentication workflow and send the token using the password field.
on the server side i like to use all the available Token-based staff from aspnetcore team, but this code is not able to handle this "abuse"
my "middleware" does the translation between "NuGet" and "aspnetcore" implementation.
AND
the middleWare should do this Transaltion only in case of a NuGet.client (detected by session ID) to prevent any impact on UI or other implementations that are using token based security in the right way....

Ah okay, so this allows you to support AAD without needing a custom NuGet authentication plugin? That’s pretty smart, good work!

support AAD without needing a custom NuGet authentication plugin
not exactly, but in this direction.

Let me explain:
The middleware has a big impact on how we implement the BaGet Backend
It allows us to use all the assemblies/nuget packages for authentication ("Microsoft.AspNetCore.Authentication*") provided by the AspNetCore Team without hacks and deep customizing -  we can use them in a default way (!!)
What do you mean with "custom NuGet authentication plugin" ?
I'm assuming do you mean a "NuGetCredentialProvider" that must be used/provided on the client side (Nuget.exe)
We need a plugin for AAD that does the complex UI-Workflow for AAD and other token based security workflows.
BUT: our implementation behaves like all the other NuGet Server implementations (MyGet, Nexus, Azure Artifact) so there is a high probability that we can use "NuGetCredentialProviders" implemented by somewhere else because the implementation must support EXACT the same server behavior.....
With this in mind, the current NuGetCredentialProvider for Azure DevOps/Visual Studio/TFS may work for us, we must only customize the address/instance of AAD.....


Failed to call service tapo_control/xxx_xxxxxxxx_xxxx. Service not found.

Description
Since the last updates of HA (Home Assistant 2022.10.5, Supervisor 2022.10.0, Operating System 9.2, Frontend 20221010.0 - latest) and Tapo-Control, I get the following error messages when I click on the privacy mode button.
Failed to call service tapo_control/set_privacy_mode. Service not found.
The camera is a C210 with firmware 1.3.0.
Reproduction Steps
See above!
Expected behavior
The cameras should be able to be controlled in the same way as before.
If applicable, add error logs.
No response
Device Firmware
1.3.0.
Using stream component
Yes
Does camera work via official integrations?
Yes
Camera has all attributes filled out in developer tools
Yes
HASS Environment
Raspberry Pi
Search for similar issues
Yes
Additional information
No response

Search for similar issues
Yes

Try again.

Happy to help! Thanks for wasting my team by being lazy to do 5 second search!

You wasted most of your time with your infantile answers ;-) Therefore, thank you again, I was very amused. :-)


Add the banner to the Login admin page.

Context
As a Admin i can see the banner in the Login Page

already done


Upgrade to ASM 6.1 to work with Java 10

My local testing indicated that the Jacoco Upgrade #2349 was the only change needed to get tests passing in Java 10.  What prompted this?

ASM is used during testing. It might not be needed in our case. But to be on the safe side, ASM 6.1 is indeed the version supporting Java 10 officially.

Test this please.


Harmonise task - subticket 3 - PMID 33414548

6 empty studies:
GCST90010743
GCST90010756
GCST90010808
GCST90010729
GCST90010780
GCST90010753
(also GCST90010744)
It looks like the problem is the p-values are in quotation marks.
I've deleted the harmonised folders on public.
Removed quotation marks from the original raw files on staging. Hopefully will sync to pub overnight. There's no need to recreate formatted files as headers are already in standard pre-SSF format.
All studies have yaml files so should re-enter harmonisation queue.

done

@jiyue1214 to prioritise for hm

not harmonised successfully, @jiyue1214 investigating


Failed to build D3D12 backend with Clion

When I used Clion to build D3D12 backend, it failed.
The compilers said that " fatal error C1083: can not open file: “GenerateMips/GenerateMipsLinearCS.h”: No such file or directory".

https://github.com/DiligentGraphics/DiligentEngine/blob/master/Troubleshooting.md
CLion is not an officially supported build tool. Use Visual Studio.

@TheMostDiligent Any plans on when we can get clean CMake support outside Visual Studio?