Documentation: hidden tooltips prevent the content under them from being selected

Description
I hope if this is the right place for bugs related to the online documentation.
Tooltips that appear when you hover over a question mark in the documentation are hidden using opacity: 0;, which means they still have pointer events. For example, you cannot select the text under where a tooltip would appear, but you can select the content of the tooltip when it is hidden.
A possible fix would be to set pointer-event: none; or use display: none; when a tooltip is hidden. I think the latter might work the best with screen readers.
The text under the tooltip bellow cannot be selected, even when the tooltip is hidden:

Please note that a tooltip has to have been displayed once for the bug to happen with this specific tooltip.
Reproduction URL
E.g., https://typst.app/docs/reference/meta/figure/#parameters-kind
Browsers
No response
OS
No response

Shouldn't this issue be moved to typst/webapp-issues?


Bug 1889956: CVE-2020-8564

I was unable to cherry-pick #193 and had to create a new pull request manually.

/assign @adambkaplan

/retest

/retest
VPC limit exceeded

@bparees somehow Corey was able to green-button merge this. IMO this is a safe thing to keep, QE and patch managers just need to be aware.


Document that module name must be comment

I had been fighting an error that I could not understand until I looked into sources. In module trait the module is hardcoded as "comment" and I had it as comments. I would prefer a Widget setting where we can pass module name or if it have to remain hardcoded then it should clearly be stated to avoid pains like mine.
Thank you for great extensions. I personally appreciate them

Hi, you can change documentation and create PR for this issue.

https://github.com/yii2mod/yii2-comments/pull/70

Thanks


Can this be used with Razor Pages?

Now that Blazor can be mixed and matched with MVC and Razor Pages, it make sense if a system has a single unified localization support. I wonder if this library supports Razor Pages/MVC as well.

Never mind. It works with Razor Pages


Transform front-matter markdown metatada into html

When I put some markdown into metadata, I want get html strings.
---
myfield: |-
    **Title**
    content
myarray:
- title: Title
  description:  |-
      **Title**
      content
---
lorem

Like the phenomic/lib/loader-plugin-markdown-transform-body-property-to-html plugin, we should write a loader-plugin-markdown-transform-metadata-property-to-html plugin

This should not be default, just write a plugin so that user can add it easily.

I want to make this the default behavior. Lot's of people expect this according to feedback on our chat.
But we will have to be careful for things like "title" that must be able to be used in <title>.

Perhaps an option for the default plugin pack because:

processing markdown on each frontmatter attributes slows down the loader
IMHO, I don't expect this at all .

I could be great to make this the default behavior if is not slow down much the build

If we keep the same remark instance, pretty sure this won't be a problem. Anyway, this might be subject for a new plugin included in the default markdown preset, so people might skip it if not needed. Let me make a twitter poll for this https://twitter.com/Phenomic_app/status/800737752794939392

I tried to handle this but when my webpack load the plugin I have the error Unexpected token import for my first import

Try using es5. What phenomic version are you using?

Thank you. I was my phenomic version indeed.
So in my webpack config I did
phenomic: {
      plugins: [
        ...require("phenomic/lib/loader-preset-default").default,
        ...require("phenomic/lib/loader-preset-markdown").default,
        ...require("./src/loader-plugin-transform-md-head-property-to-html").default,
      ]
    },

And that is my plugin code :
import flatten, {unflatten} from "flat"
import remark from "remark"
import slug from "remark-slug"
import autoLinkHeadings from "remark-autolink-headings"
import highlight from "remark-highlight.js"
import toc from "remark-toc"
import html from "remark-html"

function mdify(frontMatter) {
   const flattenHead = flatten(head)
   const flattenHeadMdify = flattenHead.map(prop => remark()
     .use(slug)
     .use(autoLinkHeadings, {
       content: {
         type: "text",
         value: "#",
       },
       linkProperties: {
         className: "phenomic-HeadingAnchor",
       },
     })
     .use(html, { entities: "escape" })
     .use(highlight)
     .use(toc)
     .process(text, {
       commonmark: true,
     })
     .toString()
   )

  return unflatten(flattenHeadMdify)
}

export default ({result, frontMatter}) => ({
  ...result,
  head: mdify(result.head),
})

But I have the error Module build failed: TypeError: plugin is not a function for all my markdown files. What I did wrong ?

If it's for all files, it's because it happen each time the loader is executed. In your case the "plugin" is called by the loader on each file. Your plugin seems good, not sure what is wrong here.
(btw it's suboptimal for 2 reasons: a new remark instance created each time + flatten/unflatten, which should may be handled by a recursive function with a swtich on the typeof data)

Hi! I'm currently writing a static website with phenomic as well and would also like to get markdown for frontmatter items. I get the same error Module build failed: TypeError: plugin is not a function and found that the plugins array passed to the loader plugins contains only null values in case I use plugins: [ ...require("phenomic/lib/loader-preset-markdown").default,], in my webpack config. But looking at the code in the loader plugin, the default is to do exactly the same, in which case the plugins array will contain 5 functions. Any ideas what's happening here?

"plugins" section need to be in the top level webpack config for webpack 1 (that serialize query as string then unserialize, which make function=== null).
Please read carefully this section https://phenomic.io/docs/usage/plugins/#consuming-plugins and tell me if you still have the "null" issue.

Yep, this worked, thanks for the quick reply!

@westlicht If you succeed, can you share your work or just tell me what is wrong my my snippet ? Thank you

@xuopled I just moved the loader plugins section to the top level webpack config, as suggested by @MoOx, because I'm using webpack 1. I'm considering switching to webpack 2 beta, but for now, this should work. The plugin code you wrote did not work as is for me and I will probably rewrite it for my needs early this week. I only want MD to be applied to certain fields, still have to figure out the best way of achieving this, maybe some configuration options or some prefix embedded to the strings that require conversion. I can post the code here once I get it working properly.

I like the idea of font-matter parsing to spit out the key and value for each line so you can determine how you want to process it, and not make it the same for all fields.
Similar to how postCSS plugins work would be ideal in my honest opinion.

Doable in user land.


[DEMO] Thursday demo

depends on https://github.com/input-output-hk/cardano-node/pull/88 (to rebase after it's merged)
updates to latest cardano-shell
makes the SK/cert optional
factors CommonCLI
adds mainnet genesis & proxy follower topology
scripts:

scripts/mainnet-proxy-follower.sh -- to connect to a running cardano-byron-proxy
scripts/3node-demo.sh -- the ex-demo2.sh

TODO

[ ] make systemStart come from genesis for RealPBFT (tricky, needs demo API considerations)
[ ] review remaining unaddressed suggestions from https://github.com/input-output-hk/cardano-node/pull/60

Closed in favour of #91


Text size abnormally large on Android with Font Size on Large in Phone Settings

Description of the Bug
The font size appears VERY BIG when compiled and ran on android where font size in settings is set to large. Is there a way to make kivymd / kivy ignore that?
Code and Logs
from kivy.app import App
from kivy.lang import Builder

kv = """
Screen:
    MDLabel:
        text: "TEST"
"""

class MainApp(App):
    def build(self):
        self.root = Builder.load_string(kv)

if __name__ == '__main__':
    MainApp().run()

Screenshots
Bug: https://user-images.githubusercontent.com/69392733/107875481-77f12280-6ec0-11eb-89d7-cdb41f12221e.jpg
How it should be: https://drive.google.com/file/d/1-ESJocPnKqQifPS1dP7sp9BK5_TpQP4w/view?usp=sharing
Versions

OS: Android 10 & 11 Tested
Python: 3.7.6
Kivy: 2.0.0
KivyMD: 0.104.1

@YarostheLaunchpadder Try testing on other versions of Android devices ...

Tried on multiple phones, same thing!

@YarostheLaunchpadder Can you provide more specific versions of the device and screenshoots?

I think the screenshot is a Samsung device, not my phone (running android 6 or 7), also tested on a Samsung A50, same problem; Google Pixel 4 same thing!

@YarostheLaunchpadder I need screenshots.

@YarostheLaunchpadder You did not understand me. We need screenshots not of one model, but of all on which you tested.

Okay, I will close this issue now because I tried on other phones / devices and only with two devices running lower versions of android this bug happened!


Search through list of stored PINs and check each.

false

https://github.com/project-chip/connectedhomeip/blob/f0f5de0d79ae1c51d7fddbf70900ebac0acd899a/src/app/clusters/door-lock-server/door-lock-server.cpp#L253-L258

This issue was generated by todo-issue based on a TODO comment in f0f5de0d79ae1c51d7fddbf70900ebac0acd899a.

Should be already addressed in the SDK.


Is poly2 always convex?

The API description states that poly2 is always convex:
https://github.com/jscad/OpenJSCAD.org/blob/f5f31084843adebaeb481c064eb09634fb7490a0/packages/modeling/src/geometries/poly2/create.js#L2
But looking in the code, poly2 is created from geom2 outlines, which may not be convex:
https://github.com/jscad/OpenJSCAD.org/blob/f5f31084843adebaeb481c064eb09634fb7490a0/packages/modeling/src/operations/expansions/offsetGeom2.js#L28-L30
Is this a possible bug? Or is the documentation outdated?

Ah… poly2… it was added as part of V2 but is not really 100%. So, I’m not sure where this geometry will be useful. I believe it was added to scope those special functions.
Do you have some use cases?

I was just wondering. I looked into the code for existing parts useful for the implementation of the Minkowski sum. If poly2 isn't guaranteed to be convex, a poly2.isConvex() function would make sense. Some goes for a decompose function to split a non-convex polygon into convex polygons.

Ah. Now, I understand where this is going. Feel free to contribute functionality to poly2. Please follow name and style similar to poly3, if possible. And of course, some rudimentary tests.

@jue89 theres a set of changes coming for poly2. Can you follow and add comments? Or even better... enhancements.


V1 0.0.5

Presubmit test failures appear to be an infrastructure issue, tracked by bug 136591800, which should be easier to fix once the code is merged. The code here is identical to the internal version that passes presubmit. LGTM to merge.


Always more than 10 sec to "down" the container

As can be seen by the image above when running docker-compsoe down, it always takes a bit more than 10 seconds to "down" the container. Other containers are much faster.
Just want to maybe start a discussion on why this is happening? Is it on purpose, is it a bug?

So by default docker is under this rule
Docker has limited patience when it comes to stopping containers. There’s a timeout, which is 10 seconds by default for each container. If even one of your containers does not respond to SIGTERM signals, Docker will wait for 10 seconds at least.

There is a flag at least for docker where timeouts can be lowered, if you need a shorter time to stop them you should look into that one and if docker-compose supports that flag in the config.
But the core issue is bash and that this container runs a bash script that runs all redis servers.
 Starting the server with & will background it into the shell's job control system, with $! holding the server's PID (to be used with wait and kill). Calling wait will then wait for the job with the specified PID (the server) to finish, or for any signals to be fired.

I guess adding in a SIGTERM trap should not be that difficult to exit out the container more correct


Lab 4 - having a hard time pasting my DB id and DB pw into the notebook

Although I can delete characters and manually type characters, the ability to paste is very wonky.  I eventually got it to work, but I don't know why it finally allowed me to paste.

I'm really confused about this one.  I'm on a Mac with Chrome and have never had an issue.  What does your environment look like?

Now the Jupyter script will be asking for input from users.

Tested in Chrome and Safari.


python311Packages.objgraph: 3.6.0 -> 3.6.1

Description of changes
Changelog: https://github.com/mgedmin/objgraph/blob/3.6.1/CHANGES.rst

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 291981 run on x86_64-linux 1

6 packages marked as broken and skipped:

python311Packages.mapbox
python311Packages.mapbox.dist
python312Packages.mapbox
python312Packages.mapbox.dist
spring
springLobby

4 packages failed to build:

cwltool
cwltool.dist
python312Packages.slack-bolt
python312Packages.slack-bolt.dist

127 packages built:

adapta-gtk-theme
arc-theme
arx-libertatis
asciidoc-full
asciidoc-full-with-plugins
asciidoc-full-with-plugins.dist
asciidoc-full.dist
ayu-theme-gtk
breeze-hacked-cursor-theme
capitaine-cursors
ceph (ceph-dev)
ceph-client
ceph-csi
ceph.dev (ceph-dev.dev)
ceph.doc (ceph-dev.doc)
libceph (ceph.lib ,libceph.dev ,libceph.doc ,libceph.lib ,libceph.man)
ceph.man (ceph-dev.man)
cinnamon.cinnamon-screensaver
cinnamon.iso-flags-png-320x420
cinnamon.iso-flags-svg
clevis
clevis.man
coreboot-configurator
dblatexFull
disorderfs
emojione
fim
flexget
flexget.dist
frigate
hackneyed
hikounomizu
home-assistant-component-tests.profiler
inkscape
inkscape-extensions.textext
inkscape-extensions.textext.dist
inkscape-with-extensions
iso-flags
k40-whisperer
kabeljau
rep (kakounePlugins.rep)
luksmeta
mate.mate-utils
material-cursors
mnemosyne
mnemosyne.dist
mojave-gtk-theme
numix-cursor-theme
numix-solarized-gtk-theme
pdm
pdm.dist
pip-audit
pip-audit.dist
plata-theme
poetry
poetry.dist
poetryPlugins.poetry-audit-plugin
poetryPlugins.poetry-audit-plugin.dist
poetryPlugins.poetry-plugin-up
poetryPlugins.poetry-plugin-up.dist
pop-gtk-theme
python311Packages.cachecontrol
python311Packages.cachecontrol.dist
python311Packages.cherrypy
python311Packages.cherrypy-cors
python311Packages.cherrypy-cors.dist
python311Packages.cherrypy.dist
python311Packages.cwl-upgrader
python311Packages.cwl-upgrader.dist
python311Packages.cwl-utils
python311Packages.cwl-utils.dist
python311Packages.diagrams
python311Packages.diagrams.dist
python311Packages.objgraph
python311Packages.objgraph.dist
python311Packages.orange-canvas-core
python311Packages.orange-canvas-core.dist
python311Packages.orange-widget-base
python311Packages.orange-widget-base.dist
python311Packages.orange3
python311Packages.orange3.dist
python311Packages.pyalgotrade
python311Packages.pyalgotrade.dist
python311Packages.pylxd
python311Packages.pylxd.dist
python311Packages.schema-salad
python311Packages.schema-salad.dist
python311Packages.scikit-bio
python311Packages.scikit-bio.dist
python311Packages.slack-bolt
python311Packages.slack-bolt.dist
python311Packages.ws4py
python311Packages.ws4py.dist
python312Packages.cachecontrol
python312Packages.cachecontrol.dist
python312Packages.cherrypy
python312Packages.cherrypy-cors
python312Packages.cherrypy-cors.dist
python312Packages.cherrypy.dist
python312Packages.cwl-upgrader
python312Packages.cwl-upgrader.dist
python312Packages.cwl-utils
python312Packages.cwl-utils.dist
python312Packages.diagrams
python312Packages.diagrams.dist
python312Packages.objgraph
python312Packages.objgraph.dist
python312Packages.pylxd
python312Packages.pylxd.dist
python312Packages.schema-salad
python312Packages.schema-salad.dist
python312Packages.ws4py
python312Packages.ws4py.dist
qemu_full
qemu_full.debug
qemu_full.ga
sabnzbd
sambaFull (samba4Full)
sambaFull.dev (samba4Full.dev)
sambaFull.man (samba4Full.man)
tang
tang.man
tebreak
tebreak.dist
vimix-cursors
volantes-cursors
wpa_supplicant_gui

Done in https://github.com/NixOS/nixpkgs/commit/ff6699ec05088f8ff5cf3fff5fe14465f286b75a?


Generate galaxy catalog on lightcone

After all the galaxies have been assigned to their positions using the core tracking approach, the lightcone code is run again to produce the galaxy lightcone

Lightcone from the  galaxy snapshots was created. Now the last step is to integrate everything and then ingest it into DESCQA2

Just for fun a picture of the galaxy positions in most closely to z=0

Super cool!


Hydration mismatch at {{ "" }} inside v-for

Versions

nuxt: v2.15.18
node: v16.14.0

Reproduction

{{ "" }}Test

Important for me was it in this usecase
{{ index > 0 ", " : "" }}<wbr v-if="index > 0">{{obj.text}}

Steps to reproduce
Use nuxt dev with target: 'static' in nuxt.config
What is Expected?
normal hydration
What is actually happening?
I get a hydration mismatch because nuxt on ssr somehow thinks there is a node with empty content and undefined type and on clientside there correctly no node.
Workaround
use instead:
{{ "" }}Test

resp.
 0">, <wbr v-if="index > 0">Test

Cheers :-)

This is Vue SSR behaviour. I'd recommend raising an issue there instead.


Continue to submit new keys following a positive diagnosis

According to the documentation:
Keys will be reported for the previous 14 days of contact tracing. The app will also be launched every day after the daily tracing key changes to allow it to request again to get the key for each previous day for the next 14 days.

It's unclear though how this subsequent daily launch will work. Permission needs to be granted to retrieve the keys, but this makes it sound like the app will be woken in the background.
I believe this is a missing aspect of their documentation thus far.

Do you think the client will need to persist the submission ID after its initial submission from the server? Otherwise if new keys are uploaded how do you prove that they are associated with a positive test result without going through the whole confirmation process again?

Yes I agree with that. I will update the YAML so it can return such an identifier.
Additionally, the same submit endpoint will be used but with an option to include that identifier instead of re-verifying.

One other thing I noted here, that isn't in Apple's documentation:
https://blog.google/documents/55/Android_Contact_Tracing_API.pdf
In order to be whitelisted to use this API, apps will be required to timestamp
and cryptographically sign the set of keys before delivery to the server
with the signature of an authorized medical authority.

I'm not sure exactly how to handle this on the client side.

I've updated the YAML here in c6573f3d788d49b2ef63b05b1408374d05bfda08.
Note that the iOS app and my server sample just have placeholders for now and aren't actually properly making use of it.

I've updated the YAML in c6573f3 and 866b727

Understood. I've open an issue to track this work https://github.com/tatey/trace_privately/issues/14.

I do have a security concern about just blindly submitted the ID with new keys, since it could potentially be extracted from the Core Data database, but obviously the device needs to be authenticated.
Perhaps the server needs to rate limit the submissions for an existing submission. For example, only accepted an average of 1 key per 24 hours from the date of the original submission, up to X days (14? 21?) until the diagnoses is no longer considered active.
Then in the case of abuse, it's only going to be fake keys that don't match against other users' databases so I don't believe it will actually cause any impact (such as false indicating an exposure).

Opened #37 related to this.

I do have a security concern about just blindly submitted the ID with new keys, since it could potentially be extracted from the Core Data database

Actually, this is a really good point that I overlooked. Right now I'm returning the primary key of the submission which is an auto incrementing ID from the database. Perhaps this should be some sort of token that only the client which made the submission would know. This would stop tampering with other submissions.
The other option is to make it so that you get an account behind the scenes and we limit it so that the account you're authenticating with can only updates its own submissions. I like this one less because then it's linking submissions and keys to an authenticated device.

Yea, I don't think using the auto-incrementing ID is a good idea. In sample implementation it generates a UUID and assigns it to the record and returns that
(Currently it's not storing it in the database, but I've made a note to do so. This would likely be an indexed field).
I don't think an account is necessary. If it's throttled then there's really not going to be much / any effective tampering. Not sure of the key data format from Apple yet, but some validation against that could also occur.

Yea, I don't think using the auto-incrementing ID is a good idea. In sample implementation it generates a UUID and assigns it to the record and returns that

My bad. I missed that. I've patched it so the server conforms to this behaviour.

I don't think an account is necessary. If it's throttled then there's really not going to be much / any effective tampering. Not sure of the key data format from Apple yet, but some validation against that could also occur.

Cool, agreed.


6th section of community page w/signoff

Description
This PR fixes #1271
Notes for Reviewers
Signed commits

[x] Yes, I signed my commits.

Keep up the good work, @ayushsinha25

@ayushsinha25, thank your for your efforts

Should I make all those changes that you have recommended? And a new pr? @thisiskaransgit

Lets consider this pr for merge i guess and raise another issue for enahancements // @thisiskaransgit @vishalvivekm

You've put in a good shift, @ayushsinha25. Well done!

Thanks man!!!

@ayushsinha25
We could not cover this in the last call, Please add it in today's websites call meeting mintes if you would. :)
Let's discuss this today.

merge conflicts @ayushsinha25

hii @ayushsinha25
Let's discuss it on the websites call.
Please add this as an agenda item in the meeting minutes, if you would :)
https://docs.google.com/document/d/1XczAHXVe2FIWPqiF57ospJ43zw5cZQ7ui8mn39v5EvA/edit#heading=h.lohhtewfwima
Make sure to attend tommorow's website call for review

hii @ayushsinha25
Let's discuss it on the websites call.
Please add this as an agenda item in the meeting minutes, if you would :)
https://docs.google.com/document/d/1XczAHXVe2FIWPqiF57ospJ43zw5cZQ7ui8mn39v5EvA/edit#heading=h.lohhtewfwima
Make sure to attend tommorow's website call for review

Ya, sure

@ayushsinha25 please resolve the conflicts so that we can proceed with merging your efforts here.

@ayushsinha25 please resolve the conflicts so that we can proceed with merging your efforts here.

Fine, will do it by Today evening

@ayushsinha25 please resolve the conflicts so that we can proceed with merging your efforts here.

done

Great job on your PR @ayushsinha25! Your efforts and contributions to the project are highly appreciated.
Just wanted to inform you that we will be closing this PR as another one with similar changes has been merged. Please know that this decision does not diminish the value of your work.
Keep up the fantastic work, and we look forward to seeing more of your contributions in the future!


Profiler tools v2

This PR adds the profiler code to the framework, including:

Report analyses
Plugins to trace events
A tutorial to use the profiling tools

LGTM :-)


Terminal

Hello
First: Thank you! Very nice
Could you tell us something about the Distribution / Terminal you use? A Standard Terminal like bash, sh or xterm did not handle DISPLAY AT line-column

It should work in most terminals after compiling with open-cobol.
Did cobc -W -x -o TicTacTOBOL TicTacTOBOL.cbl work to compile the application?
The shell shouldn't impact execution (bash/sh etc...) shouldn't impact execution but the terminal emulator might. I was able to run this on xterm in Ubuntu.

It should work in most terminals after compiling with open-cobol.
Did cobc -W -x -o TicTacTOBOL TicTacTOBOL.cbl work to compile the application?
The shell shouldn't impact execution (bash/sh etc...) shouldn't impact execution but the terminal emulator might. I was able to run this on xterm in Ubuntu.


example_simple - getting CAS Authentication wanted!

Seeing:
CAS Authentication wanted!
You should already have been redirected to the CAS server.  Click here to continue.

Issue due to own php problem.


ciao-image: Add support for "public" images

We need to support public images, these images will be available for all tenants. Below are the initial rules for public images:

Will use a new flag -public  for specifying it in the ciao-cli creating command
Admin can only create public images

Closed with #1123


Question on country

Only histograms are used to show changes over time in the number of peace agreements and violent conflicts in each country and region, not enough to see the overall trend.

I see that. Thank you! The presentation of the overall trend is very important concerning the following analysis. Maybe a line chart is more suitable for the presentation of the trend than histogram. I will use line chart to demonstrate the comparison between the number of all conflicts and the number of all peace agreements signed between 2017 and 2020.


Use dict to preserve order of retrieved URLs for population

While dictionaries are insertion ordered, sets are not. It is better to use a dictionary to preserve the order of dataset URLs read from the usage dashboard so that we get the URLs for submission to the Datalad registry instance in a consistent order across multiple runs of the population.py script.

sounds good


Update domains.list (2o7.Net & omtrdc.net)

Good catch @smed79 :+1:
Unfortunately @FadeMind must approve this as it is the list he maintain :smile_cat:
I'm ready to merge this but if we merge this it will be overwritten by the system because the system get the list from https://github.com/Ultimate-Hosts-Blacklist/2o7.net/blob/48bc96f0b7ea000ffff06c3f9a0787f2e35a71be/info.json#L10
Maybe a PR in the upstream repository may be better @smed79 :+1:
Have a nice day/night and keep up the good work :+1:
Cheers,
Nissar


Add quick-vue-admin (Category:admin-template)

Project：   https://github.com/linzhixian/quick-vue-admin
Doc：https://linzhixian.github.io/quick-vue-admin-document/
Category:admin-template
Most arresting feature: base on config for CRUD
Thanks a lot

@linzhixian Please create a PR if this is still relevant


Fix some remaining issues

Motivation
As a reference to # 57, this PR is aiming at making up some blanks of testing work and fixing some remaining bugs, which is our long-term goal.
Changes
Changes of this PR are more than belows:

Move some public methods to util package
Add a Setter interface for testing
Remove fake.go file in client package and move all stuffs to volumecontroller_test.go
Redefine the meaning Endpoint field from host_ip to host_ip:port
Add some testing files (finished and unfinished)

That's all, any thoughts?

Since there is no comment during these days, I will merge this PR right now.
More bug reports and fixed patchs are welcomed : )


docs: mention that custom-theme-directory must be set

By default, the value of custom-theme-directory in Emacs is "~/.emacs.d". So if a user tried to run the code snippet in the documentation, he would be met with an error saying that "catppuccin" couldn't be loaded.
If we tell the user to install the theme to "~/.emacs.d/themes", then we should also tell him to set custom-theme-directory to that value.

Didn't know that! thanks for fixing it :)


[Discussion] Removal of Azure Functions version 2.0.13759

Discussion issue for removal of Azure Functions version 2.0.13759

This is release is 15 days old. When will it be removed? Why is it being removed?

@DanielLarsenNZ releases are typically replaced by newer versions. That causes them to be removed in favor of a version that brings a super set of their functionality (fully backwards compatible). In this case, this is being replaced by 2.0.13900.

Thanks @fabiocav. I am trying to understand this for a customer who was impacted by a recent release.
Is the replacement version arbitrary? Or do you always replace the current revision - 2. For example there was a release 2.0.13862 that was not a replacement?
I have several Azure Functions in production with FUNCTIONS_EXTENSION_VERSION = ~2 with reported Runtime version 2.0.13759.0. Why don't we wait until all Function hosts are upgraded before removing the version?
I also have one version 2.0.13351.0 which does not have a notice for removal. Does this mean if I set FUNCTIONS_EXTENSION_VERSION = 2.0.13351.0 it will be used instead of a later version?

The removal is not arbitrary. The normal replacement process is new -2, so the current version (superseded by new) is not removed with a regular deployment.
There are exceptions to the rule. If a hotfix is being deployed (which is the case for the latest announcement), the version it patches will be removed. They're essentially equivalent, with the patched version bringing fixes to potential regressions/issues detected with the new deployment.
Unless recommended by support/Azure Functions team, you should try to avoid pinning. If the support or product team makes a pinning recommendation, those versions are flagged and will not be removed until the issue impacting your app is addressed in a later version.
If you pin to a version that does not exist (or is eventually removed), the latest will be used instead.


chore(package): update eslint-plugin-standard to version 2.0.1

https://greenkeeper.io/

Current coverage is 100% (diff: 100%)

Merging #53 into develop will not change coverage

@@           develop   #53   diff @@
====================================
  Files            3     3
  Lines           95    95
  Methods         17    17
  Messages         0     0
  Branches        10    10
====================================
  Hits            95    95
  Misses           0     0
  Partials         0     0

Powered by Codecov. Last update 448f4b6...6d04fb6


Add standard files

What does this PR do?
Adds in standard files like Development.md, releasing.md, support.md etc
Also changes the license here to be apache 2.0 (Thats currently what the cloudformation java plugin thats used to help build some files in this project uses - https://github.com/aws-cloudformation/cloudformation-cli-java-plugin/blob/master/LICENSE)
Motivation
Add in standard files across repos

Will squash and commit after a final approval.
The labeler had an issue where it would only see the first 30 changed PR files, thats fixed now and looks like its working, so this PR should be all set 😄


cmd/cue: could vet emit a warning instead of an error when an import is not used?

Currently cue vet is failing if something like "strings" is imported but not used with this error message:

imported and not used: "strings"

I believe this is painful in practice as you often comments part of your configuration to debug issues. It forces you to comment back and forth your import statements as well.
Could this be a simple warning instead of an error ?

@PierreR I think the right fix here is a tooling one, because we having different error modes adds (cognitive) complexity if nothing else.
FWIW, this is exactly the same behaviour as Go; likewise, unused variables are errors, just as let declarations and aliases are in CUE.
The particular problem you refer to with import statements and commented-out code, is largely addressed in Go by the use of goimports-like behaviour. This adds/removes imports as required. See https://pkg.go.dev/golang.org/x/tools/cmd/goimports for more details.
With #142 we will add similar support within CUE. Hence, taking you example, when commenting out code that required an import, if no other code requires that import it will be removed.
So this is working as intended, and given the fix is really #142 I'll close this for now.


govpp: Use sequence numbers to match request with response

Coverage decreased (-0.1%) to 76.372% when pulling 2cf5e13ea2d15eb7c4dd22fc4a88c6f393765db7 on milanlenco:release-1.1 into db477048c3a1cec9ce4a4d72be050569c3a35ab4 on contiv:release-1.1.


Add multiple definitions

Adds a command/code-action Add Definitions... that will prompt the user to select undefined functions in the current file to add definitions for, and prompt the user for the target file. If a matching source file doesn't already exist, the user can select to create a new file (this invokes Create Matching Source File). Additionally, invoking Create Matching Source File directly will also ask the user if they want to add definitions. If the user selects a function that must be defined in the same file (inline, unspecialized templates, etc.), then it will skip the prompt to select a target file. Create Matching Source File will only show functions that can be defined in the source file.
Add Definitions... is accessible through the Refactor... menu or command palette.

Note to self: Fix bug when generating constructors and when generating in an empty namespace.


When trying to send email

Command lines that fail are as followings
./bin/nodemailer.js<EMAIL_ADDRESS><EMAIL_ADDRESS>"Hello World" -j "Testing" -n
./bin/nodemailer.js<EMAIL_ADDRESS><EMAIL_ADDRESS>"Hello World" -j "Testing"
crypto.js:91
this._handle.init(hmac, toBuf(key));
^
TypeError: Key must be a buffer
at TypeError (native)
at new Hmac (crypto.js:91:16)
at Object.Hmac (crypto.js:89:12)
at SMTPConnection._actionAUTH_CRAM_MD5 (/home/james/tmp/x/nodemailer-cli/node_modules/smtp-connection/src/smtp-connection.js:802:27)
at SMTPConnection. (/home/james/tmp/x/nodemailer-cli/node_modules/smtp-connection/src/smtp-connection.js:249:22)
at SMTPConnection._processResponse (/home/james/tmp/x/nodemailer-cli/node_modules/smtp-connection/src/smtp-connection.js:516:16)
at SMTPConnection._onData (/home/james/tmp/x/nodemailer-cli/node_modules/smtp-connection/src/smtp-connection.js:353:10)
at emitOne (events.js:96:13)
at Socket.emit (events.js:188:7)
at readableAddChunk (_stream_readable.js:176:18)

This utility hasn't been updated in over two years; I'm wondering if it's an issue with recent node versions. Can you let me know what version of Node.js you are on? (to see this, run node --version.) I personally don't use this utility any more, but will see if it's an easy fix.

…and actually, are you specifying a username, password, and server? it's not going to work without those.

Node version is v6.7.0. It will run fine with the later version of nodemailer just not with the cli.
It does not seem to matter if I specific a server or not. I assume it defaults to localhost but I tried passing -s localhost to it anyway. The username / password isn't required for a localhost relay.
But I tried it anyway with a real server with server / username / password on the command line. Instead I get an auth rejected for valid credentials with
{ Error: Invalid login sequence while waiting for "235": 535 5.7.0 authentication failed
at SMTPConnection._formatError (/home/james/tmp/x/nodemailer-cli/node_modules/smtp-connection/src/smtp-connection.js:384:15)
at SMTPConnection._actionAUTH_CRAM_MD5_PASS (/home/james/tmp/x/nodemailer-cli/node_modules/smtp-connection/src/smtp-connection.js:826:30)
at SMTPConnection. (/home/james/tmp/x/nodemailer-cli/node_modules/smtp-connection/src/smtp-connection.js:810:14)
at SMTPConnection._processResponse (/home/james/tmp/x/nodemailer-cli/node_modules/smtp-connection/src/smtp-connection.js:516:16)
at SMTPConnection._onData (/home/james/tmp/x/nodemailer-cli/node_modules/smtp-connection/src/smtp-connection.js:353:10)
at emitOne (events.js:96:13)
at Socket.emit (events.js:188:7)
at readableAddChunk (_stream_readable.js:176:18)
at Socket.Readable.push (_stream_readable.js:134:10)
at TCP.onread (net.js:543:20)
code: 'EAUTH',
response: '535 5.7.0 authentication failed',
responseCode: 535 }
If you don't use / maintain this utility any more or don't have an interest in keeping it up to date. I might volunteer take it off your hands for you if you like.

Sure, I'd happily add you as a contributor on github and npm if you have the time. If you have a fix please file a pull request, and I'll add you afterwards. Thanks tons! I really appreciate the offer.

I sent a pull request which fixes my immediate issue. Which is nodemailer trying to push undefined as a username / password.
The deeper issue I also have with another server is that it won't support MD5 auth as the password hashes are not stored that way and wants a PLAIN auth method used instead since smtp doesn't have the hash method and most client's won't support it either. But its is still ok to use plain when TLS/SSL is in use since its encrypted anyway.
So more pull requests to probably follow. Kept the first one small to see how it goes..
Also there is a bunch of argument changes for nodemailer for ssl / tls options that will need fixed when updating the package to the later version. Will probably tackle this issue next.

Awesome; merged that and will close this issue. Will be publishing that as a patch version shortly. Also, sent you an invite as a collaborator on the project. Big thanks!

Are you happy for me to commit directly to the repo. Or do you want me
to keep sending pull requests or I can work on a branch instead if you
would preferred.
I was going to bring the deps up to date and also fix the other tls /
ssl issues over the next few days.
I will of course try not to break anything!
On Mon, 2016-10-03 at 11:37 -0700, Nathan Wittstock wrote:

Awesome; merged that and will close this issue. Will be publishing
that as a patch version shortly. Also, sent you an invite as a
collaborator on the project. Big thanks!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

@mistralol please submit pull requests; i'll always make sure to get reviews on them within a few days. Travis CI is running on the repository so that way we'll see failures on a PR rather than having them hit master.
When you do the dependency updates, it'd be great to get the latest node versions added to the Travis config so we can ensure things are working on newer node (right now only 0.10 is tested because of the age of this util). Thank you!

(oh, i will say though that i always just commit changelog updates to the README and version bumps directly to master, since the npm tooling makes that convenient; so def feel free to do that)


Makefile: Fix install.* prerequisites

Without this change, hitting these targets directly will fail.  For example:
$ make clean
$ make MANDIR=/tmp install.man
install  -d -m 755 /tmp/man5
install  -d -m 755 /tmp/man8
install  -m 644 docs/crio.conf.5 -t /tmp/man5
install: cannot stat 'docs/crio.conf.5': No such file or directory
make: *** [Makefile:150: install.man] Error 1

LGTM


Revert math::round_half_to_even

This caused quite a bump in CPU usage, which would be acceptable if this actually improved sound quality. However, it turns out that this function only has one decimal precision, i.e. it would consider all values from 0.50..0.60 (exclusive) as 0.5 which is in
error for our purposes.

I introduced this myself in 8062bd25185c0205a890cadce37661167a187b5a. I'll merge when the CI passes and finally close this chapter.

This makes my poor little Pi Zero Happy,lol!!!

Out of interest, did you look at what this crate was doing? It's horrible and surely inaudible for round(99.9999%) of cases.

Well I didn't look good enough, that's for sure. Sorry for the (non-audible) noise.


[Custom Viz] Show timing details between steps in a funnel

Description
It would be useful to be able to track the amount of time users are spending between steps in a funnel. The specific ask was for the quickstart value stream Map to track time being spent between clicks from the Public Catalog through to installing a quickstart, but this would be useful if it was made more generic so that any funnel query can be inputted.
The metrics we are looking for are:

Average time spent between step clicks
Minimum time spent between step clicks
Maximum time spent between step clicks

Acceptance Criteria

[x] A custom viz that accepts a funnel NRQL and displays Avg/Min/Max time spent between each step click

Here is the current Visualization. JV is updating with docs and we need to move it to newrelic-experimental:
https://github.com/rudouglas/value-stream-funnel


allocate capacity in advance

Please provide a description of this PR:
This is to improve mem usage
To help us figure out who should review this PR, please put an X in all the areas that this PR affects.

[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

Please check any characteristics that apply to this pull request.

[x] Does not have any user-facing changes. This may include CLI changes, API changes, behavior changes, performance improvements, etc.

@hzxuzhonghu: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

release-notes_istio
9e55863edbeecdb6c99efb484142243a16f0a4d9
link
true
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@hzxuzhonghu: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

release-notes_istio
9e55863edbeecdb6c99efb484142243a16f0a4d9
link
true
/test release-notes_istio

integ-cni-k8s-tests_istio
9e55863edbeecdb6c99efb484142243a16f0a4d9
link
true
/test integ-cni-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test release-notes_istio


Google Assistant turns Tradfri light off when asked for a color

Home Assistant release (hass --version):
0.56.0
Python release (python3 --version):
Python 3.6.3
Component/platform:
Google Assistant/Tradfri
Description of problem:
When asking Google Assistant to set a color to a Tradfri warm/cold bulb, the light is turned off, or is not turned on if it is already off. This happens with all colors I have tried, including "cool white".
Expected:
If the color is in range, the bulb should be set to it, and if the color is not available, the light should at least stay on.
Problem-relevant configuration.yaml entries and steps to reproduce:

Configure Google Assistant
Ask Google Assistant to set bulb to cool white
Light turns off or doesn't turn on if it was already off

Traceback (if applicable):
Even with logger set to debug, there is not much to see.
2017-10-22 13:42:01 INFO (MainThread) [homeassistant.components.http] Serving /api/google_assistant to <IP_ADDRESS> (auth: False)
2017-10-22 13:42:01 INFO (MainThread) [homeassistant.core] Bus:Handling <Event call_service[L]: domain=light, service=turn_off, service_data=entity_id=light.cuisine, service_call_id=140689166097768-12>

Additional info:
I only have Tradfri warm/cold bulbs, which is why I have added Tradfri in the title and component/platform. I do not know if the same issue happens with colored bulbs.

I see the same thing.
Looking more at the code it seems the default command is TURN_OFF if a specific command isn't supported.
COMMAND_COLOR is defined in const.py but not yet implemented.
https://github.com/home-assistant/home-assistant/blob/9d20a53d631d774a569a0dc657a9699cf9e3930d/homeassistant/components/google_assistant/smart_home.py#L161
Debug data first row is when setting up the device and second when the request comes in.
2017-10-22 15:07:05 DEBUG (MainThread) [homeassistant.components.google_assistant.smart_home] {'willReportState': False, 'traits': ['action.devices.traits.OnOff', 'action.devices.traits.Brightness', 'action.devices.traits.ColorTemperature'], 'name': {'name': 'Ceiling'}, 'type': 'action.devices.types.LIGHT', 'id': 'light.vardagsrummet_taket'}
2017-10-22 15:09:51 DEBUG (MainThread) [homeassistant.components.google_assistant.http] {'inputs': [{'payload': {'commands': [{'devices': [{'id': 'light.vardagsrummet_taket'}], 'execution': [{'params': {'color': {'spectrumRGB': 16776960, 'name': 'yellow'}}, 'command': 'action.devices.commands.ColorAbsolute'}]}]}, 'intent': 'action.devices.EXECUTE'}], 'requestId': 'xxx'}```

I noticed the same thing when trying to change color with Osram RGB bulbs as well.

Yes, its fixed in https://github.com/home-assistant/home-assistant/issues/10039


[QUESTION] 404 error

I'm on CentOS 7 and have the following installation:
PuppetDB Yum package: puppetdb-6.13.1-1.el7.noarch
Puppetboard Pip packages:
Package      Version
------------ ---------
certifi      2020.12.5
chardet      4.0.0
click        7.1.2
commonmark   0.9.1
Flask        1.1.2
Flask-WTF    0.14.3
idna         2.10
itsdangerous 1.1.0
Jinja2       2.11.3
MarkupSafe   1.1.1
pip          9.0.3
puppetboard  2.2.0
pypuppetdb   2.2.0
requests     2.25.1
setuptools   39.2.0
urllib3      1.26.3
virtualenv   15.1.0
Werkzeug     1.0.1
WTForms      2.3.3

When pointing the Puppet server to the PuppetDB server, I noticed these POST requests:
<IP_ADDRESS> - - [10/Feb/2021:14:11:14 +0100] "POST /pdb/cmd/v1?checksum=cc6b8f538b18118b170558b44fa84d309d66fd15&version=9&certname=<fqdn1>&command=replace_catalog&producer-timestamp=2021-02-10T13:11:14.703Z HTTP/1.1" 200 53 "-" "Apache-HttpAsyncClient/4.1.4 (Java/11.0.8)" 11
<IP_ADDRESS> - - [10/Feb/2021:14:11:15 +0100] "POST /pdb/cmd/v1?checksum=1584cb0518a17092ca49d3f7f50362c870c2ed90&version=5&certname=<fqdn2>&command=replace_facts&producer-timestamp=2021-02-10T13:11:15.077Z HTTP/1.1" 200 53 "-" "Apache-HttpAsyncClient/4.1.4 (Java/11.0.8)" 8
...etc...

Since the result is 200, I expect that all is fine and that the PuppetDB contains a couple of nodes. I read that I may get a 404 when the PuppetDB is empty, so I doublechecked:
>>> from pypuppetdb import connect
>>> db = connect()
>>> nodes = db.nodes()
>>> for node in nodes:
...     print(node)
...
<fqdn1>
<fqdn2>
<..etc..>

For configuring Puppetboard I've followed "Apache + mod_wsgi" on this page, except for different file locations: https://pypi.org/project/puppetboard/
/etc/puppetlabs/puppetboard/settings.py:
PUPPETDB_HOST = 'localhost'
PUPPETDB_PORT = 8080
PUPPETDB_PROTO = 'http'
SECRET_KEY = 'my super secret key'
LOGLEVEL = 'debug'
DEFAULT_ENVIRONMENT = 'linux'

(I'm connecting to localhost, so I'd preferably not bother with https, and no I don't normally use the debug loglevel.)
/var/www/puppetboard/wsgi.py:
from __future__ import absolute_import
import os

# Needed if a settings.py file exists
os.environ['PUPPETBOARD_SETTINGS'] = '/etc/puppetlabs/puppetboard/settings.py'
from puppetboard.app import app as application

In /etc/httpd/conf.d/10-puppetboard.conf:
<VirtualHost *:80>
    ServerName puppetboard2.netmatch.eu

    WSGIDaemonProcess puppetboard user=apache group=apache threads=5
    WSGIScriptAlias / /var/www/puppetboard/wsgi.py

    ErrorLog logs/puppetboard-error_log
    CustomLog logs/puppetboard-access_log combined

    Alias /static /usr/local/lib/python3.6/site-packages/puppetboard/static
    <Directory /usr/local/lib/python3.6/site-packages/puppetboard/static>
        Satisfy Any
        Allow from all
    </Directory>

    <Directory /usr/local/lib/python3.6/site-packages/puppetboard>
        WSGIProcessGroup puppetboard
        WSGIApplicationGroup %{GLOBAL}
        Require all granted
    </Directory>
</VirtualHost>

Apache says it's WSGI enabled. I'm using this module: rh-python36-mod_wsgi-4.5.18-1.el7.x86_64
# apachectl -M | grep wsgi
 wsgi_module (shared)

Apache config:
LoadModule wsgi_module /opt/rh/httpd24/root/usr/lib64/httpd/modules/mod_rh-python36-wsgi.so
<IfModule mod_wsgi.c>
  WSGISocketPrefix /var/run/wsgi
</IfModule>

Each time I request the Puppetboard page I get a 404, and this is logged by Apache:
==> puppetboard-error_log <==
[Wed Feb 10 14:45:37.572988 2021] [wsgi:error] [pid 13163] [client 192.168.x.y:41836] DEBUG:pypuppetdb.api:_query called with endpoint: environments, path: None, query: None, limit: None, offset: None, summarize_by None, count_by None, count_filter: None, payload: None
[Wed Feb 10 14:45:37.573101 2021] [wsgi:error] [pid 13163] [client 192.168.x.y:41836] DEBUG:pypuppetdb.api:_url called with endpoint: environments and path: None
[Wed Feb 10 14:45:37.574600 2021] [wsgi:error] [pid 13163] [client 192.168.x.y:41836] DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): localhost:8080
[Wed Feb 10 14:45:37.588188 2021] [wsgi:error] [pid 13163] [client 192.168.x.y:41836] DEBUG:urllib3.connectionpool:http://localhost:8080 "GET /pdb/query/v4/environments HTTP/1.1" 200 63

==> puppetboard-access_log <==
192.168.x.y - - [10/Feb/2021:14:45:37 +0100] "GET / HTTP/1.1" 404 232 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:56.0) Gecko/20100101 Firefox/56.0 Waterfox/56.4"

I verified that all required/configured files exist on the configured locations, and that they are readable.
Can anyone hint me on what's wrong or how to proceed to find out what's wrong?
Thanks in advance.

It looks like you're using the default environment as DEFAULT_ENVIRONMENT = 'linux' do you have any nodes actively reporting in to that environment?
I've noticed that if Puppetboard can't see any nodes in a given environment it will give a 404 (or sometimes and Internal Server Error on Puppet 7.5+)
Try adding an /env/ on the end of your URL where env = an environment with nodes that are actively talking to PuppetDB.
For example (assuming that your Puppetboard is on the localhost) http://<IP_ADDRESS>:8000/dev/ or http://<IP_ADDRESS>:8000/production/

Isn't this de facto the same as #515 ?

This should be fixed in v3.6.0, released today. Please comment if it is not, @storm49152.


Failed to run convert_checkpoint.py for Qwen2-7B using TRTLLM  0.12.0.dev2024080600

Version info:
image tag of triton server:  24.08-trtllm-python-py3
the version of trt-llm: 0.12.0.dev2024080600
LLM spec: Qwen2-7B, downloaded from: https://huggingface.co/Qwen/Qwen2-7B
command from here: https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/qwen
/tensorrtllm_backend/examples/qwen# python3 /tensorrtllm_backend/examples/qwen/convert_checkpoint.py 	--model_dir /qwen2-7b 	--output_dir /engines/tllm_checkpoint_8gpu 	--dtype float16 	--tp_size 8
Hardware spec: L20 X 8, this is why I specified the tp_size = 8, for I want to fully utilize the 8 gpu;
error output:

Traceback (most recent call last):
File "/tensorrtllm_backend/examples/qwen/convert_checkpoint.py", line 308, in
main()
File "/tensorrtllm_backend/examples/qwen/convert_checkpoint.py", line 300, in main
convert_and_save_hf(args)
File "/tensorrtllm_backend/examples/qwen/convert_checkpoint.py", line 256, in convert_and_save_hf
execute(args.workers, [convert_and_save_rank] * world_size, args)
File "/tensorrtllm_backend/examples/qwen/convert_checkpoint.py", line 263, in execute
f(args, rank)
File "/tensorrtllm_backend/examples/qwen/convert_checkpoint.py", line 246, in convert_and_save_rank
qwen = QWenForCausalLM.from_hugging_face(
File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/models/qwen/model.py", line 315, in from_hugging_face
weights = load_weights_from_hf_model(hf_model, config)
File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/models/qwen/convert.py", line 1221, in load_weights_from_hf_model
weights = convert_hf_qwen(
File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/models/qwen/convert.py", line 756, in convert_hf_qwen
k_bias = dup_kv_weight(k_bias, num_key_value_heads,
File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/models/qwen/convert.py", line 526, in dup_kv_weight
v.shape[1])
IndexError: tuple index out of range
However, if I use tp_size=4, the error is gone.
Can anyone explain why I can't use tp_size=8 here?
Thank you.

Hi @sdecoder , thank you for the feedback. Due to the TP split logic, currently we don't support the situation when num_attention_heads % tp_size != 0. Since qwen2-7B's num_attention_heads is 28, I would recommend using TP=4 or TP=2.

Hi @sdecoder , thank you for the feedback. Due to the TP split logic, currently we don't support the situation when num_attention_heads % tp_size != 0. Since qwen2-7B's num_attention_heads is 28, I would recommend using TP=4 or TP=2.

Sure, thank you so much for your answer.
However, could you please give me a hint if I want to fully leverage all my 8 L20 GPUs?
For example, if I use TP=4, what data-parallelism size is recommended in such case?
Thanks again for any hint here.

Hi @sdecoder, since you have 8 GPUs, I would recommend you trying a bigger Qwen2 model like the 72B one for better quality. But if you just want to maximize the throughput with the 7B model, I think the best way is to deploy multiple instances to avoid extra TP communication. You can refer to this doc for multi instance deployment guide.

Hi @sdecoder, since you have 8 GPUs, I would recommend you trying a bigger Qwen2 model like the 72B one for better quality. But if you just want to maximize the throughput with the 7B model, I think the best way is to deploy multiple instances to avoid extra TP communication. You can refer to this doc for multi instance deployment guide.

I am going to exam the doc you provided.
Thank you so much for your help! 👍


Update ngModel.js documentation

ngModel documentation says that $setViewValue doesn't trigger a '$digest' but its does. it triggers $apply.

I signed it!

Hey @akhlesh thanks for the PR. The change is not completely correct, though. First, an $apply triggers a $digest, so this is basically correct.
The second part is a bit more complex. $setViewValue will only trigger a $digest if you haven't specified specific updateOn triggers, like blur. If you have specified a debounce it'll be triggered only after the debounce.
Do you think you could incorporate this information into the PR?

@Narretz so $setviewValue always calls $digest at some point of time.
sure i'll make this change in my PR.


podvm-mkosi: let 'make binaries' works on RHEL with podman

remove the useless docker buildx use default command
fix Error: unrecognized command podman buildx use
with podman version 4.9.4-rhel on RHEL 9.4

With this one line delete. I can run make binaries from rhel 9.4 machine.
[root@a3elp66 podvm-mkosi]# make binaries
Building binaries...
rm -rf ./resources/binaries-tree
cp -rf ../../../.git ../../.git
docker buildx build \
	--build-arg BUILDER_IMG=fedora-binaries-builder-s390x \
	--build-arg ATTESTER=none \
	 \
	-o type=local,dest="./resources/binaries-tree" \
	-f ../podvm/Dockerfile.podvm_binaries.fedora ../../
WARN[0000] missing "CLOUD_PROVIDER" build argument. Try adding "--build-arg CLOUD_PROVIDER=<VALUE>" to the command line
...
   Compiling oci-distribution v0.10.0
   Compiling sigstore v0.8.0 (https://github.com/sigstore/sigstore-rs.git?rev=d5ba303#d5ba3031)
   Compiling kata-agent v0.1.0 (/src/kata-containers/src/agent)
...

@liudalibj does the command still work on ubuntu for docker?

@liudalibj does the command still work on ubuntu for docker?

If we use docker buildx plugin in ubuntu this command works.
I assume it's use less, the docker buildx instance do NOT need be reset at here.

Reason for that line was #1489 (comment), cc @mkulke

Thanks @katexochen , @mkulke if we need run this command in some dev env, can we put the command out of the make binaries task? maybe run the command by hand at prepare dev env part?

@stevenhorsman Are we in code freeze window now as we are doing new release, can we merge code to main branch now?

@stevenhorsman Are we in code freeze window now as we are doing new release, can we merge code to main branch now?

We aren't in code freeze just yet, so low risk changes can still be merged.


Update version in core.py to 1.0.1

Seems like version update in core.py got missed in commit https://github.com/kennethreitz/legit/commit/7345d702aa0ad56ae92d450dcac2991d2db35f7a. :sparkles:
Also, shouldn't we have version number in one file i.e. either only in setup.py or in core.py and import in setup.py.
:octocat: Thanks

shouldn't we have version number in one file i.e. either only in setup.py or in core.py and import in setup.py

Good idea.
I think maybe defining version in core.py and setup.py can import it.


Change the AssertSubscriber items list back to copyOnWrite

This changes the AssertSubscriber items list back tyo CopyOnWriteArrayList.
The synchronized list is not very practical to write assertions with concurrent access to the list. It requires that the list access is also synchronized on the list.

Do you need a fast-tracked patch release for Reactive Messaging?


Allowing styles to be const

On most MCUs, const objects can be stored in flash rather than occuping precious RAM. A typical LVGL application has numerous lv_style_t objects, which are rather large and do not change during program execution, and are thus a prime candidates to be made const. Storing these styles in RAM seems unecessary, but as the code stands, it's currently not possible to make them const.
For an application recently, I made my styles const and modified all of the functions that accept lv_style_t types (e.g. lv_xxx_set_style etc.) to be const, but after the program crashed I quickly discovered that in (at least) one area of the code, the style in use is temporarily modified.
Specifically in lv_page_design, the page background style is modified for a few specific drawing operations and then reverted:
/*Draw without border*/
lv_coord_t border_width_tmp =  style->body.border.width;
style->body.border.width = 0;
lv_draw_rect(&page->coords, mask, style, lv_obj_get_opa_scale(page));
style->body.border.width = border_width_tmp;
...
/*Draw only a border*/
lv_style_t * style = lv_page_get_style(page, LV_PAGE_STYLE_BG);
lv_coord_t shadow_width_tmp =  style->body.shadow.width;
uint8_t empty_tmp =  style->body.empty;
style->body.shadow.width = 0;
style->body.empty = 1;
lv_draw_rect(&page->coords, mask, style, lv_obj_get_opa_scale(page));
style->body.shadow.width = shadow_width_tmp;
style->body.empty = empty_tmp;

As far as I can tell, this function is the only one I noted that modifies the styles, but there may be others.
I'm wondering what it would take to avoid this and thus allow all styles to be immutable. This could free all of the RAM used by styles and would also have the added benefit of not needing to deal with guarding this if multi-threading support is added at some point.
Temporarily copying the style would work, but seems like overkill. Modifying the style before calling lv_draw_rect doesn't really feel like the right solution in the first place, but I'm not familiar enough with the painting workflow to really understand why it's required and the possible alternatives.
With my current changes, I can make all styles except LV_PAGE_STYLE_BG const, but it's a bit of a hack. @kisvegabor, any thoughts?

Temporarily copying the style would work, but seems like overkill.

A better solution, in my opinion, would be to regenerate the extra modified styles every time lv_page_set_style is called and keep them in the object's ext_attr area. This would avoid the need for a temporary copy.
As far as I can tell the cost of the extra two lv_style_t objects per page would be offset significantly by the removal of all other default lv_style_t objects from RAM.

As far as I can tell the cost of the extra two lv_style_t objects per page would be offset significantly by the removal of all other default lv_style_t objects from RAM.

You could get even away with a single mutable copy of the LV_PAGE_STYLE_BG style and modify it in the same way as it's done currently. It's just one or two instructions during the call, which is probably preferable to the extra copy.
As you said, the RAM savings here are all of the built in styles, plus any user created ones, minus one style for each live page object. It's quite significant.

@canardos One issue I see is that the way themes are initialized would need to be redesigned. Here's an example:
https://github.com/littlevgl/lvgl/blob/master/lv_themes/lv_theme_default.c
This kind of inheritance wouldn't work with constant styles.

You're right that the initialization code would need to be changed for the built in styles to take advantage (i.e. get them out of RAM), but it wouldn't be necessary to make those changes initially if it's an issue.
As a first step, the function parameters could be made const and the changes discussed above made to the page component. That would at least allow (but not require) custom or other styles to be const and avoid them taking up RAM. As a second step, the built-in styles/themes could be migrated. The const function parameter additions are good practice regardless.
With respect to the built-in styles/themes, most of my code is C++, where I am able to copy, modify, and assign existing styles to new static const vars at compile time, but to my knowledge this is not possible in C. Initializing static objects in C require constant expressions and existing const objects wont work.
I'm not a C expert, but I would think C99 designated initializers in combination with defines for shared values is probably the cleanest and most maintainable method. It would result in more verbose source code for the style definitions, but smaller compiled code (and quicker startup) compared with the current code, since they would now just be static objects in the .text/.rodata section, requiring no copying/initialization on startup.

@canardos Sounds like a good plan. Would you like to make a pull request that changes the parameters to enforce const?

@canardos It's a good idea to make styles and lv_xxx_set_style parameters const.
In case of lv_page, I see no problem with copying the style into a local variable. I've already done this with several objects.
I agree to deal with themes later once this basic step is done.
So please send a PR! The whole source code was changed in dev-6.0 because code formatter was run so don't forget to pull the lates version.

Will do.
With regard to copying to a local variable, did you mean making a copy on the stack in the  lv_page_design function, or a permanent copy that remains with the object in its attributes?

Will do.

Great!

With regard to copying to a local variable, did you mean making a copy on the stack in the lv_page_design function, or a permanent copy that remains with the object in its attributes?

A copy on the stack is enough in this case. The computational overhead is smaller then the memory overhead would be.

A copy on the stack is enough in this case. The computational overhead is smaller then the memory overhead would be.

I agree. Will try to get it submitted this week.

The PR is merged so I close this issue.
Thank you very much!


Weird behaviour with .read_csv(header=0)

So I was testing both a pandas and a dask solution for an assignment of parsing a CSV file, when I found a weird behavior with dask.read_csv().
A) Let's assume we have the next CSV file (input.csv) with 13 unique rows.
1 city1,date,193 < This 193 value will be important going forward
2 city2,date,14
(...)
13 city13,date,18
Given this CSV file, we run this script
df = dd.read_csv("input.csv", header=0, names=["city", "date", "sales"], usecols=["city", "sales"], blocksize=16 * 1024 * 1024)
result = df.groupby("city").sum()
result = result.compute().to_csv("./output.csv")
This give us an output CSV file with 2 columns, city and sales, and 12 rows. The CSV has not header info in line 1, and because of the (header=0) we're "skipping" the first line and city1 is missing. It was my error to put the header=0 there, but everything is working as intended here.
But groupby().sum() in the second line is not doing anything since we just have 13, unique rows. So I replicated those lines 780k times, ending with a 10.140.000 rows file.
This is where things start to get messy.
Remember, city1 have a sales value = 193. Given the new input file, the output file should have a value for city1 = 150539807. This is 193 * 779.999 (We have the same rows 780k times, but we're skipping the first line, which is a city1 row)
However, the result for city1 in the output file is 150542123. So it's looks like if additional 12 rows for city1 are being parsed.
What if instead of 780k times, I repeat the rows half of it (390k times)? The additional rows also halts, and additional 6 rows are parsed.
Halts again, repeat the rows 195k times, and value for city1 is 37634807, which is exactly 193 * 194.999, cause first rows is skipped. So issue disappears at this point.
But if I replicate the rows 194.999/998 times, or even adds up to 195.001/002 times, first line is not skipped and we also have an additional row in total count.
All other 12 cities values are always, in all cases, what they should be.
Why is this happening? It's really weird.
If I run the same script but just with pandas, the first line is always skipped and total count for the city rows is exactly what should be.
Environment:

Dask version: 2022.10.0
Python version: 3.11.0
Operating System: Windows 10.0.19043
Install method (conda, pip, source): pip

@cryptohabitat -- Here's an attempt at a reproducer.
import pandas as pd
import dask.dataframe as dd
from dask.dataframe.utils import assert_eq

# create your csv file
pdf = pd.DataFrame({1: ["city1", "date", 193],
                    2: ["city2", "date", 14],
                    3: ["city3", "date", 15],
                    4: ["city4", "date", 16],
                    5: ["city5", "date", 19],
                    6: ["city6", "date", 22],
                    7: ["city7", "date", 25],
                    8: ["city8", "date", 28],
                    9: ["city9", "date", 29],
                    10: ["city10", "date", 30],
                    11: ["city11", "date", 31],
                    12: ["city12", "date", 32],
                    13: ["city13", "date", 22],
                  )
df = pdf.T
df.columns=["city", "date", "sales"]
dfs = 780_000*[df]
df = pd.concat(dfs, axis=0)
df.to_csv("input.csv")

# Then load with both dask and pandas
df = pd.read_csv("input.csv", usecols=["city", "sales"], header=0)
df = df.groupby("city").sum()

ddf = dd.read_csv("input.csv", usecols=["city", "sales"], header=0, blocksize=16 * 1024 * 1024)
ddf = ddf.groupby("city").sum().compute()
assert_eq(ddf, df)

This return True for me on python==3.9.13 and dask==2022.10.1
Also tested on python=3.11 and dask==2022.10.1 and it passes.
Did I miss something in your reproducer?

@cryptohabitat -- that's for the report. I'm having a little trouble following exactly what's going on here. But here's my take:

if you use header=None instead of header=0, everything seems to work as expected
However, if you do use header=0 (which is an easy mistake to make), and when you have a CSV with some large number of rows, it appears that dask is artificially adding additional rows/partitions/something

Do I have that right?
Could I ask you to re-write your example where City1 has a value of 100 or 10 so that these number are easier to track in your issue?
Things working correctly with `header=None`
from io import StringIO
import pandas as pd
import dask.dataframe as dd

csv = StringIO("""\
1 city1,date,193
2 city2,date,14
13 city13,date,18
""")

N = 780_000
_df = pd.read_csv(csv, header=None)
df = pd.concat([_df for _ in range(N)], ignore_index=True)
df.to_csv("temp.csv", index=False, header=False)

pdf = pd.read_csv("temp.csv", header=None, names=["city", "date", "sales"], usecols=["city", "sales"])

ddf = dd.read_csv("temp.csv", header=None, names=["city", "date", "sales"],
                usecols=["city", "sales"], blocksize=16 * 1024 * 1024)
res = ddf.groupby("city").sum().compute()
print(res)
assert(res.loc["1 city1", "sales"] == 193 * N)

I've been doing some more test around this.
First of all, according to pandas docs (v 1.5.1) for header :

'Default behaviour is to infer the column names: If no names are passed the behaviour is identical to header=0 and columns name are inferred from the first line of the file. If column names are passed explicitly then the behaviour is identical to header=None. Explicitly pass header=0 to be able to replace existing names.'

So, just to be clear, the problem kicks in when you're using header=0 and names=[] to replace the header info, but no header is being provided on line 1.

(Sorry, I closed the issue on a missclick. Re-opened with my answers and new code to replicate)
I've been doing some more test around this.
First of all, according to pandas docs (v 1.5.1) for header :

'Default behaviour is to infer the column names: If no names are passed the behaviour is identical to header=0 and columns name are inferred from the first line of the file. If column names are passed explicitly then the behaviour is identical to header=None. Explicitly pass header=0 to be able to replace existing names.'

So, just to be clear, the problem kicks in when you're using header=0 and names=[] to replace the header info, but no header is being provided on line 1.
The reason why @hayesgb script is ending with True when comparing the 2 data frames is because your input.CSV have header info in line 1. If header info exist in line one then there's no problem.
Now, about the @phobson answer, yes, you are right. Given the example, pandas is actually parsing the file correctly. It just skip the first line, so city1 = value * 779.999 (Because it skipped it that first time). However, dask is adding additionals 'something', because value for city1 have ~10 additional row.
Here's the code to replicate both cases, including CSV file. I'm using 10 as initial value for city1's sales for it to be more "traceable"

Code
from io import StringIO
import pandas as pd
import dask.dataframe as dd
from dask.dataframe.utils import assert_eq

# create your csv file W/O HEADERS on line 1
csv = StringIO("""\
city1,1992-09-13,10
city2,1992-09-13,14
city3,1992-09-13,98
city4,1992-09-13,13
city5,1992-09-13,45
city6,1992-09-13,64
city7,1992-09-13,72
city8,1992-09-13,12
city9,1992-09-13,31
city10,1992-09-13,27
city11,1992-09-13,46
city12,1992-09-13,51
city13,1992-09-13,80
""")

N = 780_000
_df = pd.read_csv(csv, header=None)
df = pd.concat([_df for _ in range(N)], ignore_index=True)
df.to_csv("input.csv", index=False, header=False)

#Load with pandas
df = pd.read_csv("input.csv", header=0, names=["city", "date", "sales"], usecols=["city", "sales"], chunksize=100000)
chunks = [chunk.groupby('city').sum() for chunk in df]
df = pd.concat(chunks).groupby(level=0).sum()

#Load with dask
ddf = dd.read_csv("input.csv", header=0, names=["city", "date", "sales"], usecols=["city", "sales"], blocksize=16 * 1024 * 1024)
ddf = ddf.groupby("city").sum().compute()

#Compare both df.
print(assert_eq(ddf, df))```
</details>

There is definitely a bug.  I modify a bit the reproducer  and try multiple things, and something is not right:
from io import StringIO
import pandas as pd
import dask.dataframe as dd
from dask.dataframe.utils import assert_eq

# create your csv file W/O HEADERS on line 1
csv = StringIO("""\
city1,1992-09-13,10
city2,1992-09-13,14
city3,1992-09-13,98
city4,1992-09-13,13
city5,1992-09-13,45
city6,1992-09-13,64
city7,1992-09-13,72
city8,1992-09-13,12
city9,1992-09-13,31
city10,1992-09-13,27
city11,1992-09-13,46
city12,1992-09-13,51
city13,1992-09-13,80
""")

N = 780_000 #>>>> we'll get back to this number
_df = pd.read_csv(csv, header=None)
df = pd.concat([_df for _ in range(N)], ignore_index=True)
df.to_csv("input.csv", index=False, header=False)

#Load with pandas
df_orig = pd.read_csv("input.csv", header=0, names=["city", "date", "sales"], usecols=["city", "sales"])
df = df_orig.groupby("city").agg("sum")

#Load with dask no blocksize provided
ddf_no_bs = dd.read_csv("input.csv", header=0, names=["city", "date", "sales"], usecols=["city", "sales"])
ddf1 = ddf_no_bs.groupby("city").sum().compute()

#Load with dask with blocksize
ddf_w_bs= dd.read_csv("input.csv", header=0, names=["city", "date", "sales"], usecols=["city", "sales"], blocksize=16 * 1024 * 1024)
ddf2 = ddf_w_bs.groupby("city").sum().compute()

This is what I found

N= 780_000
Only city 1 has different values, and in the 3 cases is different. But the rest of the cities all match

>>> all(df.sales[1:] == df.sales_dask_no_bs[1:])
True
>>>all(df.sales[1:] == df.sales_dask_w_bs[1:])
True

For N=200_000

The discrepancy is still in city1 but only when we use blocksize. The rest of the cities are the same.

For N=100_000
Everything matches.

header = None, has no problems.

I'm not sure what's going on, but something is up!
Maybe @rjzamora you might have an idea of what could be going on here?

@ncclementi

This is what I found

N= 780_000
Only city 1 has different values, and in the 3 cases is different. But the rest of the cities all match

Yes, only city 1, cause that's the row that is having the "weird" interaction with the header=0 "targeting" at it in line 1.

(...) but only when we use blocksize.

Maybe there's something happening at every block/chunk when using blocksize.
So I just checked every block by saving them in separates output files.
- N = 780_000
ddf = dd.read_csv("input.csv", header=0, names=["city", "date", "sales"], usecols=["city", "sales"], blocksize=16 * 1024 * 1024)
outputs = ddf.to_csv('blocks/part_*.csv', index=False, header=False)

We have 11 blocks as a result. And there's the problem:

part_00.csv starts with city2 (because the first row is skipped) and ends with city1. So the first block have the same number of rows for each city.
part_01.csv starts with city1 again (Why?) and ends also with city1 < +1 additional row.

The same behavior in part_01.csv appears up until part_10. There we have our 10 additional rows.
The thing is, for N = 780_000, delta between pandas and dask is 11 rows. So 10 of those rows are being additionally parsed, but there's 1 extra row I don't know where is coming from.
For N = 200_000 we have just 3 blocks:

part_00.csv starts with city2 (because the first row is skipped) and ends with city6.
part_01.csv starts with city1 again and then keep counting from city7, so we have an additional row for city1. Same happens with part_02.

For N = 100_000 we just have 1 block that start with city2 and ends with city13, so there are not additional rows being parsed
For what is worth, looks like line 1 is being added to every block/partition as header except for the first one, and then being parsed (even with header=0).
BUT why I didn't have problems with N = 195.000 while I did for N = 195.001/194.999? Didn't check partitions for N = those values. Will continue in the morning.

The unexpected behavior can all be explained by the header=0 argument. Since we think (incorrrectly) that the 0th row is a header, we prepend a copy of that row to the byte range used for every other partition in the dataset.  Partitions 1+ then end up including the copied row as "real" data for some reason. Still trying to figure out what the correct logic is here. I do know that the bug is resolved if we avoid dropping the header option when names is specified.

That's what I thought when checked each partition.
I'm not too familiar with the library code, but will check it later today. In any case, if header=0, names=[] then the header row should be what user pass in names. First row should be dropped, no?

I'm not too familiar with the library code, but will check it later today. In any case, if header=0, names=[] then the header row should be what user pass in names. Header row should be dropped, no?

Yes, I think that's right. The header should be dropped, or the header=0 setting should be preserved for every partition. The code can be a bit difficult to follow in some places, but what Dask is trying to do fhere is pretty simple:

Figure out which row (if any) corresponds to the header in the very first partition/byte-range
Copy the "header" row, and prepend it to all other partitions/byte-ranges
Modify the kwargs for all other partitions (after the first), to treat the first row as a header. In main, this is done by using the default header= argument in the final pd.read_csv call (rather than the one passed in by the user to dd.read_csv)

This approach usually makes sense, but there is obviously a bug for the case that the user passes in both header=0 and names=[...]. The problem is that the default behavior for pd.read_csv is to "infer" what the header should be, and this inferrence depends on the names setting (if names is specified, the first row will be treated as real data - leading to bad results!).


pop up modal set up page

no configuration
modal with drop down to demo data
option to add data like set up page

draft in branch modal-plus-url-option


Why remove parallelization for bifacial.py?

Previous versions of pvlib had a run_parallel_calculations=True option in bifacial.pvfactors_timeseries(). Why was this removed in the current version?
Ideally I'd like to see parallel calculations restored.
The default could be to set run_parallel_calculations=False, which might help with avoiding runtime errors.

Hey @toddkarin, my understanding is that the parallelization is not necessary or desirable anymore because of improved vectorization in pvfactors -- see the discussion here: https://github.com/pvlib/pvlib-python/pull/934#issuecomment-606029318
Is that inconsistent with your experience?  If you're seeing a difference, we might need to take another look at that decision.
This is another time that #419 would be handy.

Thanks for the reference to #934. My main problem was that the change broke my code.
Unfortunately, I can't easily benchmark the parallel code because I've installed pvfactors 1.4.1. So, let's just call this good.


Run databricks task locally

TL;DR
Submit databricks job from local, and save intermediate data in the blob store.
Type

[ ] Bug Fix
[x] Feature
[ ] Plugin

Are all requirements met?

[ ] Code completed
[ ] Smoke tested
[ ] Unit tests added
[ ] Code documentation added
[ ] Any pending items have an associated Issue

Complete description

import datetime
import os
import random
from operator import add

from click.testing import CliRunner

import flytekit
from flytekit import Resources, Secret, task, workflow, ImageSpec
from flytekit.clis.sdk_in_container import pyflyte
from flytekitplugins.spark import Databricks

SECRET_GROUP = "token-info"
SECRET_NAME = "token_secret"

image = ImageSpec(base_image="pingsutw/databricks:v4", registry="pingsutw")

@task(
    task_config=Databricks(
        # this configuration is applied to the spark cluster
        spark_conf={
            "spark.driver.memory": "600M",
            "spark.executor.memory": "600M",
            "spark.executor.cores": "1",
            "spark.executor.instances": "1",
            "spark.driver.cores": "1",
        },
        executor_path="/databricks/python3/bin/python",
        applications_path="local:///usr/local/bin/entrypoint.py",
        databricks_conf={
            "run_name": "flytekit databricks plugin example",
            "new_cluster": {
                "spark_version": "13.3.x-scala2.12",
                "node_type_id": "m6i.large",  # TODO: test m6i.large, i3.xlarge
                "num_workers": 3,
                "aws_attributes": {
                    "availability": "ON_DEMAND",
                    "instance_profile_arn": "arn:aws:iam::479331373192:instance-profile/databricks-agent",
                    "ebs_volume_type": "GENERAL_PURPOSE_SSD",
                    "ebs_volume_count": 1,
                    "ebs_volume_size": 100,
                },
            },
            "timeout_seconds": 3600,
            "max_retries": 1,
        },
        databricks_instance="dbc-2889d011-7c0b.cloud.databricks.com",
    ),
    limits=Resources(mem="2000M"),
    # container_image=image,
    container_image="pingsutw/databricks:v7"
)
def hello_spark(partitions: int) -> float:
    print("Starting Spark with Partitions: {}".format(partitions))

    n = 100000 * partitions
    sess = flytekit.current_context().spark_session
    count = (
        sess.sparkContext.parallelize(range(1, n + 1), partitions).map(f).reduce(add)
    )
    pi_val = 4.0 * count / n
    print("Pi val is :{}".format(pi_val))
    return pi_val

def f(_):
    x = random.random() * 2 - 1
    y = random.random() * 2 - 1
    return 1 if x**2 + y**2 <= 1 else 0

@task(cache_version="1")
def print_every_time(value_to_print: float, date_triggered: datetime.datetime) -> int:
    print("My printed value: {} @ {}".format(value_to_print, date_triggered))
    return 1

@workflow
def wf(
    triggered_date: datetime.datetime = datetime.datetime.now(),
) -> float:
    """
    Using the workflow is still as any other workflow. As image is a property of the task, the workflow does not care
    about how the image is configured.
    """
    pi = hello_spark(partitions=50)
    print_every_time(value_to_print=pi, date_triggered=triggered_date)
    return pi

if __name__ == '__main__':
    runner = CliRunner()
    result = runner.invoke(pyflyte.main,
                           ["run",
                            "--raw-output-data-prefix",
                            "s3://flyte-batch/spark/",
                            "/Users/kevin/git/flytekit/flyte-example/databricks_wf",
                            "wf"])
    print(result.output)

FROM databricksruntime/standard:13.3-LTS
LABEL org.opencontainers.image.source=https://github.com/flyteorg/flytesnacks

ENV PYTHONPATH /databricks/driver
ENV PATH="/databricks/python3/bin:$PATH"
USER 0

RUN sudo apt-get update && sudo apt-get install -y make build-essential libssl-dev git
RUN /databricks/python3/bin/pip install git+https://github.com/Future-Outlier/flytekit.git@master#subdirectory=plugins/flytekit-spark
RUN /databricks/python3/bin/pip install markupsafe==2.0.0

COPY flyte-example/databricks_wf.py /databricks/driver/
WORKDIR /databricks/driver
ENV PYTHONPATH /databricks/driver

Tracking Issue
https://github.com/flyteorg/flyte/issues/3936
Follow-up issue
NA

Here's how I try to specify the path.
if __name__ == '__main__':
    runner = CliRunner()
    result = runner.invoke(pyflyte.main,
                           ["run",
                            "--raw-output-data-prefix",
                            "s3://flyte-batch/spark/",
                            "/mnt/c/code/dev/example/plugins/databricks_wf",
                            "wf"])
    print(result.output)

Can you explain how to set the --raw-output-data-prefix?
I tested it in the local environment.
Environment variable:
# for flyte s3 minio
export FLYTE_AWS_ENDPOINT="http://localhost:30080/"
export FLYTE_AWS_ACCESS_KEY_ID="minio"
export FLYTE_AWS_SECRET_ACCESS_KEY="miniostorage"

How to Setup
Let's say you have a python file called databricks_wf.py,
and you want to run it on the databricks platform.
0. Databricks SETUP
(0) Setup your workspace
https://docs.flyte.org/en/latest/deployment/plugins/webapi/databricks.html#deployment-plugin-setup-webapi-databricks
(1) Enable BYOC (bring your own container)
curl -X PATCH -n \
  -H "Authorization: Bearer <your-personal-access-token>" \
  https://<databricks-instance>/api/2.0/workspace-conf \
  -d '{
    "enableDcs": "true"
    }'

Note: remeber to use the token, it doesn't be written in the docs
reference: https://docs.databricks.com/en/clusters/custom-containers.html
(2) Upload your entrypoint.py to dbfs (DataBricks File System)
Copy the python file in (0) and upload it to dbfs.
(3) instance profile
(Kevin, please elaborate more, details is so important, or please give an example)
reference: https://docs.databricks.com/en/aws/iam/instance-profile-tutorial.html
1. Build your Dockerfile (Will support ImageSpec in the future)
FROM databricksruntime/standard:13.3-LTS
LABEL org.opencontainers.image.source=https://github.com/flyteorg/flytesnacks

ENV PYTHONPATH /databricks/driver
ENV PATH="/databricks/python3/bin:$PATH"
USER 0

RUN sudo apt-get update && sudo apt-get install -y make build-essential libssl-dev git
RUN /databricks/python3/bin/pip install git+https://github.com/Future-Outlier/flytekit.git@master#subdirectory=plugins/flytekit-spark
RUN /databricks/python3/bin/pip install markupsafe==2.0.0

COPY flyte-example/databricks_wf.py /databricks/driver/
WORKDIR /databricks/driver
ENV PYTHONPATH /databricks/driver

docker built -t pingsutw/databricks:v7 .

Note: you have to put your python file to your PYTHONPATH
2. Run the code
Locally
(Wait for Kevin's reply)
Remotely
You can use pyflyte register or pyflyte register --non-fast
the second one will skip zipping and uploading the package
(which means you don't need to download the input from s3, will make the workflow faster)
pyflyte register databricks_wf.py --version DB-FIRST

pyflyte register --non-fast databricks_wf.py --version DB-SECOND

Now, you can run it!

I run it successfully in remote development, since I don't have the AWS s3's secret, so I can't put the data, but all functions work well!

Is the applications_path="local:///usr/local/bin/entrypoint.py", in config necessary?

This is the command to run it locally.
pyflyte --verbose run --raw-output-data-prefix s3://flyte-batch/spark/ flyte-example/databricks_wf.py wf

I thought about it some-more, after delibration i think this is confusing.
Here is what I think makes more sense,

By default all local executions of tasks with agent will automatically try to invoke remote services. For agents with PythonFunctionTask or AgentExecutorMixin we should check if raw-output-prefix is set. If not, we raise an error

class AgentFunctionTaskExecutor():
  ...
def execute():
   if ctx.raw_output_prefix is local:
      raise AssertionError("Using agent {self.name} locally needs to have a way to pass the data/code from local to remote. This needs the configuration of a common shared blob store like S3, gcs etc. This can be achieved using `--raw-output-prefix` in `pyflyte run`. If you want to run the task code locally without invoking the remote service (e.g. testing) use `--local-agent-emulation` flag in `pyflyte run`
    ... continue to execution

Thus a user has to specify pyflyte run --raw-output-prefix or pyflyte run --local-agent-emulation to run it correctly.
This is almost self documenting


FRONTEND: "Prove this!" gives the opposite implication

Go to:
https://teorth.github.io/equational_theories/implications/?359
You will see "Unknown By" and the first item is "Equation65[x = y ◇ (x ◇ (y ◇ x))]".
When you click on "Prove this!" it seems to take me to a template for the opposite implication than it should take me, namely:
theorem Equation359_implies_Equation65 (G: Type _) [Magma G] (h: Equation359 G) : Equation65 G := by
  sorry
theorem Equation359_not_implies_Equation65 : ∃ (G: Type) (_: Magma G), Equation359 G ∧ ¬ Equation65 G := by
  sorry

claim

Ahha. Just to confirm, it's only the lower right box where you see this (only "unknown by"). All of the forward ones are correct, and the backward proven ones are correct?

I think it is only the bottom right box, but I wasn't testing everything.

Also it seems to me that the button "I'm feeling lucky" always takes me to an actual open problem (never to the flipped one).

What do you mean by "flipped" one?

I observed that the button "I'm feeling lucky" always takes me to an actual open problem (never to the "wrong" one).

propose PR #473

Ignore my last two messages. I was just confirming that it was all right on the examples I tried.

There are several I'm aware of:
https://github.com/teorth/equational_theories/blob/c27789ec7ea82c2d870080bf8c5bc0ff3af12a51/scripts/generate_graphiti_data.rb#L16
https://github.com/teorth/equational_theories/blob/c27789ec7ea82c2d870080bf8c5bc0ff3af12a51/scripts/explore_magma.py#L13
https://github.com/teorth/equational_theories/blob/c27789ec7ea82c2d870080bf8c5bc0ff3af12a51/scripts/find_dual.py#L169
https://github.com/teorth/equational_theories/blob/c27789ec7ea82c2d870080bf8c5bc0ff3af12a51/scripts/generate_equation_implication_js.py#L109

Oops I responded with the above message to the wrong place

The PR was merged, this issue can be closed. (Not sure why that didn't happen automatically.)

Yes, it works correctly now. Thanks a lot!


Link all specs and all versions

Change the site to "Publications" as it is both specs and registries.

This has some issues and I need to post a rendered version- I'll open a new PR shortly.


Support for hierarchy of messages

Hi,
I have a two level hierarchy of objects: (1) the base level object contains some fixed length fields and some variable length fields; (2) the derived objects similarly contain some fixed length fields and some variable length fields.
One obvious but tedious way to generate codecs for my set of message types would be to create a schema where I repeat the base fields in every derived message, eventually changing the order of the fields to conform to the recommended pattern (fixed length fields first followed by variable length fields).
Is there any other way? Something like defining a base message type and specifying a hierarchy of messages in the schema, and generating a hierarchy of codecs? Do you have any examples?
Thanks,
Vladimir

Common fields can be grouped into a component and reused across messages.  See the use of the Engine in the Car example.
https://github.com/real-logic/simple-binary-encoding/blob/master/sbe-samples/src/main/resources/example-schema.xml#L44
https://github.com/real-logic/simple-binary-encoding/blob/master/sbe-samples/src/main/resources/example-schema.xml#L78
https://github.com/real-logic/simple-binary-encoding/blob/master/sbe-samples/src/main/java/uk/co/real_logic/sbe/examples/ExampleUsingGeneratedStub.java#L123

Hi Martin, thanks for your reply. I have not seen an example where a composite has a reference to a String and I am not quite sure whether it's supported or not.
I tried this:
        <composite name="Base">
            <type name="pubType" primitiveType="int8"/>
            <type name="requestId" primitiveType="int64"/>
            <ref name="subject" type="varStringEncoding"/>
        </composite>

I was expecting my generated BaseEncoder class to have a subject(String) method similar to
   public CarEncoder manufacturer(final String value)

in your example code, but that does not seem to be the case.
Any suggestions?
Thanks,
Vladimir

Strings and var data are only allowed at the end of a message or repeating group. Therefore it would not be possible to do what you are suggesting even with inheritance. The SBE specification is strict on how messages can be laid out.

Thanks Martin


[web] Zoom and markers update

Decrease zoom level granularity
Add distance markers every kilometer
Use round start / end markers

The distance marks are quite cool and the round begin/end markers definitely look nicer than the previous pin-shaped markers

@flopp thanks !
Making good progress :)
What's next ?

I've added some issues with some general ideas of what one could do


Default asset types in CAI export to only assets Forseti supports.

Request from Cloud Asset team to default to limiting asset types to only those used.
Thanks for opening a Pull Request!
Here's a handy checklist to ensure your PR goes smoothly.

[x] I signed Google's Contributor License Agreement
[x] My code conforms to Google's python style.
[x] My PR at a minimum doesn't decrease unit-test coverage (if applicable).
[x] My PR has been functionally tested.
[x] All of the unit-tests still pass.
[x] Running pylint --rcfile=pylintrc passes.

These guidelines and more can be found in our contributing guidelines.

Good call, done.


[9.x] Fix bug in BelongsToMany where non-related rows are returned

While working on a new package for laravel, I encountered a failing test on laravel 9.x with no apparent reason. After digging a bit deeper, I came to understand that firstOrNew(), firstOrCreate() and updateOrCreate() were not doing what I expected.
Expected behaviour
These three methods only return or affect related rows that are in fact related to the current model.
Actual behaviour
The firstOr* methods will return any row in the related table, not subject to the relation constraint. Likewise, the updateOrCreate method updates any one row in the related table, not subject to the relation constraint. In all cases, any passed $attributes are correctly used, it is the relation constraint that is missing.
Fix
In commit 3e66eb75ae379f14eb760a19071a07134de797e3 a change in laravel 9 was introduced where the lookup would be done using $this->related instead of $this. This PR changes this to $this->clone(). Three tests are added which fail on the current 9.x version, and are fixed using this PR.
Targetting 9.x because 8.x does not use the $this->related approach.
Considerations
In commit 3e66eb75ae379f14eb760a19071a07134de797e3, no tests are added backing the change to use $this->related instead of $this. I can imagine it has something to do with not wanting to pollute the current query builder with the additional where() method? In that case I'm confident this PR is still compatible, otherwise you might want to double check.

I think this would be a breaking change now. It basically reverts it back to the original behavior that some others were complaining about. Typically, you don't want duplicate models in a many to many relationship. For example, you want one tag named "Technology" and you want to share that same Tag record across all tagged Posts with that Tag.
I don't think it's possible to please everyone with these methods. If they aren't behavior how you want them to, I suggest just making more manual queries to check for the tags existence yourself, etc.

@taylorotwell Do you have a reference to the complaints? I could not find any in the related commit.
Your observation sounds solid, but also in that case I think the implementation is misleading. Consider that any method on the BelongsToMany relationship returning a model, returns a model that is related to the model operated upon. Except that the two firstOr* methods may return 'related' models that are not related at all, and may not be related throughout the entire lifetime of the request.
My understanding of firstOrCreate is that you will always receive a related model instance that already was related ('first') or is newly related ('create') to the model operated upon. In case a new related record is created, attach() is explicitly called. Would hence something along these lines not be better? (Rough outline, may be missing some details)
    public function firstOrCreate(array $attributes = [], array $values = [], array $joining = [], $touch = true)
    {
        if (!is_null($instance = $this->clone()->where($attributes)->first())) {
            return $instance;
        } elseif (!is_null($instance = $this->related->where($attributes)->first())) {
            $this->attach($instance); // Make sure this model is related
            return $instance;
        }

        return $this->create(array_merge($attributes, $values), $joining, $touch);
    }

Similar arguments hold for firstOrNew, but you cannot automatically attach in that case due to the nature of the method. What you have here is really a kind of undefined behaviour, since a model returned can either be:

A genuine related record
A record from the related table, that is present (so exists = true), but is not related
A new instance (exists = false)
While in the old situation you can distinguish between the two possible situations, you cannot distinguish cases 1. and 2. here. To me it seems that in a fetching setting this would render this method useless (you cannot know its exact state, right?), and in a mutating setting (in which you want to store the related model) this would imply that you would always have to attach() the related model as you cannot tell whether it is already attached or not?
Also consider that findOrNew() does only return related records, while firstOrNew() doesn't.

updateOrCreate is different story. Consider that $model->relation()->update() will always only update rows that are related. Changing it into $model->relation()->updateOrCreate() introduces the possibility that (without clear indication) a row is being updated that is not related to the model at all. One could argue that also in this case (similar to firstOrCreate()), the implementation should automatically attach() a related model if it is not related already.
I am not entirely convinced that the ->attach() approach in above rough outline would be the best approach. If I understand the past issue correctly, maybe a better alternative would have been to leave these three methods as they originally were, and introduce new methods firstOrCreateAndAttach() and updateOrCreateAndAttach(). But probably that is a bit of a hindsight.
Long story short, I am concerned that these three methods violate the invariant that all similar BelongsToMany methods always act on related data, and these three now do not behave in this same way. I can see what the intention behind the change might have been, but maybe that should have required a different solution? Currently I feel the behaviour can be rather unexpected, especially in some edge cases that likely will occur in applications.
I would love to hear what others think of this issue.