Current Power drops to 0 every hour

https://github.com/sockless-coding/panasonic_cc/blob/5746f9ccf8d33bcbc0f1d5bc4fab6c4a5d74bad8/custom_components/panasonic_cc/panasonic.py#L132
I was wondering if we could revisit the included calculation for 'current power', as my power seems to update each hour at minute 33 to the correct value for the hour and then resets to zero at 5 minutes past the hour, so half of each hour is zero:

I think it would be better to see something like this as the standard output:

Based on the output from pcomfortcloud, fixed value for each hour:
`$ pcomfortcloud $user $passwd history 1 Day 20210728
id                       : xxxxxxxxxxxx
parameters
currencyUnit         : $
energyConsumption    : 12.600000000000001
estimatedCost        : 0.0
historyDataList      :
dataNumber       : 0
consumption      : 0.1
cost             : 0.0
averageSettingTemp: 17.0
averageInsideTemp: 16.5
averageOutsideTemp: 6.25
    dataNumber       : 1
    consumption      : 1.1
    cost             : 0.0
    averageSettingTemp: 17.0
    averageInsideTemp: 17.0
    averageOutsideTemp: 6.5

    dataNumber       : 2
    consumption      : 1.0
    cost             : 0.0
    averageSettingTemp: 17.0
    averageInsideTemp: 17.0
    averageOutsideTemp: 7.75

    dataNumber       : 3
    consumption      : 0.9
    cost             : 0.0
    averageSettingTemp: 17.0
    averageInsideTemp: 17.0
    averageOutsideTemp: 7.5

    dataNumber       : 4
    consumption      : 0.9
    cost             : 0.0
    averageSettingTemp: 17.0
    averageInsideTemp: 17.0
    averageOutsideTemp: 9.75

    dataNumber       : 5
    consumption      : 0.9
    cost             : 0.0
    averageSettingTemp: 17.0
    averageInsideTemp: 16.75
    averageOutsideTemp: 10.25

    dataNumber       : 6
    consumption      : 0.9
    cost             : 0.0
    averageSettingTemp: 18.5
    averageInsideTemp: 18.25
    averageOutsideTemp: 10.0

    dataNumber       : 7
    consumption      : 2.4
    cost             : 0.0
    averageSettingTemp: 18.5
    averageInsideTemp: 17.75
    averageOutsideTemp: 10.0

    dataNumber       : 8
    consumption      : 1.1
    cost             : 0.0
    averageSettingTemp: 19.25
    averageInsideTemp: 18.75
    averageOutsideTemp: 10.75

    dataNumber       : 9
    consumption      : 3.3
    cost             : 0.0
    averageSettingTemp: 19.5
    averageInsideTemp: 18.75
    averageOutsideTemp: 10.0

    dataNumber       : 10
    consumption      : 0.0
    cost             : 0.0
    averageSettingTemp: 19.5
    averageInsideTemp: 19.0
    averageOutsideTemp: 9.0

    dataNumber       : 11
    consumption      : -255
    cost             : -255
    averageSettingTemp: -255
    averageInsideTemp: -255
    averageOutsideTemp: -255

    dataNumber       : 12
    consumption      : -255
    cost             : -255
    averageSettingTemp: -255
    averageInsideTemp: -255
    averageOutsideTemp: -255`

Hmm, guessing Panasonic has different update intervals for their devices when it comes to energy readings mine updates every 15min


Changes to DOM with jQuery are not preserved in sticky yields

Hey Guys--
I'm getting some unexpected functionality with my templates:tabs package when I test it with Iron Router. I believe the source of the problem is the new sticky yield functionality in Iron Router.
It seems like changes I make to the DOM using jQuery (which runs in the tabbed interface's rendered callback) are not preserved when navigating to another route and then back (refreshing the browser fixes the problem, obviously, but the template instance's state is also refreshed).
I don't have time to go through Iron Router's code at the moment, but I'd guess that yields are being cached before the rendered callback (or perhaps without any Template.instance state)--after a route loads? If so, ideally all of this stuff would be gathered up and preserved when the route changes rather than when the route loads.

Nevermind this--it didn't have anything to do with IR after all.


fix(debugging): Debugging problem with hydrateUtxos()

:tada: This PR is included in version 4.2.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


filter_complex should not be used for basic pipelines

I'm trying to use ffmpeg-python for a very simple task, concatenating many mp3s together.
I set up my pipeline like so:
import ffmpeg
import os
import os.path

audio_files = sorted(os.listdir("."))[:5]
audio_inputs = list(map(lambda audio_file: ffmpeg.input(audio_file), audio_files))
ffmpeg.concat(*audio_inputs).output("concat_test.mp3").run()

Which produces the command:
ffmpeg -i "file-0" -i "file-1" -i "file2" -i "file3" -i "file4" -filter_complex [0][1][2][3][4]concat=n=5[s0] -map [s0] concat_test.mp3

This fails with the error message:
Stream specifier '' in filtergraph description [0][1][2][3][4]concat=n=5[s0] matches no streams.
However if I remove the -filter_complex and -map flags then the command runs as expected.

Try adding v=0, a=1 in your concat call:
ffmpeg.concat(v=0, a=1, *audio_inputs).output("concat_test.mp3").run()

Source: https://stackoverflow.com/questions/19417412/ffmpeg-concatenation-filters-stream-specifier-0-in-filtergraph-matches-no-st

Although the error message is not very user friendly you need to tell the concat filter the number of output video streams. The default is 1 for video and 0 for audio, so it's usually a good idea to be explicit:
$ ffmpeg -i OHIn.wav -i OHOut.wav -filter_complex \
 '[0:0] [1:0] concat=n=2:v=0:a=1 [a]' -map '[a]' output.wav

If you leave the filter_complex + map args out as you described, ffmpeg runs but it doesn't actually concat the files.

Ah! That did it. Thanks for your help. I must have been listening to an earlier output when I thought removing those args worked.
This tool is really great by the way, I really like the fluent API.

Glad to help. Enjoy!


The library is really nice but does it really need angular as a dependency?

We would like to use this library in our project but the strict tie to angular is making it less attractive. Can we just lower the version of angular to minimum of version 8 or something?

Hi @duydnguyen07
Thank you for your interest.

does it really need angular as a dependency?

Angular is a peer dependency, and needs to be so because this library imports from Angular.

Can we just lower the version of angular to minimum of version 8 or something?

Version 1.2.0 of this library supports Angular 8 to 13.  Use this version for Angular 8 apps.
Version 2.0.0 raised the minimum version of Angular to 14.


[Feature Request] Configure days to remove UserSessions/Sessions

Hi there, thanks for all your hard work on this project :).
Right now, Sessions are kept around for 30 days. With two users, the sessions table hovers around 1000 rows, and I'd like to add more users without exceeding the limits on the free database we're using.
Can this duration be made configurable a la ARCHIVE_READ_DAYS? Would there be any drawbacks to using a shorter duration?
Happy to implement this, too, if you don't have the time.

You can prepare a pull-request if you would like. The SQL queries that need to be updated are here:

https://github.com/miniflux/miniflux/blob/master/storage/user_session.go#L130
https://github.com/miniflux/miniflux/blob/master/storage/session.go#L106


Example code?

Hi. I would love to try this. It is not clear how to run it. Is there an example for Sink and Source. It looks like it is configuration driven.

You can refer to https://github.com/DataReply/kafka-connect-mongodb/issues/4


pam_gnupg: 0.3 -> 0.4

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/cruegge/pam-gnupg/releases.
meta.description for pam_gnupg is: "Unlock GnuPG keys on login"
meta.homepage for pam_gnupg is: "https://github.com/cruegge/pam-gnupg"
meta.changelog for pam_gnupg is: ""
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.4 with grep in /nix/store/y6fqra9hdcbris2h54d16x5vrsjidal5-pam_gnupg-0.4
found 0.4 in filename of file in /nix/store/y6fqra9hdcbris2h54d16x5vrsjidal5-pam_gnupg-0.4

Rebuild report (if merged into master) (click to expand)

1 total rebuild path(s)

1 package rebuild(s)

First fifty rebuilds by attrpath
pam_gnupg

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/y6fqra9hdcbris2h54d16x5vrsjidal5-pam_gnupg-0.4 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A pam_gnupg https://github.com/r-ryantm/nixpkgs/archive/37c63077c002fc7372dee47558c81a97dce50845.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/y6fqra9hdcbris2h54d16x5vrsjidal5-pam_gnupg-0.4
ls -la /nix/store/y6fqra9hdcbris2h54d16x5vrsjidal5-pam_gnupg-0.4/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package built:

pam_gnupg

Maintainer pings
cc @mtreca for testing.

LGTM but I cannot test it directly since I am not using NixOS anymore

LGTM but I cannot test it directly since I am not using NixOS anymore

Whaaat? Stopped with NixOS? Curious why.

Result of nixpkgs-review pr 183707 run on x86_64-linux 1

1 package built:

pam_gnupg

@superherointj Thanks for the merge! I got an M1 MacBook air. I should give nix-darwin a try someday though. I am missing NixOS, not going to lie.


Fixes #1041 - transparent title bar on auth window

resolves #1041

Pull Request Test Coverage Report for Build 1137

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 53.449%

Totals

Change from base Build 1134:
0.0%

Covered Lines:
4224

Relevant Lines:
7127

💛  - Coveralls


fix(InputStepper): onChange is triggered when disabled

LiveURL: https://orbit-components-fix-stepper-disabled.surge.sh

Works correctly in the Storybook, but a test wouldn't hurt 🙂 Something like this:
it("should not work if disabled", () => {
  const onChangeMock = jest.fn();
  component.setProps({ disabled: true, onChange: onChangeMock });
  const btnDecrement = component.find(ButtonLink).first();
  btnDecrement.simulate("click");
  expect(onChangeMock).not.toHaveBeenCalled();
  // cleanup
  component.setProps({ disabled, onChange });
});

but this doesn't work for some reason, setting disabled to false makes it pass as well. Maybe you can finish it? I'm a little rusty with Enzyme.

Oh, I realized that it passed either way because of minValue. You can e.g. select increment button instead:
it("should not work if disabled", () => {
  const onChangeMock = jest.fn();
  component.setProps({ disabled: true, onChange: onChangeMock });
  const btnIncrement = component.find(ButtonLink).last();
  btnIncrement.simulate("click");
  expect(onChangeMock).not.toHaveBeenCalled();
  // cleanup
  component.setProps({ disabled, onChange });
});


Spline questions, unclarities in chapter 1

The first issue was that you write "Control points are user-defined, pre-specified points through which the curve must pass."
From my understanding (which largely stems from Freya Holmers video), curves created by B-Splines do not neccesarily pass through the control points.
The other thing is that the graphs do not show partition of unity(i.e that the functions add up to one at every point) of B-Splines in order 2 and 3. My matplotlib skills are very basic, but I imagine that this is because the curve doesnt show the negative parts of the basis functions.
I apologize if these issues dont exist, I am a new to the topic. I hope that even if I missed a detail, other newcomers will miss these too and this will improve the clarity of the tutorial at least. Thanks for your time.

Hi,
Thanks for the question!

Thanks for pointing this out. You are right that the curve need not pass through all the control points. The order of the curve also matters.

The graphs of partition need not sum to unity. This can be worked through the recursion formula.

I worked the recursion formula a bit by hand, and although I could not prove partition to unity, I think your code has a small bug which makes the numbers go so high. In the denominator of the first term in the recursion formula you iterate the list 1:k when it should be k:1. So instead of
value21 = (x_ - grid_[:, :-(k+1)]) / (grid_[:, 1:-k] - grid_[:, :-(k+1)]) * value1[:, :-1] it should be
value21 = (x_ - grid_[:, :-(k+1)]) / (grid_[:, k:-1] - grid_[:, :-(k+1)]) * value1[:, :-1]
Your bug makes it so it basically does not normalize the term for higher degrees.
If I did not miss something and this is a mistake, it is very interesting that even though this bug happened, your tutorial runs flawless and the network can learn all these toy examples with a not normalized cubic spline. Certainly something interesting.


[Question]: How does GraphIndex retrieve nodes? Does it rely on LLM?

Question Validation

[X] I have searched both the documentation and discord for an answer.

Question
How does GraphIndex retrieve nodes? Does it rely on LLM?

Yes, it uses the LLM to extract keywords


[Question] 关于总数据量的问题

Required prerequisites

[X] I have read the documentation https://beavertails.readthedocs.io.
[X] I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
[X] Consider asking first in a Discussion.

Questions
我看到论文中提到总数据量在3万左右，但是hugggingface上开源的数据总共有30万左右，想问下这部分多出来的数据是如何构造、标注的呢，也是由alpaca-7b生成+人工标注的吗？
Checklist

[X] I have provided all relevant and necessary information above.
[X] I have chosen a suitable title for this issue.

We are about to update our manuscript soon (after the NeurIPS camera-ready deadline), where there will be more detail provided for the 330k partition. The short answer is, that we applied the same methodology but just expanded the dataset to 300k.

If no further question, closing this issue for now.


Rotate Keycloak Secrets

Add the experimental feature to the keycloak.conf file. The configuration will be done in the tdr-configurations repo.
https://github.com/nationalarchives/tdr-configurations/pull/121

It turns out I missed quite a bit off this PR. I also seem to have enabled client secret rotation on the PR to upgrade the library so that's now missing from here.


[operator] Allow using own self-signed certificate

Addresses https://github.com/open-telemetry/opentelemetry-helm-charts/issues/757

@Allex1 thanks for the heads-up.
Updated. Please let me know if you think the description added to guideline should be clearer.

hey @Allex1 and @TylerHelmuth . sorry to bother but is there anything else blocking the merge? we need this feature so we can move forward. thanks


fix: elysia adapter context.set type

Issue Reference
https://github.com/lucia-auth/lucia/issues/1143
I made similar change locally in the node_modules and it worked

I think we can just remove status and redirect?

I did just that locally and it worked! thank you for the feedback @pilcrowOnPaper , just pushed the changes

Thanks!

Awesome, thank you! First PR merged 😎


awesome-promises

Maybe link up https://github.com/wbinnssmith/awesome-promises somewhere? It's a useful resource.
// @wbinnssmith

Since awesome-promises links here, it could cause an infinite spiral of recursive awesomeness. Worth a try.

I approve. Been busy this week but would accept a pr. 🙌


add override in DrawingInterfaceAllegro5

commented out due to legacy.

also in DrawingInterfaceHTMLCanvas

also in DrawingInterfaceSVG

fixed in https://github.com/MarkOates/allegro_flare/commit/6d3ce4dc5b3e81f0636cf57a9330e6107ac02688


MISC removes set and gets a unique array by other means

set was being uglified, leading to local being ok but prod broken :(
fixed now; Set removed

👍


simply the links in download checkponts in readme.md.

There are a lot of links to various sites in this section. It could be just me but I think It would be nice if the links were added to point without URLs.
This point can also be written in a note rather then a point.
We include all the optimizer params for Adam, so the checkpoint is big. You could only keep the "state_dict" to make it much smaller.

@XavierCHEN34 I was hope it would be assigned to me but, I gess that works.


[FEATURE REQ] Add Sample on how to use Sessions (Producer & Receiver) for Java Spring Boot

Is your feature request related to a problem? Please describe.
When implementing Topics with Named Sessions in the project, we looked for Java Spring Boot Samples that could give an initial starting point for Producer and Receiver scenarios.
We were able to find some samples, but they were related to Queues or related to Receiver only.
https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/servicebus/azure-messaging-servicebus/src/samples/java/com/azure/messaging/servicebus
Describe the solution you'd like
Sample code available that demonstrates the use of Topics with Named Sessions(Producer and Receiver) for Java Spring Boot applications.
Describe alternatives you've considered
N/A
Additional context
N/A
Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[x] Description Added
[x] Expected solution specified

Hi, @madossan01 , thanks for reaching out.
We provided some related samples in Azure-Samples/azure-spring-boot-samples
We also provided starters that may help you quickly use the Service Bus with Spring Boot, such as spring-cloud-azure-starter-servicebus, spring-cloud-azure-starter-integration-servicebus and spring-cloud-azure-stream-binder-servicebus, click https://microsoft.github.io/spring-cloud-azure/current/reference/html/index.html#getting-started to get more information.

Hi, @madossan01 , thanks for reaching out. We provided some related samples in Azure-Samples/azure-spring-boot-samples
We also provided starters that may help you quickly use the Service Bus with Spring Boot, such as spring-cloud-azure-starter-servicebus, spring-cloud-azure-starter-integration-servicebus and spring-cloud-azure-stream-binder-servicebus, click https://microsoft.github.io/spring-cloud-azure/current/reference/html/index.html#getting-started to get more information.

Thanks for sharing the links. Sorry, my bad do not have mentioned that in the issue, we indeed used the starters and they saved considerable time. Even though the samples are interesting starting points, our scenario required the use of Topics and named Sessions with Services Bus SDK for Java. Most samples are using JMS only or they target Queue instead of Topics.  Learning how to better use the Topics and Sessions and how to make them work properly took us some time on try and fix loops.  As the use of Topics with Sessions may represent a recurrent scenario, a sample with the best practices may save some precious time from Java developers and improve final code quality.


[Snyk] Fix for 1 vulnerabilities

Snyk has created this PR to fix one or more vulnerable packages in the `npm` dependencies of this project.
Changes included in this PR

Changes to the following files to upgrade the vulnerable dependencies to a fixed version:

package.json

Vulnerabilities that will be fixed
With an upgrade:

Severity
Priority Score (*)
Issue
Breaking Change
Exploit Maturity

768/1000  Why? Proof of Concept exploit, Recently disclosed, Has a fix available, CVSS 7.5
Regular Expression Denial of Service (ReDoS) SNYK-JS-ANSIREGEX-1583908
Yes
Proof of Concept

(*) Note that the real score may have changed since the PR was raised.

Commit messages

Package name: eslint
    The new version differs by 210 commits.

3dd6741 7.0.0
9a722f9 Build: changelog update for 7.0.0
b98d8bd Upgrade<EMAIL_ADDRESS>(#13271)
4c0b028 Fix: remove Node.js and CommonJS category from build process (#13242)
401a687 Chore: fix rules list for prereleases (#13230)
4ef6158 Breaking<EMAIL_ADDRESS>(#13270)
b5c8d73 Docs: update 7.0.0 migration guide for consistency (#13267)
356fdb4 Docs: add migration guide (#12692)
015edf6 Sponsors: Sync README with website
fdfa364 7.0.0-rc.0
8d1b4db Build: changelog update for 7.0.0-rc.0
0b1d65a Update: Improve report location for array-callback-return (refs #12334) (#13109)
d85e291 Fix: yoda left string fix for exceptRange (fixes #12883) (#13052)
2ce6bed Chore: added tests for nested arrays (#13145)
d3aac53 Update: report backtick loc in no-unexpected-multiline (refs #12334) (#13142)
8e7a2d9 Fix: func-call-spacing "never" reports wrong message (fixes #13190) (#13193)
bcafd0f Update: Add ESLint API (refs eslint/rfcs#40) (#12939)
3eeae56 Upgrade: some (dev) deps (#13155)
6b7030b Chore: Run tests on Node.js v14 (#13210)
ebc28d7 Fix: Remove default .js from --ext CLI option (#13176)
5c1bdeb Update: Improve report location for getter-return (refs #12334) (#13164)
56d2bee Docs: fix typos (#13204)
e13256e Chore: use espree.latestEcmaVersion in config-initializer (#13157)
e4f57b7 Chore: add nested array tests for array-element-newline (#13161)

See the full diff

Package name: jest-watch-typeahead
    The new version differs by 6 commits.

35f8d38 0.6.0
e256697 chore(breakng): upgrade Jest to v26 (#35)
cc1fd7c v0.5.0
aee1f93 chore: update dependencies
efd5a49 chore: fix link to jest logo in readme
7344853 chore: refresh lockfile

See the full diff

Package name: webpack-dev-server
    The new version differs by 250 commits.

c9271b9 chore(release): 4.0.0
18bf369 test: fix stability (#3676)
cdcabb2 fix: respect protocol from browser for manual setup (#3675)
1768d6b fix: initial reloading for lazy compilation (#3662)
4f5bab1 docs: improve examples (#3672)
f2d87fb fix: improve https CLI output (#3673)
0277c5e chore: remove redundant console statements (#3671)
16fcdbc docs: add `ipc` example (#3667)
8915fb8 test: add e2e tests for built in routes (#3669)
4d1cbe1 docs: ask `version` information in issue template (#3668)
b6c1881 chore(deps-dev): bump core-js from 3.16.1 to 3.16.2 (#3666)
ffa8cc5 chore(deps-dev): bump supertest from 6.1.5 to 6.1.6 (#3665)
f1fdaa7 chore(release): 4.0.0-rc.1
c4678bc fix: legacy API (#3660)
d8bdd03 test: fix stability (#3661)
22b1414 refactor: remove `killable` (#3657)
75bafbf test: add e2e tests for module federation (#3658)
493ccbd chore(deps): update `ws` (#3652)
ae8c523 test: add e2e test for universal compiler (#3656)
f94b84f chore(deps): update (#3655)
1923132 test: fix cli
2adfd01 test: fix todo (#3653)
6e2cbde fix: proxy logging and allow to pass options without the `target` option (#3651)
c9ccc96 fix: respect infastructureLogging.level for client.logging (#3613)

See the full diff

Check the changes in this PR to ensure they won't cause issues with your project.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
🛠 Adjust project settings
📚 Read more about Snyk's upgrade and patch logic

Will update dependencies manually


Changed RecordType type value to AuditRecordType

Docs Build status updates of commit 06e5278:
:clock10: Full build: the average full build time is 13 min(s) 31 sec(s), this is based on the last 510 build(s) for this repository.

Docs Build status updates of commit 06e5278:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/policy-and-compliance-audit/Search-UnifiedAuditLog.md
:white_check_mark:Succeeded
View (exchange-ps)

exchange/exchange-ps/exchange/policy-and-compliance/New-ActivityAlert.md
:white_check_mark:Succeeded
View (exchange-ps)

exchange/exchange-ps/exchange/policy-and-compliance/Set-ActivityAlert.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Anonymous Mode

I know what the answer is but I'm going to ask anyhow. Is there a possibility of anonymous mode?

This is a duplicate. Yeah, we aren't going to add anonymous mode, but I think there might be forks that are looking into doing so.


EXPIREADD not working

Fixed feedback issue with EXPIREADD but looking at the code EXPIREADD expires before it can check the effect for EXPIREADD

BCEG issue. fixed


how to verify if the implemented API is CAMARA Complaint

Hi Team ,
Once i implement the API , is there any process to verify if my implementation is CAMARA Complaint.
Please suggest how can i get this validated.
Regards
Prashant Sidana

this topic should be discussed inside Commonalities WG


Journeys not being bundled correctly when using modules in journey/ subdirectory

I initially thought that this was the same issue as Issue #693, but I did a deeper dive after the response I got on that issue and think that while it may be related, it's not the same issue.
I am using a factory pattern to generate identical tests which run against multiple endpoints. When I separate the factory module from the journey file, only the factory module is bundled by NPM and pushed to the cluster.
This has been validated with @elastic/synthetics-1.7.2.
Steps to reproduce

Initialize a new @elastic/synthetics project.
Create the following files:

/**
 * File 1: journeys/journey-factory.journey.ts
 */
import { JourneyFactory } from './modules/journey-factory';

const googleJourney = new JourneyFactory().createJourney();
googleJourney();

/**
 * File 2: journeys/modules/journey-factory.ts
 */
import { journey, step, monitor, expect } from '@elastic/synthetics';

interface JourneyFunction {
  (): Promise<void>;
}

export class JourneyFactory {
  private env: string;  // Test environment

  createJourney(): JourneyFunction {
    return this.syntheticsJourney;
  }

  // Journey function
  syntheticsJourney = async (): Promise<void> => {
    journey(`google_factory_test`, ({ page, params }) => {
      monitor.use({
        id: '35a0908a-dac1-4c95-ba9c-28afbe1b4c07',
        schedule: 5,
        tags: ['google', 'factory', 'module', 'test',],
      });

      step('Open URL', async () => {
        await page.goto('https://www.google.com/');
        await expect(page.getByText('Google')).toBeTruthy();
      });
    });
  }
}

Run locally: npx @elastic/synthetics .
Push to remote.

Results
Everything runs as expected when running locally from the command line with npx @elastic/synthetics ..
When I push it to my Elastic Cluster, it generates the following error:

When I look at the correct folder on the Agent, I see the following:
root@elastic-synthetics-agent-XXXXXXXXXX-XXXXX:/tmp/elastic-synthetics-unzip-XXXXXXXXXX/journeys# ls -l
total 0
drwxrwx--- 2 root root 32 Mar 19 14:04 modules
root@elastic-synthetics-agent-XXXXXXXXXX-XXXXX:/tmp/elastic-synthetics-unzip-XXXXXXXXXX/journeys# ls -l modules/
total 4
-rwxrwx--- 1 root root 2039 Mar 19 14:04 journey-factory.ts
root@elastic-synthetics-agent-XXXXXXXXXX-XXXXX:/tmp/elastic-synthetics-unzip-XXXXXXXXXX/journeys/modules#

When I examine the contents of journey-factory.ts, it corresponds to the module, I'm not able to find the actual journey file.
If I define the factory class and instantiate it in the same file, everything runs as expected.
Originally posted by @renzedj in https://github.com/elastic/synthetics/issues/693#issuecomment-2007373096

Thanks for raising the issue @renzedj, The way push command works is by

Traversing for all the journey invocations and then getting the location information of these files
Bundling them and pushing them over to the Kibana backend

Since the factory contains the invocation of the journey() inside the Class definition itself, the journey-factory.journey.ts is not useful here since its not doing anything other than calling the invoker itself where the actual journey is defined. The other issue is, even though the bundle contains the all the necessary information to run the required journey, The reason why its failing is because the filename is journey-factory.ts, Synthetics runner by default looks for files that are matching /*.journey.(ts|js)$/ format. Since the filename is not matched, the run yields no Steps to run.
We follow similar factory patterns, but try to do one of these things

Either move the helper function inside helper.journey.ts and let the caller invoke different journeys based on individual configuration (be it journey name, environments, etc).
Keep the journey invocation in the corresponding file and use a helper API for the individual steps.

import {createJourney} from "./helper";
import {journey} from "@elastic/synthetics";

journey("name", async() => {
   createJourney();
})

Let us know if you have more questions or feedback. Thanks.

Thanks for raising the issue @renzedj, The way push command works is by

Traversing for all the journey invocations and then getting the location information of these files
Bundling them and pushing them over to the Kibana backend

This is what I suspected, when I compared it with the way that the advanced-example.journey.ts handles the import. Can you please clarify this in the documentation?  as it implies that it starts from the *.journey.ts and traverses from there.

Since the factory contains the invocation of the journey() inside the Class definition itself, the journey-factory.journey.ts is not useful here since its not doing anything other than calling the invoker itself where the actual journey is defined. The other issue is, even though the bundle contains the all the necessary information to run the required journey, The reason why its failing is because the filename is journey-factory.ts, Synthetics runner by default looks for files that are matching /*.journey.(ts|js)$/ format. Since the filename is not matched, the run yields no Steps to run.

While I can see why the decision was made to go with this for bundling, IMO this limits the utility of the factory pattern, which would otherwise be able to return a complete journey to be invoked, like in the example I provided. If the synthetics agent looks for the *.journey.(js|ts) to execute, then the bundling SHOULD do the same. Seems to me that the implementation would actually be easier to scan for journeys, as you don't have to open each file to look for the journey definition. Doing this would also allow the use of feature flags to more easily promote tests through regions.
While I realize that this would likely be lower in priority, I would like to request this as a change.

We follow similar factory patterns, but try to do one of these things

Either move the helper function inside helper.journey.ts and let the caller invoke different journeys based on individual configuration (be it journey name, environments, etc).
Keep the journey invocation in the corresponding file and use a helper API for the individual steps.

import {createJourney} from "./helper";
import {journey} from "@elastic/synthetics";

journey("name", async() => {
   createJourney();
})

Let us know if you have more questions or feedback. Thanks.

This was the next thing I was going to try on this. Thanks for the confirmation. Thx also for the quick response, even if it's not what I wanted to hear. 😄


Blog app - controllers specs

Performed Activities

Added RSpec as a gem to your project.
Created a Request spec for all your controllers.
Created the required controller tests.

Thank you @mikethreels.


Keras ValueError: 'Error when checking model target: expected no data, but got: ', array

Hi. I am training a variational autoencoder model using my own data generator. The model compiles fine but when I call the fit_generator method following error appears:

    Traceback (most recent call last):

     File "C:/Users/Vlad/Desktop/bachelorarbeit/autoencoder/vae_script.py", line 77, in <module>
        vae.fit_generator(generator=xtrain, use_multiprocessing= True, workers= 6)
      File "C:\Users\Vlad\Desktop\bachelorarbeit\autoencoder\venv\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper
        return func(*args, **kwargs)
      File "C:\Users\Vlad\Desktop\bachelorarbeit\autoencoder\venv\lib\site-packages\keras\engine\training.py", line 1418, in fit_generator
        initial_epoch=initial_epoch)
      File "C:\Users\Vlad\Desktop\bachelorarbeit\autoencoder\venv\lib\site-packages\keras\engine\training_generator.py", line 217, in fit_generator
        class_weight=class_weight)
      File "C:\Users\Vlad\Desktop\bachelorarbeit\autoencoder\venv\lib\site-packages\keras\engine\training.py", line 1211, in train_on_batch
        class_weight=class_weight)
      File "C:\Users\Vlad\Desktop\bachelorarbeit\autoencoder\venv\lib\site-packages\keras\engine\training.py", line 789, in _standardize_user_data
        exception_prefix='target')
      File "C:\Users\Vlad\Desktop\bachelorarbeit\autoencoder\venv\lib\site-packages\keras\engine\training_utils.py", line 63, in standardize_input_data
        'expected no data, but got:', data)
    ValueError: ('Error when checking model target: expected no data, but got:', array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
           1, 1, 1, 1, 1, 1, 1, 1, 1, 1]))

    Process finished with exit code 1

As far as I can see the ValueError is raised by following lines in the trainings_util.py in the standardize_input_data method:

        if not names:
        if data is not None and hasattr(data, '__len__') and len(data):
            raise ValueError('Error when checking model ' +
                             exception_prefix + ': '
                             'expected no data, but got:', data)
        return []

where 'names' is according to the docstring a list of expected array names. I tried solving this problem by feeding the generator with a tuple of dicitionaries of the type ({'input': X}, {'output': y}) where X is the data and y the targets, but the same error is raised.
My question is what names should I pass to the generator or should I pass any names at all? Also another question would is how to solve this problem? Can you give me a hint on what I am missing?
I am using the lastest Keras version (2.2.4) with Tensorflow backend (version 1.12.0) on a Windows 10 64-Bit machine. The code for the generator class and the  model script can be found here:
(1) Model: https://pastebin.com/YR9y9WAN
(2) Data generator: https://pastebin.com/UK7VyQe4
Thank you!
Please make sure that the boxes below are checked before you submit your issue.
If your issue is an implementation question, please ask your question on StackOverflow or on the Keras Slack channel instead of opening a GitHub issue.
Thank you!

[ x] Check that you are up-to-date with the master branch of Keras. You can update with:
pip install git+git://github.com/keras-team/keras.git --upgrade --no-deps

[ x] Check that your version of TensorFlow is up-to-date. The installation instructions can be found here.

[x] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

In order to expedite the trouble-shooting process, please provide a minimal code snippet to reproduce the issue reported here. Thanks!

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!

I had the same problem. Here's my code：
width = 229
height = 229
input_tensor = Input((height, width, 3))
x = input_tensor

x = keras.applications.inception_resnet_v2.preprocess_input(x)
base_model = keras.applications.inception_resnet_v2.InceptionResNetV2(input_tensor=x, weights='imagenet', include_top=False)

model = Model(base_model.input, GlobalAveragePooling2D()(base_model.output))

gen = ImageDataGenerator()
train_generator = gen.flow_from_directory("train2", (229,229), shuffle=False,
                                              batch_size=16)
test_generator = gen.flow_from_directory("test2", (229,229), shuffle=False,
                                             batch_size=16, class_mode=None)

train = model.predict_generator(train_generator, train_generator.nb_sample)
test = model.predict_generator(test_generator, test_generator.nb_sample)
with h5py.File("data_%s.h5"%MODEL) as h:
    h.create_dataset("train", data=train)
    h.create_dataset("test", data=test)
    h.create_dataset("label", data=train_generator.classes)

here's the error:
Found 4500 images belonging to 3 classes.
Found 1500 images belonging to 1 classes.
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-32-bbdd7e707b04> in <module>()
     15                                              batch_size=16, class_mode=None)
     16
---> 17 train = model.predict_generator(train_generator, train_generator.samples)
     18 test = model.predict_generator(test_generator, test_generator.samples)
     19 with h5py.File("data_%s.h5"%MODEL) as h:

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/keras/engine/training.py in predict_generator(self, generator, steps, max_queue_size, workers, use_multiprocessing, verbose)
   1520             workers=workers,
   1521             use_multiprocessing=use_multiprocessing,
-> 1522             verbose=verbose)

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/keras/engine/training_generator.py in predict_generator(model, generator, steps, max_queue_size, workers, use_multiprocessing, verbose)
    451                 x = generator_output
    452
--> 453             outs = model.predict_on_batch(x)
    454             outs = to_list(outs)
    455

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/keras/engine/training.py in predict_on_batch(self, x)
   1266             Numpy array(s) of predictions.
   1267         """
-> 1268         x, _, _ = self._standardize_user_data(x)
   1269         if self._uses_dynamic_learning_phase():
   1270             ins = x + [0.]

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/keras/engine/training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, check_array_lengths, batch_size)
    749             feed_input_shapes,
    750             check_batch_axis=False,  # Don't enforce the batch size.
--> 751             exception_prefix='input')
    752
    753         if y is not None:

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/keras/engine/training_utils.py in standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix)
     61             raise ValueError('Error when checking model ' +
     62                              exception_prefix + ': '
---> 63                              'expected no data, but got:', data)
     64         return []
     65     if data is None:

ValueError: ('Error when checking model input: expected no data, but got:', array([[[[255., 255., 255.],
         [255., 255., 255.],

Thanks!

may be the weights was trained by the higher version, i have the same issue, but i figure it out by install the version from 2.1.6 to 2.2.4

I have the same issue but my version is 2.3.0

Any update here? Were you able to solve the problem? I am stuck in the same. The model works fine using standard loss function such as 'mse'. However, when I use a custom loss function it displays this error. I am using a multi-output model. The custom loss functions follow appropriate signature as it works in other cases.

So what I have figured in my case is that when you have multi-output models, passing loss functions as a list does not work reliably. However, using a dictionary to define loss for each output works fine. That fixed my issue.

Try 'compiling' before do 'predict', worked for me.
model = .....
optimizer = tf.keras.optimizers.Adam(0.001, epsilon=1e-08)
model.compile(optimizer=optimizer, loss='binary_crossentropy', metrics=['accuracy'])
model_predictions = model.predict(test_generator,
verbose=1,
steps=math.ceil(test_generator.samples/test_generator.batch_size))

I have similar problem.
Here is my code:
def get_mlp_model(input_shape):
mlp = Sequential([
Flatten(name='flatten'),
Dense(64,        activation='relu', name='dense_1'),
Dense(128,      activation='relu', name='dense_2'),
Dense(64,        activation='relu', name='dense_3'),
Dense(32,        activation='relu', name='dense_5'),
Dense(10,        activation='softmax', name='dense_5')
])
mlp.build(input_shape)
mlp.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
return mlp
print(x3_train[0].shape)     #  shape is (32, 32, 3)
mlp = get_mlp_model(x3_train[0].shape)
problem here
history=mlp.fit(x3_train,y3_train,batch_size=32,epochs=1,verbose=1,validation_split=0.15)
error message pasted:
ValueError                                Traceback (most recent call last)
 in
----> 1 history=mlp.fit(x3_train,y3_train,batch_size=32,epochs=1,verbose=1,validation_split=0.15)
2
3
4 #                  validation_data=(x2_test, y2_test)
5 #                 )
/opt/conda/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs)
726         max_queue_size=max_queue_size,
727         workers=workers,
--> 728         use_multiprocessing=use_multiprocessing)
729
730   def evaluate(self,
/opt/conda/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2.py in fit(self, model, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, **kwargs)
222           validation_data=validation_data,
223           validation_steps=validation_steps,
--> 224           distribution_strategy=strategy)
225
226       total_samples = _get_total_number_of_samples(training_data_adapter)
/opt/conda/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_v2.py in _process_training_inputs(model, x, y, batch_size, epochs, sample_weights, class_weights, steps_per_epoch, validation_split, validation_data, validation_steps, shuffle, distribution_strategy, max_queue_size, workers, use_multiprocessing)
514         batch_size=batch_size,
515         check_steps=False,
--> 516         steps=steps_per_epoch)
517     (x, y, sample_weights,
518      val_x, val_y,
/opt/conda/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, batch_size, check_steps, steps_name, steps, validation_split, shuffle, extract_tensors_from_dataset)
2517           shapes=None,
2518           check_batch_axis=False,  # Don't enforce the batch size.
-> 2519           exception_prefix='target')
2520
2521       # Generate sample-wise weight values given the sample_weight and
/opt/conda/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/training_utils.py in standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix)
487       raise ValueError(
488           'Error when checking model ' + exception_prefix + ': '
--> 489           'expected no data, but got:', data)
490     return []
491   if data is None:
ValueError: ('Error when checking model target: expected no data, but got:', array([[0],
[8],
[1],
...,
[0],
[5],
[8]], dtype=uint8))
anybody can help ? Thanks

Experienced same issue, but resolved error. Check loss dictionary in compile method for inconsistencies and make sure output target names and indicies match the output tails in model.
Ex:
outputs = [x1,x2]  # names = 'value_1' and 'value_2'
model.compile(optimizer=optimizer, loss = {'**value_3: **'loss function', 'value_4': 'loss function'}


Move private method to the end of the class

(based on #334)
The ResamplerConfig now has a new option align_to to align the resampling windows to some arbitrary time. This means that the timestamps of the produced samples will be a multiple of the resampling period beginning at the align_to time.
By default align_to is the Unix epoch, and if it is None it will have the current behaviour of aligning timestamps to the time when the resampler is started.
Fixes #328.
TODO:

[ ] Add tests

[ ] Check failing FormulaComposition test (looks flaky, as it only fails in pytest_min)

Details
File: tests/timeseries/_formula_engine/test_formula_composition.py::TestFormulaComposition::test_formula_composition
>           assert grid_pow is not None and grid_pow.value is not None
E           assert (Sample(timestamp=datetime.datetime(2023, 4, 14, 20, 36, 26, 900000, tzinfo=datetime.timezone.utc), value=None) is not None and None is not None)
E            +  where None = Sample(timestamp=datetime.datetime(2023, 4, 14, 20, 36, 26, 900000, tzinfo=datetime.timezone.utc), value=None).value

tests/timeseries/_formula_engine/test_formula_composition.py:51: AssertionError

[ ] Add release notes

[ ] Maybe add example to the documentation

Creating it as an early draft so I can already get feedback while working on the TODO. I did some local tests and it seems to be working fine, so the implementation itself is ready for a review.

I was just looking at the moving window and ring buffer to see if the alignment functions can be unified, but it doesn't look trivial, as the moving window just forwards the alignment problem to the ring buffer and the ring buffer has other issues to deal with, like wrapping around, so it needs more operations related to the alignment.
But I think we could unify the API at least:

The ring buffer uses time_index_alignment
The moving window uses window_alignment
The resampler uses align_to

@matthias-wende-frequenz @mathias-baumann-frequenz do you think it makes sense to unify everything to use align_to? The other too are too specific and don't play well in other components. I think this value needs good documentation because it is not trivial to understand, so I guess using a more general keyword argument name is not a big loss, at least compared to the added clarity if we use the same term for aligning different components. At the end the concept is the same, right?

This particular test already has a fix in main,  can you rebase?

This particular test already has a (temporary) fix in main, can you rebase?

I will, but these failing tests are rebased on top of all relevant changes already (only the bump on the frequenz-floss/setup-git-user action dependency is missing).
Also locally I tried increasing the sampling period even up to 0.25 and it was still failing very often. Didn't tried with higher values though, but that already looks excessive.

I just tried and not even using 1 second removes the flakeyness :thinking: :scream:

I just tried and not even using 1 second removes the flakeyness

how about locally?

Remove draft status, as the only pending thing is the flaky tests.

how about locally?

Yes, I tried it locally.

I wonder if now this is flaky based on if the now is aligned to the epoch or not. Since I'm not familiar with the MockMicrogrid and how it interacts with sdk.micrigrid, I'm not sure if this could be the issue. I could try to align to now in the tests to see if it fixes it.

OK, this seems to remove the flakyness, so I guess that was the reason:
diff --git a/tests/timeseries/mock_microgrid.py b/tests/timeseries/mock_microgrid.py
index 7a6133a..7531b08 100644
--- a/tests/timeseries/mock_microgrid.py
+++ b/tests/timeseries/mock_microgrid.py
@@ -119,7 +119,9 @@ class MockMicrogrid:  # pylint: disable=too-many-instance-attributes

         # pylint: disable=protected-access
         _data_pipeline._DATA_PIPELINE = _data_pipeline._DataPipeline(
-            ResamplerConfig(resampling_period=timedelta(seconds=self._sample_rate_s))
+            ResamplerConfig(
+                resampling_period=timedelta(seconds=self._sample_rate_s), align_to=None
+            )
         )
         # pylint: enable=protected-access

Now the question is, should I add this change or should I adapt the tests to work with alignment? I think it would be too big of a change for this PR, specially because I think the most sensible way to make this work would be to mock time, otherwise is hard to align the tests to the resampler alignment... :roll_eyes:

I just pushed those changes to check if now test pass more reliably.

It seems to be working now.

Updated with some fixes for comments made by @mathias-baumann-frequenz

I fixed the remaining typos as suggested and enabled auto-merge.


Packaging: Use versioningit for maintaining the package version

About

versioningit, we love very much, and use it on many Python projects we publish to PyPI.
In order to prepare migration to pyproject.toml, phase out the Debian package builder to a dedicated utility program bin/mkdeb.py. It can be hooked into a corresponding poethepoet task afterwards, to improve invocation convenience if you need it.

Status
This patch is also stacked upon GH-532, so its merge order is currently on the last position of the synthetic merge queue.

got some nasty merge conflicts on this one. i'm down to merge though, once resolved

While I've used it like this before, I currently can't get versioningit to work, with a mixture of setup.py and pyproject.toml.
ValueError: Could not parse `git describe` output

-- https://github.com/kennethreitz/responder/actions/runs/11518104403/job/32064158112?pr=535#step:5:22
Let's postpone this patch to a later iteration, after migrating to a pure pyproject.toml-based metadata setup.


[backport] Support negative indices in array_split

Backport of #1454

[automatic post] Jenkins, test this please.

Jenkins CI test (for commit 3a36f2af9579e819a1956c61cce73d095f6658b6) succeeded without errors!

LGTM!


Added optional configuration of deadlines/due dates to Dependabot Widget

Added the ability to display deadlines for each severity level of dependabot alert in the table.
I'm a new contributor and relatively new to Backstage plugin development overall, so I'm not entirely familiar with all the procedures here. Please let me know if I've missed anything.
WIP: Add tests
:heavy_check_mark: Checklist

[ ] Added tests for new functionality and regression tests for bug fixes
[x] Added changeset (run yarn changeset in the root)
[x] Screenshots of before and after attached (for UI changes)
[x] Added or updated documentation (if applicable)

Went ahead and moved to ready for review - haven't looked at tests in JS yet so I'll try today and see if it's actually important to add for this change. Otherwise code is mostly complete, might get refactored a bit so the changes I made are more functionalized and stand a bit separate from the main code path

Few test failures before we can get this merged.

Thanks for the review - I got pulled onto some other stuff but I will attempt to get these tests fixed in the coming week


[4.2][BUG] select_and_order does not respond to touch events.

Hi ..
I added a field of type "select_and_order" in my project, which is a great feature when I open the control panel on desktop...
But when I open it on mobile :

I can't move items between lists.
I can't scroll item in lists.

I tried opening the control panel with more than one phone and more than one browser, and it gives the same problem.
Does anyone know solve the problem ??
Thank you very much for your help

Hello guys.
I think I have found the culprit. So here it goes my findings:
1 - we need to remove blocking css.
This css is located in field file.
.ui-sortable-handle {
            -ms-touch-action: none;
            touch-action: none;
}

2 - making sortable work on touch devices.
we need http://touchpunch.furf.com/ to make the drag and drop work on mobile. (at least is one of the solutions, i tested and works great).
3 - solving asset loading

This js library needs to be loaded after jquery and jquery-ui. Problem is when you have multiple fields in the same CRUD and any of them require jquery-ui after select_and_order. No more drag for you with touch.

You can test my solution by:

removing the css i told you before.
making this the last field in crud
added <script src="//cdnjs.cloudflare.com/ajax/libs/jqueryui-touch-punch/0.2.2/jquery.ui.touch-punch.min.js"></script> after the jquery-ui load.

and .. BAM https://recordit.co/ke51U14bsH
@tabacitu if you search for: jquery sortable touch most of top results point to that package. it hasn't been updated for years, but well in our scenario, i just pulled it and it worked.

@pxpm Yes, it works for me. Thank you very much.
But for the scroll problem, it is logically not possible to make a scroll with drag when touched, so I reduced the width of the items in list to the possibility of scroll from the side .

Maybe we should apply the drag only to the 'multi-arrow icon' and not to the full element.
That way we probably can scroll on the element body ? Thing is to drag you would need to click on the icon instead of the full element.
But we have other re-orders that works on icon too so I think this is probably the right way to do it.

I've just open #4018 to fix this issue 🙌
Thank you @pxpm for the suggestions, let's move the conversation there.


Create or Update API with 0 Retries

Hi!
I am using the terraform-provider-kong (thank you so much for it, btw!) and I'm having an issue that I was wondering if you could give me some pointers on how to proceed.
I need to set Retries to 0 when creating or updating an API, but the field "retries" disappears in the JSON Marshalling process, basically because retries=0 will be considered empty, not being then passed in the payload to Kong and falling back to the Retries default value (5) of the Kong itself.
To better explain what I mean I've added the following tests to api_tests.go that in my opinion they should pass both.
The first one is failing due to what I've said before and the second is passing and I think it is the correct behavior - when omitted it gets the kong defaults.
func Test_ApisCreateZeroRetries(t *testing.T) {
	apiRequest := &ApiRequest{
		Name:                   "zero-retries-test-" + uuid.NewV4().String(),
		Hosts:                  []string{"example.com"},
		Uris:                   []string{"/example"},
		Methods:                []string{"GET", "POST"},
		UpstreamUrl:            "http://localhost:4140/testservice",
		Retries:                0,
	}
	result, err := NewClient(NewDefaultConfig()).Apis().Create(apiRequest)
	assert.Nil(t, err)
	assert.NotNil(t, result)

	assert.Equal(t, apiRequest.Retries, result.Retries)
}

func Test_ApisCreateDefaultRetries(t *testing.T) {
	apiRequest := &ApiRequest{
		Name:                   "no-retries-test-" + uuid.NewV4().String(),
		Hosts:                  []string{"example.com"},
		Uris:                   []string{"/example"},
		UpstreamUrl:            "http://localhost:4140/testservice",
		Methods:                []string{"GET", "POST"},
	}
	result, err := NewClient(NewDefaultConfig()).Apis().Create(apiRequest)
	assert.Nil(t, err)
	assert.NotNil(t, result)

	assert.Equal(t, 5, result.Retries)
}

If we remove the "omitempty" in ApiRequest struct then the first test passes but the second one fails now because it will add "retries":0" to the payload.
type ApiRequest struct {
	Name                   string   `json:"name"`
	Hosts                  []string `json:"hosts,omitempty"`
	Uris                   []string `json:"uris,omitempty"`
	Methods                []string `json:"methods,omitempty"`
	UpstreamUrl            string   `json:"upstream_url"`
	StripUri               bool     `json:"strip_uri"`
	PreserveHost           bool     `json:"preserve_host"`
	Retries                int      `json:"retries"`
	UpstreamConnectTimeout int      `json:"upstream_connect_timeout,omitempty"`
	UpstreamSendTimeout    int      `json:"upstream_send_timeout,omitempty"`
	UpstreamReadTimeout    int      `json:"upstream_read_timeout,omitempty"`
	HttpsOnly              bool     `json:"https_only"`
	HttpIfTerminated       bool     `json:"http_if_terminated"`
}

{
    "created_at":1521279579200,
    "strip_uri":false,
    "id":"e0507fa4-14d3-435e-bdba-9d7dbbc6d190",
    "hosts":["example.com"],
    "name":"no-retries-test-31ffbdb2-dba9-4360-9947-aa127d6a643c",
    "methods":["GET","POST"],
    "http_if_terminated":false,
    "preserve_host":false,
    "upstream_url":"http:\/\/localhost:4140\/testservice",
    "uris":["\/example"],
    "upstream_send_timeout":60000,
    "upstream_connect_timeout":60000,
    "upstream_read_timeout":60000,
    "retries":0,
    "https_only":false'
}

Thank you so much!

Hey @mfamador, no worries glad you are getting use out of the provider!
I see exactly what you mean with regards to the issue.  As you say this is due to the way that default values work in go.  We will have to think about how we can change gokong so that it supports passing in 0 retries.

Just been thinking about this some more and its quite a conundrum we have.  The crux of the issue is that gokong has no way to tell the difference between the user not setting the value for retries and the user setting it to 0 (as your tests prove).
We need a way for you to signal that you actually omitting it and want the default behaviour (5 retries) vs you are explicitly setting it to 0.  This problem also applies to every field on every request object where the Kong default differs from the go default for that type.
I'd like any solution to this to be generic and not just a workaround for just this field.  It would be good to fix this issue everywhere.
Any suggestions welcome

Hi @kevholditch!
Yes, we think that we have the same problem in other fields, (like when trying to remove an anonymous user from basic auth plugin config, the plugin does not actually remove it from kong configuration) we should get a generic solution, I agree.
I'm not that proficient in go, but one solution perhaps could be using pointers, I'm not sure if it's feasible.
Thank you for your help!

@mfamador I have fixed this by changing the ApiRequest struct to take retries as a string.  This is automatically handled by kong.  This allows me to tell if you set it to 0 or do not set it as I will get the string "0" or "".
With this in place there wasn't much to change in the terraform-kong provider.  It still takes retries as an int but I have set the default value to 5.  Meaning if you dont set it I will receive 5.  So in affect when you are using the terraform provider you will always set a value for retries but it should now work.
I am closing this issue, if you could try out the new build of the terraform kong provider as I believe it is fixed.  I have updated the api test to create an api with the default retry count (not specify it) and then update it to 0 and it works fine.

Hi @kevholditch, this unfortunately does not cover another issue with empty values, when we need to change string value to empty, for example, when disallowing the anonymous consumer for an auth plugin.
For example, we first setup anonymous consumer like this:
`resource "kong_plugin" "basic_auth" {
api_id = "${kong_api.api.id}"
name = "basic-auth"
config = {
anonymous = "${kong_consumer.api_anonymous.id}"
hide_credentials = "true"
}
}`
This works without issues, but when we try to remove anonymous user by applying this config:
`resource "kong_plugin" "basic_auth" {
api_id = "${kong_api.api.id}"
name = "basic-auth"
config = {
hide_credentials = "true"
}
}`
It simply does not get removed, though terraform detects the change.
A workaround would be to delete the whole plugin config and then re-apply without anonymous user.
It seems that using pointers and introducing null value as an indication of empty field would be a solution, but I am far from being good in go.
Do you have any other solutions?

@volodymyrpavlenko yep all we need to do is set force-recreate to true for the config parameter on the kong_plugin resource.  That should fix it.  If you create a test to reproduce this issue then you can confirm that setting force recreate to true fixes it.
Basically what force recreate does it forces kong to delete and then create the resource rather than updating it in place.

Does it mean that for a very short period of time the resource would be unavailable?
We have quite high load on our services, ~600 req/s. Even 1 second downtime would fail this many requests.

yes it would mean a short period where the resource would not be available.
This case is a bit tricky as you are effectively removing the config property but what's happening is that because terraform is passing the provider the map without the old value in when we build the PATCH call we do not have the anonymous field therefore the patch will not remove it.
Maybe we need to be a bit more clever and when doing an update we call over to kong to get the current config then we could include empty fields in the PATCH requests.  We would need to build tests around this as there may be some reasons this wouldnt work


Development stalled?

Need4swede Are you still developing this project?

I am. We just welcomed our newborn at home, so I have been busy.

Ahh sorry Take your time. Congratulations on the baby


Added setting to update debounce time

didChangeConfiguration should be changed to seamlessly handle updating all possible future changes without having to add a new line per config setting

@Strum355 Thank you! The only (minor) improvement I have made was to dynamically instantiate the configuration rather than storing its state in a (global) companion object.


Update 05-04-augmented-deliberation.md

Added some nuance about dialogue and deliberation methods along with putting in some foot notes putting to good resources in this field.

This is conflicted please resubmit.


[Bug]:

Bug Description
The class HuggingFaceLLM does not have a HuggingFace token parameter, making it impossible to use this class with an LLM that is in a gated or private repo.
See HuggingFaceLLM
Version
v0.10.37
Steps to Reproduce
Evident from code HuggingFaceLLM.
Alternatively try to instatiate with a gated repo in HuggingFace, e.g. llama3
Relevant Logs/Tracbacks
No response

@pedrohernandezgeladocma you can pass any parameter using kwargs
Here's an example with llama3
import torch
from transformers import BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
)

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B-Instruct")

stopping_ids = [
  tokenizer.eos_token_id,
  tokenizer.convert_tokens_to_ids("<|eot_id|>")
]

from llama_index.llms.huggingface import HuggingFaceLLM

llm = HuggingFaceLLM(
    model_name="meta-llama/Meta-Llama-3-8B-Instruct",
    tokenizer=tokenizer,
    context_window=7900,
    model_kwargs={"token": hf_token, "quantization_config": quantization_config},
    stopping_ids=stopping_ids,
    device_map="auto",
    is_chat_model=True,
)

You can always load the model and tokenizer outside of llama-index and pass it in as well
llm = HuggingFaceLLM(model=model, tokenizer=tokenizer, ...)

@logan-markewich thanks so much this is so helpful!


Update? Fork? Successor? Community?

https://marketplace.visualstudio.com/items?itemName=OlegVSych.T4Toolbox last update is 2013.
https://github.com/olegsych/T4Toolbox/releases last release is 2017.
https://github.com/olegsych/T4Toolbox/pulls don't seem to have made it into master since 2019.
https://github.com/olegsych/T4Toolbox/issues don't see much work either...
Oleg, can we expect any updates from your side or should the community take over?

... and VS 2022 Preview (64 bit) has just been released.

@OJacot-Descombes
https://github.com/Ch0senOne/T4Toolbox for VS 2022


Not able to Install sift-cli-linux on Ubuntu Server 16.04 LTS (HVM)

Hi,
I am not able to install the sift-cli-linux on Ubuntu Server 16.04 LTS (HVM), SSD Volume Type - ami-a4dc46db on AWS Cloud
I have gone through the below link but getting the error
https://github.com/sans-dfir/sift-cli#installation
Getting the Error Which mentioned below.
root@ip-192-168-1-8:/usr/local/bin/sift# sift install
No command 'sift' found, did you mean:
Command 'gift' from package 'gnuift' (universe)
Command 'lift' from package 'lift' (universe)
Command 'swift' from package 'python-swiftclient' (main)
Command 'swift' from package 'python3-swiftclient' (universe)
sift: command not found
root@ip-192-168-1-8:/usr/local/bin/sift#
Please help me how i can install it without any error
Thanks,

Hello, thank you for your issue. Sorry to hear you are having problems.
Unfortunately given the output you've provided there are a few things wrong.

The instructions do not appear to have been followed properly.
There should not be a directory called /usr/local/bin/sift it should be a binary and based on your output it's a directory.
You should not run this as root (actually you cannot run this as root) - must use sudo.

It also looks like you are trying to install on a 16.04 server installation which will fail unless you use mode=packages-only.

As per suggestion i have tried but giving the below message, and gone through this link https://github.com/sans-dfir/sift-cli but still getting error.
root@ip-192-168-1-8:/usr/local/bin/sift# sudo sift install
sudo: unable to resolve host ip-192-168-1-8
sudo: sift: command not found
root@ip-192-168-1-8:/usr/local/bin/sift#
Thanks,

I have run below command but issue same,please help me to install the sift in Ubuntu Server 16.04 LTS (HVM), SSD Volume Type - ami-a4dc46db on AWS Cloud
root@ip-192-168-1-8:/usr/local/bin/sift# sift install -mode=packages-only
No command 'sift' found, did you mean:
Command 'gift' from package 'gnuift' (universe)
Command 'lift' from package 'lift' (universe)
Command 'swift' from package 'python-swiftclient' (main)
Command 'swift' from package 'python3-swiftclient' (universe)
sift: command not found
Thanks,

Unfortunately you still haven't followed the installation instructions properly. /usr/local/bin/sift is supposed to be a binary, not a directory, your path shows it is a directory.
Please list every command you run in order.

Thanks, I have read the Installation of sift properly and now i am able to install the Sift in Ubuntu.
but at the end of Installation I got the error which highlight in bold mentioned below...

Sent from my iPhone

On Jun 5, 2018, at 21:04, shailu75<EMAIL_ADDRESS>wrote:
Thanks, I have read the Installation of sift properly and now i am able to install the Sift in Ubuntu.
but at the end of Installation I got the error which highlight in bold mentioned below...

Reply to this email directly, view it on GitHub, or mute the thread.

Thanks Brother for your help...
Pls can you guide me some more open source tools name for Forenisc related Network & web App.
Thanks,

Hi Ekristen,
I am new to use the SFIT tool, can you please help me out any document or videos to get the knowledge on SFIT tool (SANS).
Thanks,


[BUG] Invalid cast class names don't throw exceptions

Hi. No exceptions are thrown when an incorrect class name is used for a custom cast. For example, if we have a custom App\Cast\MyCustomCast cast and create a model:
This works:
use App\Cast\MyCustomCast;

    protected array $casts = [
        'cast_test' => MyCustomCast::class,
    ];

This doesn't work, but instead of throwing an exception the casting is ignored and the code continues to execute. The model is created with the wrong (uncasted) attribute value:
use App\Cast\MyCustomCast;

    protected array $casts = [
        'cast_test' => MyWrongCustomCast::class,
    ];

I think an incorrect class name should cause an exception. Otherwise it makes debugging very difficult.

This issue has been optimized in version 3.1


Cannot use Microsoft.AspNetCore.SignalR with Microsoft.NET.Sdk.Functions due to NewtonSoft.Json version incompatibilites

Please can we have a version of NewtonSoft.Json that can be used for both Microsoft.AspNetCore.SignalR  and Microsoft.NET.Sdk.Functions?
Microsoft.AspNetCore.SignalR 1.0.0-rc1-final has a dependency on Newtonsoft.Json (>= 11.0.2), but Microsoft.NET.Sdk.Functions 1.0.13 -> Newtonsoft.Json (= 10.0.3).
What's in Newtonsoft.Json 11.0.2 that might break Microsoft.NET.Sdk.Functions? If there's nothing then let's push the SDK.Functions team to update the dependency. This issue is causing lots of problems for developers like me.

What's in Newtonsoft.Json 11.0.2 that might break Microsoft.NET.Sdk.Functions?

I don't know. I'm not familiar with how they use Newtonsoft.Json.

If there's nothing then let's push the SDK.Functions team to update the dependency.

They are completely separate from me. There is nothing I can say that will push them to do anything. You should open an issue with them.

much as I suspected, but thanks anyway. I have raised issues with the respective Microsoft teams so hopefully they with resolve soon. It's a clear case of the left hand not knowing what the right hand is doing ;-)


[BUG] Flash attention depreciated

Prerequisites

[X] I have read the documentation.
[X] I have checked other issues for similar problems.

Backend
Local
Interface Used
CLI
CLI Command
When I use flash attention via
--use_flash_attention_2 \

It gives this warning:
The model was loaded with use_flash_attention_2=True, which is deprecated and may be removed in a future release. Please use `attn_implementation="flash_attention_2"` instead.

Any idea how to work around this? Is this a concern? Or can I still proceed normally with this? Thanks
UI Screenshots & Parameters
No response
Error Logs

Additional Information
No response

its just a deprecation warning. and doesnt mean it doesnt work :) it will be fixed in one of the future releases


[ bug ] QueryException in Database controller

I love voyager package. Hope it will be stable soon, but for now I experience some problems trying to use Database tools.
Environment: Docker-compose on linux
postgres:9.5 or postgres 9:latest (from docker hub)
php-fpm 7.0 (from laradock)
nginx  (from laradock)
Reproduction steps:

Fresh Laravel 5.3
Fresh Voyager and Debugbar
Login > Tools > Database >
Edit any table OR Edit BREAD OR Add BREAD to this table

It looks like it doesn't  work with postgres 9.5 and 9.6. I didn't try to test voyager with lower versions of postgres, but it works fine with mariadb:latest
3/3
QueryException in Connection.php line 769:
SQLSTATE[42601]: Syntax error: 7 ERROR: syntax error at or near "describe"
LINE 1: describe pages
^ (SQL: describe pages)

in Connection.php line 769
at Connection->runQueryCallback('describe pages', array(), object(Closure)) in Connection.php line 725
at Connection->run('describe pages', array(), object(Closure)) in Connection.php line 350
at Connection->select('describe pages') in DatabaseManager.php line 317
at DatabaseManager->__call('select', array('describe pages')) in Facade.php line 237
at Facade::__callStatic('select', array('describe pages')) in VoyagerDatabaseController.php line 57
at VoyagerDatabaseController->edit('pages')
at call_user_func_array(array(object(VoyagerDatabaseController), 'edit'), array('table' => 'pages')) in Controller.php line 55
at Controller->callAction('edit', array('table' => 'pages')) in ControllerDispatcher.php line 44

2/3
PDOException in PDOStatement.php line 93:
SQLSTATE[42601]: Syntax error: 7 ERROR: syntax error at or near "describe"
LINE 1: describe pages
^
in PDOStatement.php line 93
at PDOStatement->execute() in Connection.php line 336
at Connection->Illuminate\Database\{closure}(object(PostgresConnection), 'describe pages', array()) in Connection.php line 762
at Connection->runQueryCallback('describe pages', array(), object(Closure)) in Connection.php line 725
at Connection->run('describe pages', array(), object(Closure)) in Connection.php line 350
at Connection->select('describe pages') in DatabaseManager.php line 317
at DatabaseManager->__call('select', array('describe pages')) in Facade.php line 237
at Facade::__callStatic('select', array('describe pages')) in VoyagerDatabaseController.php line 57
at VoyagerDatabaseController->edit('pages')
at call_user_func_array(array(object(VoyagerDatabaseController), 'edit'), array('table' => 'pages')) in Controller.php line 55
at Controller->callAction('edit', array('table' => 'pages')) in ControllerDispatcher.php line 44

1/3
PDOException in PDOStatement.php line 91:
SQLSTATE[42601]: Syntax error: 7 ERROR: syntax error at or near "describe"
LINE 1: describe pages
^
in PDOStatement.php line 91
at PDOStatement->execute(null) in PDOStatement.php line 91
at PDOStatement->execute() in Connection.php line 336
at Connection->Illuminate\Database\{closure}(object(PostgresConnection), 'describe pages', array()) in Connection.php line 762
at Connection->runQueryCallback('describe pages', array(), object(Closure)) in Connection.php line 725
at Connection->run('describe pages', array(), object(Closure)) in Connection.php line 350
at Connection->select('describe pages') in DatabaseManager.php line 317
at DatabaseManager->__call('select', array('describe pages')) in Facade.php line 237
at Facade::__callStatic('select', array('describe pages')) in VoyagerDatabaseController.php line 57
at VoyagerDatabaseController->edit('pages')
at call_user_func_array(array(object(VoyagerDatabaseController), 'edit'), array('table' => 'pages')) in Controller.php line 55
at Controller->callAction('edit', array('table' => 'pages')) in ControllerDispatcher.php line 44

Thank you!

Describe function isn't supported by Postgres. You can checkout my PR #202 with a possible solution.


Maven dist repository

Please add the javadoc & source artifacts to repository https://dl.bintray.com/javasabr/maven
Previous repository (jitpack) supported the download a sources :)

fixed for the version:
'com.spaceshift:rlib:6.8.3-Final'


Filing and Submission Sequences

---
title: Filing Period
---
sequenceDiagram
    actor Operations Team
    Operations Team->>filing-api: POST {name, start, end, etc}
    filing-api->>submission_repo: upsert_filing_period(dto)
    Operations Team->>filing-api: existing filing period updates
    filing-api->>submission_repo: upsert_filing_period(dto)
    Operations Team->>filing-api: ?delete filing period? {id}
    filing-api->>submission_repo: delete_filing_period(id)

[!IMPORTANT]
Looks like we will not need endpoints to POST/PATCH/DELETE FilingPeriods, so nothing needed for this sequence

Closing this in favor of https://github.com/cfpb/sbl-filing-api/issues/34


Compiler detection for LLVM/Clang with Visual Studio integration

Provided are some changes to the compiler detection with cmake.
The invocation on the command line (cmd.exe) actually only requires for a 32bit build
cmake .. -TLLVM-vs2014

A 64bit build is generated with
cmake .. -TLLVM-vs2014 -G"Visual Studio 14 2015 Win64"

Other options should not be required on a clean cmake run. The additional comparisons are for the case when the initial invocation is missing the -T argument, and hence everything get's really messy. This happens in particular when an existing solution is changed afterwards, e.g. by invoking cmake again with other options, but might also happen by other manual changes in the solution and the automatically triggered cmake run.
The check for MSVC must be the first. Otherwise the MSC compiler will fail at the first #warning. MSC uses a #pragma message for the same purpose.
The standard library of MSVC can be checked by the compiler version _MSC_VER since its bundled. However, detecting MSVC is not sufficient; _clang_ must also be present. Unfortunately, also this might not be enough since the compiler could be changed manually, hence the platform toolset must also start with LLVM-vs... to ensure the correct MSVC-internally provided configuration with the LLVM/Clang integration.

For reference, this pull-request is related to issue #201.

Once this passes, could you please squash the three first commits together (since they are just reworks one of another), and I'll merge this.

Oh yes, and if you could rebase on top of the current develop branch, it would be really nice.

Hmm, that didn't work as expected...

 At some point I managed to push my squashed changes to my develop branch.
So just take that squashed commit db9daa69aff2c17cf37841f8d3b38644434cdfc0 on my develop branch. It should merge with your develop branch.

Well, I should clean that up later. At some point I managed to push my squashed changes to my develop branch.

You want to push to your windows_build branch, not your develop branch. Pushing to your windows_build branch will update this pull request.

Yeah, well, adding the + sign before the branch to enforce the push after squashing also helps a lot...

Yeah, well, adding the + sign before the branch to enforce the push after squashing also helps a lot...

You can also use git push -f or git push --force


TypeError: unicode argument expected, got 'str'

Using TensorFlow backend.
Loading weights.
('Weights Header: ', 0, 2, 0, array([32013312]))
Parsing Darknet config.
Traceback (most recent call last):
File "convert.py", line 262, in
_main(parser.parse_args())
File "convert.py", line 83, in _main
unique_config_file = unique_config_sections(config_path)
File "convert.py", line 53, in unique_config_sections
output_stream.write(line)
TypeError: unicode argument expected, got 'str'

@hejie the test environment is Python 3.5.2, I guess your environment is Python2.7.

if your environment is Python2.7, you can change code :
line 10
import io
and
line 47
 output_stream·=io.StringIO()
to
line 10
from io import BytesIO as StringIO
line 47
 output_stream·= StringIO()

@zhoul14 worked for me, thanks!

@hejie the test environment is Python 3.5.2, I guess your environment is Python2.7.

Make sure you use python3

If you need both python2 and python2 support you can/should opt for six.StringIO
import six

six.StringIO()

if your environment is Python2.7, you can change code : line 10 import io and line 47  output_stream·=io.StringIO()
to line 10 from io import BytesIO as StringIO line 47  output_stream·= StringIO()

I used this tip to solve a problem in the plugin to generate migrations for laravel in MySQL Workbench.
Thank you very much!


Parallel I/O and computation? (with Dask, etc.)

I am getting ~1 Terabyte of VTK files (for a class project). Processing them takes forever... I wonder if vtki is able to perform parallel I/O (assuming that you have a parallel file system like Lustre and GPFS)?
I notice that VTK/ParaView supports parallel I/O:
https://www.paraview.org/ParaView/index.php/Parallel_I/O
Is this interface exposed to vtki? VTK seems to have both MPI and non-MPI implementations, so MPI probably doesn't have to be a dependency.
An alternative way is using Dask. This is how Xarray handles parallel I/O and computation (http://xarray.pydata.org/en/stable/dask.html).
This is a follow-up of #175, but is about the performance side.

First, does the ~1 TB of files make one single mesh?

does the ~1 TB of files make one single mesh?

They are actually individual VTK files, each represents a time step of an evolving system. I can probably hack it with multiprocessing or Dask on my own. I just wonder if vtki has any sort of built-in parallelism.
The computation pattern is pretty simple. Mostly just read the data and compute the sum.

Just to note, the vtkXMLP<type>Reader implemented for parallel file formats to solve #175 actually reads the parallel files in series, from the VTK docs:

vtkXMLPUnstructuredGridReader reads the PVTK XML UnstructuredGrid
file format.  This reads the parallel format's summary file and then
uses vtkXMLUnstructuredGridReader to read data from the individual
UnstructuredGrid piece files.  Streaming is supported. The standard
extension for this reader's file format is "pvtu".

Since vtki simply wraps a call to that reader in a standard routine, this means vtki currently does not support parallel IO/processing. I do think we could expand on the fact that those Parallel format readers support streaming to implement a parallel pipeline - the VTK discourse would have folks that are much more experienced than I on how to actually implement this.
Will ~1 TB of data, I'd personally recommend setting up ParaView on your parallel file system to do this processing as this is exactly what ParaView is built for... otherwise, this would be a feature we could probably outline implementing for future development. I'd recommend posting this question on the VTK discourse to see if 1) you need to compile VTK with MPI support yourself since the wheels on PyPI and Anaconda do not have MPI support enabled, and 2) if this big of a dataset is able to be processed in VTK alone without ParaView.

I just wonder if vtki has any sort of built-in parallelism.

There's no built-in parallelism support in vtki currently - perhaps a new feature request?

They are actually individual VTK files, each represents a time step of an evolving system. I can probably hack it with multiprocessing or Dask on my own.

Perhaps you could define a function that you fire off in parallel that only cares about a single file, pass that file to the function, have vtki read/process that single file, save the results

@banesullivan Thanks, that's very helpful information! I am not sure how many VTK users are interested in "big data" problems and whether a new feature is worth the effort. But if you want to hack some parallelism I am happy to discuss. Xarray has excellent parallel I/O performance and can probably serve as a reference & benchmark (see my blog post for a real-world example).
Feel free to close or keep this issue. Thanks again :)

I am not sure how many VTK users are interested in "big data" problems

Generally speaking, a big portion of the VTK community is interested in "big data" problems - these folks tend to use ParaView though because ParaView is built for use on supercomputers in a client-server manner.

But if you want to hack some parallelism I am happy to discuss.

At this time, I don't think I'll be able to go at this... 😞 Parallelism is something I think vtki should definitely support down the road but I don't see @akaszynski or I being able to take on that effort on our own.
My primary focus for development lately has been to make an intuitive, easy to use tool for researchers, students, and generally novice programmers to use in visualizing generally large spatial datasets (large in the sense of millions to 10s of billions of points/cells) but not so large that the dataset needs to be processed on a supercomputer - in my experience datasets of this size can be handled by a laptop with a standard GPU quite easily.
Worth Noting
The folks making VTK have put considerable thought into this when building their dataset adapter and numpy wrapping module directly in the VTK Python package - there's a suite of algorithms in that module that can handle numerical operations in parallel. Perhaps we could adapt some of those features down the road.
There's a series of blog posts that dive into this (see this 4 part series). I can't remember if the blog posts talk much about the parallel capabilities of those algorithms, but they are definitely worth checking out for this effort.
Moving forward
I'm going to close this issue for now with a tag to mark it for reevaluation down the road.


Support for government fuel price API in Queensland,Australia

Checklist

[X] I have filled out the template to the best of my ability.
[X] This only contains 1 feature request (if you have multiple feature requests, open one feature request for each feature request).
[X] This issue is not a duplicate feature request of previous feature requests.

Is your feature request related to a problem? Please describe.
The canonical source of fuel prices in Queensland, Australia is the API described at https://www.fuelpricesqld.com.au/ and this will always provide the most accurate and current prices available.
Describe the solution you'd like
Support for the https://www.fuelpricesqld.com.au/ API.
Describe alternatives you've considered
This problem does not lend itself to an alternative. I currently use an IOS app called Petty which accesses this information.
Additional context
Scraped data from the listed apps lags behind the API considerably and I would like to create automations which notify me of good prices close to where I am or to monitor particular petrol stations and send me notifications when the prices drop.

Hi,
As this requires an API token to interact with, its not something I will look at immediately, but will be added for future consideration.
Thanks

Linking related issue - https://github.com/pantherale0/pyfuelprices/issues/9

This isn't possible, not because of the API token, but because I'm in the UK and therefore the dataset isn't available to allow me to register. Plus there appears to be a cost of some description? (At least according to the API).


Pick a cute component library

All component libraries so far are not really useable for one reason or another. Yay
Requirements

Accessible
Works and can be imported properly
Not horribly outdated
Not a React exclusive
Not totally unfinished

Very nice to have

Has a select/autocomplete/dropdown where you can search for things

I tried out elix https://github.com/tuwien2020/tgi-pages/pull/75 , but ran into an issue with the elix-filter-combo-box
https://github.com/elix/elix/issues/202

Vuetify 3 really isn't ready yet https://github.com/tuwien2020/tgi-pages/pull/77
I suppose it'll become useable sometime this year.

Microsoft's Fast might be worth taking another look at, since it now has a search box https://github.com/microsoft/fast/issues/4967#event-5585251779

Ant Design is also slowly improving.
See https://github.com/ant-design/ant-design/issues/22343

I took a look at https://github.com/coreui/coreui-vue

no dark theme (at least nothing obvious in the documentation, also where is the search bar)
annoying auto-closing sidebar, which makes the documentation harder to use than it should be

Here is a list of other Vue component libraries:

https://oruga.io/documentation/#introduction
https://www.inkline.io/docs/preface
https://varlet.gitee.io/varlet-ui/#/en-US/home
https://www.fast.design/
https://www.agnosticui.com/docs/components/buttons.html
https://www.primefaces.org/primevue/setup
https://vuestic.dev/en/ui-elements/select
https://antdv.com/components/upload
https://www.naiveui.com/en-US/os-theme/components/upload
https://github.com/vuetifyjs/vuetify
https://vant-contrib.gitee.io/vant/v4/#/en-US/uploader

TODO: Check which of those actually do <button type="button" and other reasonable stuff like that.

https://vuesax.com/ is exceedingly pretty, but sadly doesn't seem to pay as much attention to other details like developer experience, accessibility, etc.

TODO: Check if their dropdown component actually updates when you update the :value or if it's as cursed as the browser one

https://headlessui.com/?ref=heydesigner
Is another one

TODO: Check if they have async buttons. Like a button that you can't spam click until the async event is over...
(Or maybe I'll just write myself that as a utility function?)


Handle pubsub messages in multi instance scenario

To fix #91, we have to handle how pubsub messages are routed to the client. To solve this, the leader instance will be asked to keep pubsub connections for all clients. Pubsub generated from other instances should forward pubsub messages to the leader instance.
Tasks:

add new configuration envvar for specifying the leader pubsub URL
forward pubsub messages generated from the instance to the leader (which includes subscriptions and cloudcode-generated messages)
leader should be aware that the messages coming from the non-leaders are generated server side and not generated client side

Revisiting this issue, in https://github.com/SkygearIO/skygear-server/pull/127 we added the configuration on slave and disabled the whole pubsub and cron feature at slave mode.
So the do @cheungpat think following two feature are still meaningful to implement

forward pubsub messages generated from the instance to the leader (which includes subscriptions and cloudcode-generated messages)
leader should be aware that the messages coming from the non-leaders are generated server side and not generated client side

Or better tol provide a deployment example on how to route the traffic correctly using the program like nginx?

@rickmak Thanks for bringing it to my attention.
I think the feature is complete in skygear-server and py-skygear to support multiple instances. Agree that we should have deployment example. Where to put it?

forward pubsub messages generated from the instance to the leader (which includes subscriptions and cloudcode-generated messages)

Cloudcode generated messages are forwarded to leader using PUBSUB_URL. In slave mode, skygear-server does not generate notifications for subscriptions

leader should be aware that the messages coming from the non-leaders are generated server side and not generated client side

This is just a reminder not to cause infinite loop forwarding notifications among multiple instances. Since only the leader will act as pubsub hub. This is a non-issue.

Let put the example here: https://github.com/skygear-demo/multiple-server


cluster setup fix

Fixes two problems:

Feature gates not set for control plane components, which prevented pod tests from running on Kubernetes 1.13 where "skip attach" was alpha and thus by default enabled; we rely on that feature. We didn't have proper tests for that, but now we do and full CI testing fails for "devel".
Kubernetes version not completely locked down (not nice, but not critical at this time).

CI testing of the branch shows that the fix works: https://cloudnative-k8sci.southcentralus.cloudapp.azure.com/job/pmem-csi/job/cluster-setup-fix/


linkedin/datahub-frontend-react:debug not found when executing docker/dev.sh

Describe the bug
executing docker/dev.sh fails, error message is:
WARNING: Some service image(s) must be built from source by running:
docker compose build datahub-frontend-react datahub-gms mysql-setup elasticsearch-setup kafka-setup
Error response from daemon: manifest for linkedin/datahub-frontend-react:debug not found: manifest unknown: manifest unknown
To Reproduce
Steps to reproduce the behavior:

git clone master branch
cd datahub
./gradlew build -x test
docker/dev.sh
See error: Error response from daemon: manifest for linkedin/datahub-frontend-react:debug not found: manifest unknown: manifest unknown

Expected behavior
doing this was well yesteday, but it fails today,
Desktop (please complete the following information):

OS: mac

linkedin/datahub-frontend-react:debug  does not exist now in hub.docker.com

Hi @yangjiandan that's an internal dev image that we build locally and don't host- the DEBUG tag is build locally only

@laulpogan Thanks for your quickly reply very
much. Could you tell me how to build Debug tag

I can not find related code or script about build debug images, @laulpogan @mhw @cburroughs @koconder please give me a hint, thanks.

Hi @yangjiandan  this seems like a troubleshooting issue, rather than a bug. We're happy to provide community support on our Slack channel, but currently reserve git issues for bugs.
If you're still having trouble, please join us at slack.datahubproject.io and we can troubleshoot there. For now, I'm going to close this issue.


[FIX] [ANCHOR: BE PYTHON] Send Confirmation Email to Users When They Unsubscribe From A Newsletter

Description
As a User, i want to receive a confirmation email when I  unsubscribe from a newsletter so I can know my unsubscription was successful.
Purpose
Implement a feature that sends a confirmation email to users when they unsubscribe from a newsletter. The email should confirm that their unsubscription was successful, provide information on how to resubscribe if they choose to change, and optionally ask for feedback on why they unsubscribed.
Acceptance Criteria
Sending a confirmation message on the website after the user unsubscribes. However, this does not provide a lasting record that the user can refer to later.
Not sending any confirmation, which might lead to user dissatisfaction and increased support queries from users unsure about their subscription status.
Expected Outcome
This feature could use the existing email service infrastructure, and the email template should align with the platform's branding. Ensuring that this email is consistent with other communications will help maintain a cohesive user experience.

Assigned to Trevor Job


fix: missing wasi files

#37  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @Brooooooklyn and the rest of your teammates on  Graphite


Use DataKey instead of Descriptor

Closes #145
Descriptor is overloaded with a DescriptorDocument, which contains many DataKeys. A DataKey is the key to a data field, describing how it should be read.

Is this still in progress?

@coretl I think it was draft as had conflicts with the bool signal, I've rebased and will see if the CI is happy

I started writing about what we would need to support tables, but stopped because I think it should now be a separate ticket.

Has that ticket been created?


Adding suite of tests for the e-mail validation field in the users model

This further strengthens our test suites from 21 tests to 35 already!

@lirantal Not that having more tests is a bad thing, but we are using validator so wouldn't this be overkill?  Guessing that module has the proper tests already.  Unless there is a way to add email outside of validator, which would probably be bad.   The more tests we have the longer CI takes.

LGTM.

@trainerbill that's exactly the point - imagine a scenario that someone in the future will replace the validator package with something else? or that we upgrade to a newer validator package? these tests ensure that the expected behavior is always maintained.
Also I wouldn't worry that much about tests time and rather would aim for a high code coverage.


Addition of EventListener

Enhancement
Additions
Added EventListener to listen up for 5 events.

ready - fired when worker is connected to server.
error - fired when error occurs from server.
arrive - fired when message arrives/received.
seen - fired when message is marked as seen.
delete - fired when message is deleted from server.

the events can be set by the on() method and the worked can be stopped after clearing all events via close() method.
Example Usage
mailjs.on("seen" , (msg : IMessagesResult)=>console.log(msg));
mailjs.on("arrive" , (msg : IMessagesResult)=>console.log(msg));

Changes

tried to add types
updated docs
updated the discord server url
added the eventsource minified build from eventsource

Testing
This latest build is tested on browser and nodejs with both ESM & CommonJs syntax and it seems to be working.
Can be tested once again for extra checks on different branch.

Merged,
Thanks.
I also updated the documentation and code to match the others.
b81a740ff2daaa41122793f142fbf17f7cc0754e


Scrape lobbyist employer expenditures

The current lobbyist scrape captures employer name (ClientName), but there is some additional metadata we could capture: https://docs.google.com/spreadsheets/d/1-c2Ony5hGjpchOfwPKhYyoJ9FPji0dfkgt0mTcBS9mo/edit?usp=sharing
Clarify with Marjorie which employer metadata lobbyist scrapes should include, then capture it.

@antidipyramid I'll email Marjorie to see what, if any, additional data about the employers she wants in the scrapes.

🚨 Glad I asked! We need to scrape the lobbyist employer expenditures from https://login.cfis.sos.state.nm.us/#/lobbyistexpendituresearch/31.

@hancush There is a good amount of data processing going on in lobbyists.mk.
Do we want to do some kind of processing on the employer expenditures?

Great question, @antidipyramid. Some context on lobbyist (and lobbyist employer) scraping: The search interface does not include one very important piece of information: the beneficiary of the expenditure / contribution. So, the original lobbyist scrape downloads all of a lobbyist's filings, then parses information out of those PDFs.
It looks like lobbyist employers file the same information in the same format, e.g., https://login.cfis.sos.state.nm.us//ReportsOutput//LAR/4a27c051-7b49-456a-9936-98d595384a08.pdf
I wonder if we could simply plug them into the existing pipeline (perhaps with some modifications, since rather than a lobbyist associated with a client [employer], there will only be clients [employers])?

Looks like there's an https://login.cfis.sos.state.nm.us/api//ExploreClients/Disclosures endpoint that gets filings for lobbyist employers (while it's https://login.cfis.sos.state.nm.us/api//ExploreClients/Fillings for lobbyists) – see the network request when you click on "Filings" here: https://login.cfis.sos.state.nm.us/#/exploreClientDetailPublic/mDJ2oXreU_grMhUIIWBeHHwquY7yN_7SNrmbDh6rMxI1/10/2024
If you can modify the script that retrieves filings so it works for both lobbyists and lobbyist employers, I think you can use the rest of the pipeline (PDF parsing) as is, or close to it! What do you think?


Card without Header

CardHeader should be optional as CardFooter is.
Currently when Card is used without CardHeader, the padding is off.
Simple fix would be replacing "pt-0" from CardContent with "pb-0" in CardHeader.

This issue has been automatically closed because it received no activity for a while. If you think it was closed by accident, please leave a comment. Thank you.