[HUDI-1269] Make whether the failure of connect hive affects hudi ingest process configurable

Make whether the failure of connect hive affects hudi ingest process configurable
Tips

Thank you very much for contributing to Apache Hudi.
Please review https://hudi.apache.org/contributing.html before opening a pull request.

What is the purpose of the pull request
Currently, In an ETL pipeline(eg, kafka -> hudi -> hive), If the process of hudi to hive failed, the job is still running.
I think we can add a switch to control the job behavior(fail or keep running) when kafka to hudi is ok, while hudi to hive failed, leave the choice to user. since ingesting data to hudi and sync to hive is a complete task in some scenes.
Brief change log
hudi-spark-datasource/hudi-spark-common/src/main/java/org/apache/hudi/DataSourceUtils.java
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncConfig.java
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java
Verify this pull request
(Please pick either of the following options)
This pull request is a trivial rework / code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end.
Added HoodieClientWriteTest to verify the change.
Manually verified the change by running a job locally.

Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

Codecov Report

Merging #2443 (0d98db7) into master (e3d3677) will decrease coverage by 41.04%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master   #2443       +/-   ##
============================================
- Coverage     50.73%   9.68%   -41.05%
+ Complexity     3064      48     -3016
============================================
  Files           419      53      -366
  Lines         18797    1930    -16867
  Branches       1922     230     -1692
============================================
- Hits           9536     187     -9349
+ Misses         8487    1730     -6757
+ Partials        774      13      -761

Flag
Coverage Δ
Complexity Δ

hudicli
?
?

hudiclient
100.00% <ø> (ø)
0.00 <ø> (ø)

hudicommon
?
?

hudiflink
?
?

hudihadoopmr
?
?

hudisparkdatasource
?
?

hudisync
?
?

huditimelineservice
?
?

hudiutilities
9.68% <ø> (-59.80%)
0.00 <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

The diff coverage is n/a.

@@             Coverage Diff              @@
##             master   #2443       +/-   ##
============================================
- Coverage     50.73%   9.68%   -41.05%
+ Complexity     3064      48     -3016
============================================
  Files           419      53      -366
  Lines         18797    1930    -16867
  Branches       1922     230     -1692
============================================
- Hits           9536     187     -9349
+ Misses         8487    1730     -6757
+ Partials        774      13      -761

Flag
Coverage Δ
Complexity Δ

hudicli
?
?

hudiclient
100.00% <ø> (ø)
0.00 <ø> (ø)

hudicommon
?
?

hudiflink
?
?

hudihadoopmr
?
?

hudisparkdatasource
?
?

hudisync
?
?

huditimelineservice
?
?

hudiutilities
9.68% <ø> (-59.80%)
0.00 <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

 Can you please address the feedback and update the PR.

Thank you for your attention. I want to make changes after verification in the production environment. I will update in the near future
------------------ Original ------------------
From: Sivabalan Narayanan<EMAIL_ADDRESS>Date: Wed,Feb 17,2021 0:18 PM
To: apache/hudi<EMAIL_ADDRESS>Cc: liujinhui<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [apache/hudi] [HUDI-1269] Make whether the failure of connect hive affects hudi ingest process configurable (#2443)

cool, sounds good 👍

@nsivabalan @yanghua  please help review

@vinothchandar @yanghua : A general doubt on supporting this config knob. As quoted above if connection fails and if we set this config of interest to true, then IMetaStoreClient will not be initialized within HoodieHiveClient. So, for all methods in this class we have to handle null.
For eg createTable, getTableSchema, etc . I am not sure how to go about this.
One approach we can take.
for some operations we can do our best to return empty or false, or something of those sort.
For eg, getPartitionEvents, doesTableExist we can return empty list / return false.
addPartitionsToTable we can just let it as no-op.
But for other operations like getTableSchema() we can throw exceptions.
What are you thoughts.

Just to confirm, we don't need to fix anything wrt HoodieHiveClient wrt this patch is it?

yes

@nsivabalan  Thanks for the suggestion, I will make changes

I have pushed a commit to add tests and address the roTableRename. But not sure if we can make the Config span any exception in general or just connect exception.
@vinothchandar @n3nash : Vino yang suggested we can ignore any exception w/ hive sync in general and proceed if the newly introduced config is set. As of now(in this patch), we only consider connect exceptions. Any thoughts.

@yanghua : I synced up w/ Vinoth on this. here is our idea. we will make the config (naming) generic for now. but lets not make it wider. bcoz, if sync explicitly fails probably user need to get notified. Since this diff was raised mainly for connect execption, we will leave it as is. But in future if some customer needs to ignore sync exceptions as well, we can consider silently ignoring those as well. wdyt?
If you agree, only fix we need for now is to fix the config naming. I can work on it and push in a small commit.

@yanghua : I synced up w/ Vinoth on this. here is our idea. we will make the config (naming) generic for now. but lets not make it wider to ignore exceptions for sync. bcoz, if sync explicitly fails probably user need to get notified. Since this diff was raised mainly for connect execption, we will leave it as is. But in future if some customer needs to ignore sync exceptions as well, we can consider silently ignoring those as well(so that we don't introduce a new config and can use the same config). wdyt?
If you agree, only fix we need for now is to fix the config naming. I can work on it and push in a small commit.

OK, agree.
The original intention of my idea is actually to not want the failure of synchronization to affect hudi's tasks. We have experienced situations where Hive is unavailable and all jobs fail. Then we completely ignored the failure, but when it failed, we implemented log warnings.
But as a realization of the community, I agree with you.

got it, thanks. I have addressed all pending feedbacks. Will land once CI turns green.


Applications: add minversions

Add minversion to appslit json. Show update warning if client version is too low.

Added minApiVersion to appslit.json.
If api version is not supported the following is shown:

Talk to me, I'll explain how to test.

checked


Add zsync3

Add https://github.com/lalten/zsync

Hello @bazelbuild/bcr-maintainers, modules without existing maintainers (zsync3) have been updated in this PR. Please review the changes.


build: add restriction Golang >= 1.10

Fixes #148
Add build tag to restrict minimum golang version to 1.10, in case we run into
Netns mix problem again and again.
Signed-off-by: Wei Zhang<EMAIL_ADDRESS>

I agree that we should use newer golang releases. There are several lock os thread related fixes in the 1.10 release.
But I'm not sure if the source code is the right way to add the restriction. How about adding it to the Makefile instead?

But I'm not sure if the source code is the right way to add the restriction. How about adding it to the Makefile instead?

This is the easier way to block user from compling it with an old Golang version.
Any good idea relating to your Makefile? Some golang version judgement?

@WeiZhang555 Yes, I was thinking about adding go version check in the Makefile. It will be easier to find in future, assuming we'll change it again some time later. And it doesn't need to compile a few files and then fail when reaching cli/oci.go. wdyt?

I quite like the code addition - but, agree that it will be easier to find and trigger earlier if in the Makefile.

@WeiZhang555  I think what @bergwolf concerns is, next time, if we want to add +build go1.11, how could we find this tag and update it rather than add it in another source file.

It will be easier to find in future, assuming we'll change it again some time later.

Both OK for me. Let's have a try...

Travis is having a big snooze - I don't see a build running at https://travis-ci.org/kata-containers/runtime/builds

oops, wrong PR - that travis comment was aimed at #764...

I'm a terrible Makefile writer, just tried and didn't get a nice result, I think I need to ask for some help here, does anyone want to pick up the Makefile part?  @bergwolf Any interest? :joy:

@WeiZhang555 Here is an example of how it's done in other projects. It's a bit complicated but feel free to borrow the script from there since ipfs is MIT license (which allows redistribution under a different license).

/me looks to @jodh-intel as a Makefile god to do review and input ;-)

How about something like this (but note the FIXME in there :)... ?
diff --git a/Makefile b/Makefile
index 96b26dc..98591e0 100644
--- a/Makefile
+++ b/Makefile
@@ -15,6 +15,8 @@ done)
 GOARCH=$(shell go env GOARCH)
 HOST_ARCH=$(shell arch)

+include golang.mk
+
 ifeq ($(ARCH),)
 	ARCH = $(GOARCH)
 endif
diff --git a/golang.mk b/golang.mk
new file mode 100644
index 0000000..6cadfdd
--- /dev/null
+++ b/golang.mk
@@ -0,0 +1,37 @@
+#
+# Copyright (c) 2018 Intel Corporation
+#
+# SPDX-License-Identifier: Apache-2.0
+#
+
+# FIXME: this should be a query of versions.yaml (languages.golang.version)
+golang_version_min_major = 1
+golang_version_min_minor = 10
+
+# for error messages
+golang_version_needed=$(golang_version_min_major).$(golang_version_min_minor)
+
+golang_version_raw=$(shell go version 2>/dev/null)
+
+ifeq (,$(golang_version_raw))
+    $(error "ERROR: cannot determine golang version")
+endif
+
+# determine actual version of golang
+golang_version=$(subst go,,$(word 3,$(golang_version_raw)))
+
+golang_version_fields=$(subst ., ,$(golang_version))
+
+golang_version_major=$(word 1,$(golang_version_fields))
+golang_version_minor=$(word 2,$(golang_version_fields))
+
+golang_major_ok=$(shell test $(golang_version_major) -ge $(golang_version_min_major) && echo ok)
+golang_minor_ok=$(shell test $(golang_version_major) -eq $(golang_version_min_major) -a $(golang_version_minor) -ge $(golang_version_min_minor) && echo ok)
+
+ifeq (,$(golang_major_ok))
+    $(error "ERROR: golang major version too old: got $(golang_version), need atleast $(golang_version_needed)")
+endif
+
+ifeq (,$(golang_minor_ok))
+    $(error "ERROR: golang minor version too old: got $(golang_version), need atleast $(golang_version_needed)")
+endif

@WeiZhang555 - just a reminder - are you OK with the Makefile fragment from @jodh-intel If you are not able to apply, maybe we can pick this up for you?

@grahamwhaley The Makefile looks quite good. Yes, you can pick this up for me, since I am in vacation in next 3 weeks. Thank you!

@jodh-intel - for the Makefile FIXME, I've added:
golang_version_min=$(shell yq r versions.yaml languages.golang.version)

ifeq (,$(golang_version_min))
    $(error "ERROR: cannot determine minimum golang version")
endif

golang_version_min_fields=$(subst ., ,$(golang_version_min))

golang_version_min_major=$(word 1,$(golang_version_min_fields))
golang_version_min_minor=$(word 2,$(golang_version_min_fields))

and that seems to work - see any issues with that? (apart from we then require yq, but hey ho, we should fail the build without it cleanly).
I'll put together a new PR

actually, whilst there @jodh-intel , I was a little thrown by the intention of the golang meta data in the yaml - any insights welcome :-) I was going to bump the version to 1.10.4, as listed on golang.org at present. What do I do with the meta.newest-version ?? is that a restriction/cap on the max version we should use - in which case we (I) can encode that lookup into the Makefile as well whilst we are there.
  golang:
    description: "Google's 'go' language"
    notes: "'version' is the default minimum version used by this project."
    version: "1.10.4"
    meta:
      newest-version: "1.10"

@grahamwhaley - the original intention was:

languages.golang.version: minimum required version
languages.golang.meta.newest-version: newest version that we know to work (not necessarily the newest available version).

The ideal would be that we'd test both. However, the tests and packaging are currently using languages.golang.meta.newest-version (only, afaik) [1].
Feel free to add a comment in the YAML (you could even add languages.golang.meta.newest-version-description or something [2].

[1] - Note that we can easily update the the newest-version simply by changing the value in versions.yaml in a PR since the CI will read it and install that version of golang.
[2] - That's a bit hacky as ideally we'd make it languages.golang.meta.newest. and add version and description below it with an anchor to keep compat with the current languages.golang.meta.newest-version.

@grahamwhaley - Your yq usage seems fine to me, although strictly yq should then be added to:

https://github.com/kata-containers/documentation/blob/master/Developer-Guide.md#requirements-to-build-individual-components

The only question is which version yq should be reading from versions.yaml :)

@WeiZhang555 ping.

Close this as we have a better pr #806 :-)

Just came across this and noticed that the issue number is wrong. No idea what it should be, but it's not #148 ;)

Actually, it might be #148 - see https://github.com/kata-containers/runtime/pull/806 and
https://github.com/kata-containers/runtime/issues/148#issuecomment-423071757
Unless we find something got closed that should not - it all feels benign now anyway.

Hmm. Too late to change it now, but I think a separate issue would have been best, linked to from the PR and #148.


[Introspection] Panic "No id field defined!" (caused by: no id at all)

Error: Schema parsing thread 'main' panicked at 'No id field defined!', src\libcore\option.rs:1190:5
panic-no-id-field-defined

Introspection failed: Introspected schema can't be parsed.
Error: Schema parsing thread 'main' panicked at 'No id field defined!', src\libcore\option.rs:1190:5
stack backtrace:
   0: backtrace::backtrace::dbghelp::trace
             at C:\Users\VssAdministrator\.cargo\registry\src\github.com-1ecc6299db9ec823\backtrace-0.3.37\src\backtrace/dbghelp.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at C:\Users\VssAdministrator\.cargo\registry\src\github.com-1ecc6299db9ec823\backtrace-0.3.37\src\backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src\libstd\sys_common/backtrace.rs:76
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src\libstd\sys_common/backtrace.rs:60
   4: core::fmt::write
             at src\libcore\fmt/mod.rs:1030
   5: std::io::Write::write_fmt
             at src\libstd\io/mod.rs:1412
   6: std::sys_common::backtrace::_print
             at src\libstd\sys_common/backtrace.rs:64
   7: std::sys_common::backtrace::print
             at src\libstd\sys_common/backtrace.rs:49
   8: std::panicking::default_hook::{{closure}}
             at src\libstd/panicking.rs:196
   9: std::panicking::default_hook
             at src\libstd/panicking.rs:210
  10: std::panicking::rust_panic_with_hook
             at src\libstd/panicking.rs:473
  11: std::panicking::continue_panic_fmt
             at src\libstd/panicking.rs:380
  12: rust_begin_unwind
             at src\libstd/panicking.rs:307
  13: core::panicking::panic_fmt
             at src\libcore/panicking.rs:85
  14: core::option::expect_failed
             at src\libcore/option.rs:1190
  15: once_cell::imp::OnceCell<T>::initialize::{{closure}}
  16: once_cell::imp::initialize_inner
  17: once_cell::imp::OnceCell<T>::initialize
  18: prisma_models::fields::Fields::id
  19: query_core::schema_builder::object_type_builder::ObjectTypeBuilder::many_records_arguments
  20: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &mut F>::call_once
  21: <core::iter::adapters::Map<I,F> as core::iter::traits::iterator::Iterator>::next
  22: <alloc::vec::Vec<T> as alloc::vec::SpecExtend<T,I>>::from_iter
  23: query_core::schema_builder::object_type_builder::ObjectTypeBuilder::new
  24: query_core::schema_builder::query_schema_builder::QuerySchemaBuilder::new
  25: prisma::cli::CliCommand::execute
  26: <std::future::GenFuture<T> as core::future::future::Future>::poll
  27: tokio_executor::enter::Enter::block_on
  28: std::thread::local::LocalKey<T>::with
  29: tokio::runtime::threadpool::Runtime::block_on
  30: prisma::main
  31: std::rt::lang_start::{{closure}}
  32: std::rt::lang_start_internal::{{closure}}
             at src\libstd/rt.rs:49
  33: std::panicking::try::do_call
             at src\libstd/panicking.rs:292
  34: __rust_maybe_catch_panic
             at src\libpanic_unwind/lib.rs:80
  35: std::panicking::try
             at src\libstd/panicking.rs:271
  36: std::panic::catch_unwind
             at src\libstd/panic.rs:394
  37: std::rt::lang_start_internal
             at src\libstd/rt.rs:48
  38: main
  39: _tmainCRTStartup
  40: mainCRTStartup
  41: sqlite3GenerateConstraintChecks
  42: sqlite3GenerateConstraintChecks
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    at Object.<anonymous> (C:\ProgramData\nvm\v10.16.3\node_modules\prisma2\build\index.js:70179:23)
    at Generator.throw (<anonymous>)
    at rejected (C:\ProgramData\nvm\v10.16.3\node_modules\prisma2\build\index.js:70074:65)
    at process._tickCallback (internal/process/next_tick.js:68:7)

probably caused by
model oseba {
  alkotest                    Float?
  alkotest_d                  String
  drzavljanstvo               String
  id_nesreca                  nesreca
  poskodba                    String?       @default("NULL")
  povzrocitelj_ali_udelezenec String?       @default("NULL")
  spol                        String
  starost                     Boolean?
  starost_d                   String?       @default("NULL")
  strokovni_pregled           Float?
  strokovni_pregled_d         String
  upravna_enota               upravna_enota
  varnostni_pas_ali_celada    String?       @default("NULL")
  vozniski_staz_d             String
  vozniski_staz_LL            Boolean?
  vozniski_staz_MM            Boolean?
  vrsta_udelezenca            String?       @default("NULL")
}

source: https://github.com/prisma/introspection-engine-output/blob/master/more_mysql/Accidents.log
possible explanation:

no primary key defined
no unique field at all

Collection of schemas where the complete absence of an @id or @@id was confirmed as the source of the panic (via adding a foo Int @id):
https://github.com/prisma/introspection-engine-output/blob/master/mysql_127/Accidents.log
This is the only (!) schema where the absence of @id or @@id caused a panic - all others are already failing during validation, which is tracked in: https://github.com/prisma/prisma2/issues/1083

Which leads to the real question:
Why does this not fail in validation already?

Seems the validation does not trigger because the schema could essentially be rewritten as:
model foo {
  i    Int   @id
  bars bar[]
}

model bar {
  i   Int
  foo foo
  baz baz
}

model baz {
  i    Int   @id
  bars bar[]
}

As soon as foo or baz are removed from bar, the validation triggers.
Is this a validation bug?

Investigating and fixing other schemas lead to discovery of more such cases:
model affiliations_documents {
  affiliation_id affiliations
  document_id    documents    @relation(references: [id])
}

model affiliations_events {
  affiliation_id affiliations
  event_id       events       @relation(references: [id])
}

model affiliations_media {
  affiliation_id affiliations
  media_id       media        @relation(references: [id])
}

[...]

https://github.com/prisma/introspection-engine-output/blob/master/postgres/sportsdb.log
These also pass validation with their relations, and then cause a panic later.

Question: Does it make sense that it does not fail in the validation step already? If not, we should add that (and by doing so give us an opportunity to educate the user about workaround (fix table, remove table from introspection) and thus avoid this panic.

See https://github.com/prisma/prisma2/issues/1515

These cases will now give a validation error asking for an Id to be added. See explanation here: https://github.com/prisma/prisma2/issues/1515


Pipelines (with non-unique names) submitted from Elyra do not show up in the corresponding DS Project / DS Pipelines pages

Description of problem:
A user can spawn a Notebook from the Jupyter tile which supports Elyra and create their runtime configuration to be able to run pipelines using one of their Data Science Projects.
However, when doing so, the Pipeline and PipelineRun that are created by Elyra will not show up in the Project, only in the Runs of the Data Science Pipelines section of the dashboard

Prerequisites (if any, like setup, operators/versions):
RHODS 1.28 RC
Steps to Reproduce
Create DS Project
Create Pipeline Server in DS Project
Move to Jupyter tile and spawn Notebook with image that includes Elyra
From the Notebook, create a runtime config
Using the endpoint of the pipelines server url for the API endpoint
Using the dashboard endpoint with /pipelineRuns/ appended for the public API endpoint
choose EXISTING_BEARER_TOKEN as Authentication Type
use your user's token for auth
For cloud object storage, fill in the details using the same s3 connection set up for the pipelines server:
AWS s3 endpoint for object storage endpoint
your bucket name for object storage bucket name,
USER_CREDENTIALS as the Authentication type,
AWS Secret Access Key ID for the username
AWS secret access key for the password
Submit the pipeline using the runtime config you've just created
Actual results:
Pipeline run is only show in the "Runs" section of the DS Pipelines section of the dashboard
Expected results:
Pipeline and PipelineRun are also shown in the DS Project where the Pipeline server was deployed
Reproducibility (Always/Intermittent/Only Once):
Always
Build Details:
Workaround:
When submitting a Pipeline, ensure to give it a unique name. If that's the case, it will be listed correctly among the Pipelines related to the DS Project
Additional info:
JIRA: https://issues.redhat.com/browse/RHODS-9374

How to replicate the issue:

Setup cluster with ODH/RHODS
Setup data-science-pipeline server, and  start a notebook.
Execute a elyra example
Run it multiple times and check the dashboard for pipeline runs.

The process to fix this;

Look for pipeline creation from elyra code.
Explore why the pipeline name is being repeated with suffixes.
Try to fix it.

Investigated the pipeline naming and pipeline versioning:
Elyra pipeline submits runs with unique pipeline_version for each execution of pipeline execution.
It seems like a natural flow to provide a pipeline version_id to create runs: https://github.com/kubeflow/kfp-tekton/blob/2b0e3adcd0f777b88b2bb705c0d06d9be8deb29b/sdk/python/kfp_tekton/_client.py#L382
With the current pattern, the odh_dashboard uses the get API Call of kfp to populate runs
as the PIPELINE_VERSION is the only option to get the list of runs, it is used.
With pipeline_id as key: https://github.com/opendatahub-io/odh-dashboard/blob/c1848e331143329c5bdd7ce70bc170c6feba42c8/frontend/src/api/pipelines/custom.ts#L79
With Elyra, we can make the change to include the pipeline version id as the pipeline id reference
for getting the pattern in order as requested in this issue.
Reference :
runs.txt
pipelines.txt


AllSky lattitude/longitude

I have entered my location in AllSky config.js as 48.2 lat and -122.6 long. The sunwait report command shows the locaton as 40.1N and 271.7E so the program switches to NIGHT mid afternoon. I can't find where the error is coming from and need some help.
Thanks.

@psenness   The Allsky Website should be showing 48.2N latitude and 122.6W longitude so I don't know where it's getting 40.1N and 271.7E.
Is your Website on the Internet?  If so, what's the URL?  If not, please attach the config.js file.
What's your "angle"?  It also impacts when daytime is.
Here's what I get for an angle of -6:
pi:~ $ sunwait list angle  -6  48.2N  122.6W
08:37, 20:10

Eric,
Thanks for your quick response. I have attached the configuration file and it shows the correct lat/long. I use -6 for the angle also. I do not have an online website for this project – yet.
I’m using a Pi 4B with 8Gb and a RPiHQ camera.
In doing more troubleshooting this morning I did a complete reinstall per the video from Patriot Astro. I got the same results. I’ve also attached the sunwait report I mentioned in my first email. It shows the correct local date, time and time zone. The times for all the twilights are off by two hours. The “Target Information… Location” is what I find confusing. Longitude 272E (which doesn’t exist) would be 92W – which is central standard time and two hours different from PST (my time zone) which would explain the off shift for day imaging to night imaging and back. And like you, I have no idea where those coordinates come from and I surely didn’t enter anything like that. The other item I’m curious about is the “General Information    (no offset)…” comment. Leads me to believe there is an “offset” setting somewhere.
If I may speculate – my understanding is that “sunwait” is a program called within AllSky and the data is passed to it. Is it possible the coding for “sunwait” has a default location that accounts for this behavior? I know that that is a longshot because if that were true then lots of other people would be experiencing the same issue.
I appreciate your help.
Kind regards,
Paul
From: EricClaeys @.>
Sent: Tuesday, February 21, 2023 8:20 AM
To: thomasjacquin/allsky @.>
Cc: psenness @.>; Mention @.>
Subject: Re: [thomasjacquin/allsky] AllSky lattitude/longitude (Issue #2257)
@psenness https://github.com/psenness  The Allsky Website should be showing 48.2N latitude and 122.6W longitude so I don't know where it's getting 40.1N and 271.7E.
Is your Website on the Internet? If so, what's the URL? If not, please attach the config.js file.
What's your "angle"? It also impacts when daytime is.
Here's what I get for an angle of -6:
pi:~ $ sunwait list angle  -6  48.2N  122.6W
08:37, 20:10
—
Reply to this email directly, view it on GitHub https://github.com/thomasjacquin/allsky/issues/2257#issuecomment-1438755831 , or unsubscribe https://github.com/notifications/unsubscribe-auth/A6ABUOEL6UOCU5D6H7E6CELWYTTJ5ANCNFSM6AAAAAAVCNYJKU .
You are receiving this because you were mentioned.  https://github.com/notifications/beacon/A6ABUOC2BOKU4JGZCVGTKW3WYTTJ5A5CNFSM6AAAAAAVCNYJKWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTSVYGV7O.gif Message ID: @.*** @.***> >
@.***:~/allsky $ sunwait report
  Current Date and Time: 21-Feb-2023 09:24 PST

Target Information ...
               Location:  40.108306N, 271.742700E
                   Date: 20-Feb-2023
               Timezone: PST

Sun directly north/south: 10:06
Twilight angle: -0.83 degrees (daylight)
Day with twilight: 04:38 to 15:34
It is: Day (or twilight)
General Information (no offset) ...
Times ...         Daylight: 04:38 to 15:34
with Civil twilight: 04:12 to 16:00
with Nautical twilight: 03:41 to 16:32
with Astronomical twilight: 03:09 to 17:03
Duration ...    Day length: 10:55 hours
with civil twilight: 11:48 hours
with nautical twilight: 12:51 hours
with astronomical twilight: 13:54 hours
@.***:~/allsky $
#!/bin/bash
NOTES:
- Variable names must start at the beginning of a line.
- There should be nothing after a variable's value, i.e., no comments on the same line.
- Values can optionally be quoted with double quotes, NOT single quotes.
- If a variable occurs multiple times, only the last value is used.
Choose between "ZWO" or "RPiHQ".
CAMERA="RPiHQ"
########## Images
Set to "true" to upload the current image to your website.
IMG_UPLOAD="true"
If IMG_UPLOAD is "true", upload images every IMG_UPLOAD_FREQUENCY frames, e.g., every 5 frames.
1 uploades every frame.
IMG_UPLOAD_FREQUENCY=1
The websites look in IMG_DIR for the current image.
When using the Allsky website from the "allsky-website" packge,
"current" is an alias for "/home/pi/allsky".
If you use the default IMG_DIR and have the Allsky website, set "imageName" in
/var/www/html/allsky/config.js to:
imageName: "/current/tmp/image.jpg",
This avoids copying the image to the website.
Only change IMG_DIR if you know what you are doing.
IMG_DIR="current/tmp"
Resize images before cropping, stretching, and saving.
Adjust IMG_WIDTH and IMG_HEIGHT according to your camera's sensor ratio.
IMG_RESIZE="false"
IMG_WIDTH=2028
IMG_HEIGHT=1520
Crop images before stretching and saving.
This is useful to remove some of the black border when using a fisheye lens.
If you crop an image you may need to change the "Text X" and/or "Text Y" settings in the WebUI.
CROP_IMAGE="false"
CROP_WIDTH=640
CROP_HEIGHT=480
CROP_OFFSET_X=0
CROP_OFFSET_Y=0
Auto stretch images saved at night.
AUTO_STRETCH="false"
AUTO_STRETCH_AMOUNT=10
AUTO_STRETCH_MID_POINT="10%"
Resize uploaded images.  Change the size to fit your sensor.
RESIZE_UPLOADS="false"
RESIZE_UPLOADS_SIZE="962x720"
Create thumbnails of images.  If you are not running the WebUI consider changing this to "false".

This only adds a few seconds per image as it's being processed.
REMOVE_BAD_IMAGES="false"
If REMOVE_BAD_IMAGES is "true", images whose mean brightness is
less than THRESHOLD_LOW or greater than THRESHOLD_HIGH percent (max: 100) will be removed.
Set to 0 to disable the brightness checks.
REMOVE_BAD_IMAGES_THRESHOLD_LOW=1
REMOVE_BAD_IMAGES_THRESHOLD_HIGH=90
Additional Capture parameters.  Run "capture_RPiHQ -h" to see the options.
e.g. activate autoexposure algorithm "mean" for RPiHQ: "-nightmean 0.3 -daymean 0.3"
CAPTURE_EXTRA_PARAMETERS=""
Set to "false" to disable daytime capture.
DAYTIME_CAPTURE="true"
Night images are always taken and saved to disk.
Setting DAYTIME_SAVE to "true" also saves daytime images to disk.
Will only save daytime images if DAYTIME_CAPTURE="true".
DAYTIME_SAVE="false"
########## Timelapse
Set to "true" to generate a timelapse video at the end of each night.
TIMELAPSE="true"
Set the resolution of the timelapse video (sizes must be EVEN numbers).
0 disables resize and uses the same resolution as the images.
TIMELAPSEWIDTH=0
TIMELAPSEHEIGHT=0
Bitrate of the timelapse video.  Higher numbers will produce higher quality but larger files.
TIMELAPSE_BITRATE="2000k"
Timelapse video Frames Per Second.
FPS=25
Encoder for timelapse video. May be changed to use a hardware encoder or different codec.
VCODEC="libx264"
Pixel format.
PIX_FMT=${PIX_FMT:-yuv420p}
FFLOG determines the amount of log information displayed while creating a timelapse video.
Set to "info" to see additional information if you are tuning the algorithm.
FFLOG="warning"
While creating a timelapse video, a sequence of links to the images is created.
Set to "true" to keep that sequence; set to "false" to have it deleted when done.
KEEP_SEQUENCE="false"
Any additional timelapse parameters.  Run "ffmpeg -?" to see the options.
TIMELAPSE_EXTRA_PARAMETERS=""
Set to "true" to upload timelapse video to your website at the end of each night.
UPLOAD_VIDEO="true"
########## Keogram
Set to "true" to generate a keogram at the end of each night.
KEOGRAM="true"
Additional Keogram parameters.  Run "keogram --help" to see the options.
KEOGRAM_EXTRA_PARAMETERS="--font-size 1.0 --font-line 1 --font-color '255 255 255'"
Set to "true" to upload the keogram to your website at the end of each night.
UPLOAD_KEOGRAM="true"
########## Startrails
Set to "true" to generate a startrails image of each night.
STARTRAILS="true"
Images with a brightness higher than this threshold will be skipped for
startrails image generation.  Values are 0.0 to 1.0.
BRIGHTNESS_THRESHOLD=0.1
Any additional startrails parameters.  Run "startrails --help" to see the options.
STARTRAILS_EXTRA_PARAMETERS=""
Set to "true" to upload the startrails image to your website at the end of each night.
UPLOAD_STARTRAILS="true"
########## Other
Size of thumbnails.
THUMBNAIL_SIZE_X=100
THUMBNAIL_SIZE_Y=75
Set this value to the number of days images plus videos you want to keep.
Set to "" to keep ALL days.
DAYS_TO_KEEP=14
Same as DAYS_TO_KEEP, but for the Allsky website, if installed.
Set to "" to keep ALL days.
WEB_DAYS_TO_KEEP=""
Set to "true" to upload data to your server at the end of each night.
This is needed if you are running the Allsky Website.
POST_END_OF_NIGHT_DATA="true"
Set to "true" if you want to enable dark frame subtraction on your night-time images.
You must first create dark frames - see the README.md file for instructions.
DARK_FRAME_SUBTRACTION="false"
If you have additional data or buttons you want displayed on the WebUI's System page,
specify one or more colon-separated files that contain that data.
You must provide scripts to update those files.
See the WEBUI_DATA_FILES configuration variable in the "Software settings" section of
https://github.com/thomasjacquin/allsky/wiki/allsky-Settings for details.
WEBUI_DATA_FILES=""
The uhubctl command can reset the USB bus if the camera isn't found and you know it's there.
Allsky.sh uses this to try and fix "missing" cameras.
Specify the path to the command and the USB bus number (on a Pi 4, bus 1 is USB 2.0 and
bus 2 is the USB 3.0 ports).  If you don't have 'uhubctl' installed set UHUBCTL_PATH="".
UHUBCTL_PATH=""
UHUBCTL_PORT=2
Path to the camera settings (exposure, gain, delay, overlay, etc) files.
Do not change unless you know what you are doing.
CAMERA_SETTINGS_DIR="/etc/raspap"
================ Do not change anything below this line.
END_OF_USER_SETTINGS="true"		# During upgrades, stop looking for variables here.
if [ "${CAMERA}" = "" ]; then
echo "${RED}ERROR: Please set CAMERA in config/config.sh${NC}"
sudo systemctl stop allsky
exit 100
fi
COMPATIBILITY CHECK
if [ -z "${DAYS_TO_KEEP}" ]; then		# Old version had AUTO_DELETE and NIGHTS_TO_KEEP
AUTO_DELETE="false"
else
AUTO_DELETE="true"
NIGHTS_TO_KEEP="${DAYS_TO_KEEP}"
fi
This is needed in case the user changed the default location the current image is saved to.
if [ "${IMG_DIR}" = "current/tmp" ]; then
CAPTURE_SAVE_DIR="${ALLSKY_TMP}"
else
CAPTURE_SAVE_DIR="${IMG_DIR}"
fi
CAMERA_SETTINGS="${CAMERA_SETTINGS_DIR}/settings_${CAMERA}.json"
Get the name of the file the websites will look for, and split into name and extension.
FULL_FILENAME=$(jq -r '.filename' "$CAMERA_SETTINGS")
EXTENSION="${FULL_FILENAME##.}"
FILENAME="${FULL_FILENAME%.}"
So scripts can conditionally ouput messages; DO NOT CHANGE NEXT LINES.
ALLSKY_DEBUG_LEVEL=$(jq -r '.debuglevel' "${CAMERA_SETTINGS}")
var config = {
comment: "See https://github.com/thomasjacquin/allsky/wiki/allsky-website-Settings for a description of these settings",
title: "AllSky",
imageName: "/current/tmp/image.jpg",
location: "H3T Observatory",
latitude_longitude_comment: "latitude and longitude must be decimal numbers with OUT the trailing N, S, E, and W.",
latitude: 48.2,
longitude: -122.6,
az: 0,
camera: "RPiHQ",
lens: "Fisheye",
computer: "Raspberry Pi 4 Model B Rev 1.4",
owner: "Paul",
overlaySize: 875,
overlayOffsetLeft: 0,
overlayOffsetTop: 0,
showOverlayAtStartup: false,
auroraForecast: false,
auroraMap: "north",
intervalSeconds: 5,
}

@psenness    Paul,  I learn something new every day.  I wasn't aware of "sunwait report".  I have no idea where it's getting the "Location" data from - mine is way off.  Allsky doesn't use "report" so ignore it.  Allsky uses:
sunwait poll angle  -6  48.2N  122.6W

Which returns either "DAY" or "NIGHT".
To see when Allsky things your daytime and nighttime start:
sunwait list angle  -6  48.2N  122.6W

Eric,
Sunwait list… returns expected times for my location. I’ll watch tonight and tomorrow and see where the switch takes place. This morning it switched from night to day at 4:38, two hours before it should have and correlated with the sunwait report listing. Maybe the reinstall fixed it. I’ll let you know.
Paul
From: EricClaeys @.>
Sent: Tuesday, February 21, 2023 1:15 PM
To: thomasjacquin/allsky @.>
Cc: psenness @.>; Mention @.>
Subject: Re: [thomasjacquin/allsky] AllSky lattitude/longitude (Issue #2257)
@psenness https://github.com/psenness  Paul, I learn something new every day. I wasn't aware of "sunwait report". I have no idea where it's getting the "Location" data from - mine is way off. Allsky doesn't use "report" so ignore it. Allsky uses:
sunwait poll angle  -6  48.2N  122.6W
Which returns either "DAY" or "NIGHT".
To see when Allsky things your daytime and nighttime start:
sunwait list angle  -6  48.2N  122.6W
—
Reply to this email directly, view it on GitHub https://github.com/thomasjacquin/allsky/issues/2257#issuecomment-1439102172 , or unsubscribe https://github.com/notifications/unsubscribe-auth/A6ABUODTZNUVA7HPPXCEYSLWYUV4DANCNFSM6AAAAAAVCNYJKU .
You are receiving this because you were mentioned.  https://github.com/notifications/beacon/A6ABUOFTCWM5FIQYPM2KO63WYUV4DA5CNFSM6AAAAAAVCNYJKWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTSVY32NY.gif Message ID: @.*** @.***> >

@psenness   Paul, is the timezone on the Pi correct?
Can you post ~/allsky/tmp/data.json ?

Eric,
Yes, I’ve confirmed the time zone on the Pi is set to America/Los Angeles.
I tried some methods to get the file off the Pi and onto my laptop, but none seem to work. If you have a suggestion that will work I’d be happy to try it and send the file requested. (I tried SSH and USB methods I found on the web.)
Paul
From: EricClaeys @.>
Sent: Tuesday, February 21, 2023 2:22 PM
To: thomasjacquin/allsky @.>
Cc: psenness @.>; Mention @.>
Subject: Re: [thomasjacquin/allsky] AllSky lattitude/longitude (Issue #2257)
@psenness https://github.com/psenness  Paul, is the timezone on the Pi correct?
Can you post ~/allsky/tmp/data.json ?
—
Reply to this email directly, view it on GitHub https://github.com/thomasjacquin/allsky/issues/2257#issuecomment-1439167041 , or unsubscribe https://github.com/notifications/unsubscribe-auth/A6ABUOA5CALERKGM5MPTC2LWYU52JANCNFSM6AAAAAAVCNYJKU .
You are receiving this because you were mentioned.  https://github.com/notifications/beacon/A6ABUOAQYI525ZJ6YTB64G3WYU52JA5CNFSM6AAAAAAVCNYJKWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTSVY7ZEC.gif Message ID: @.*** @.***> >

I mount /home/pi from my Pi onto my PC just like any other network drive.  I can then easily copy files.
To do this:
sudo apt install samba -y
sudo smbpasswd -a pi
sudo cp /etc/samba/smb.conf /etc/samba/smb.bak
# Using a text editor, put the lines below into /etc/samba/smb.conf
sudo /etc/init.d/smbd restart

I have the following in /etc/samba/smb.conf:
[global]
   workgroup = WORKGROUP
   log file = /var/log/samba/log.%m
   max log size = 1000
   logging = file
   panic action = /usr/share/samba/panic-action %d
   server role = standalone server
   obey pam restrictions = yes
   map to guest = bad user
   usershare allow guests = no
[pi_home]
   comment=Pi home directory
   path=/home/pi
   veto files = /.*/
   browseable = yes
   read only = no
   create mask = 0664
   directory mask = 0775

Closing issue per user request.  The problem is solved.


Using this with flow

I've tried to configure this to run with flow and somehow it doesn't work.
Anyone had success with this? I'm basically following the instructions for Running ES2015+ tests with ES2015+ sources (through Babel-CLI), but using flow instead. It should work?

Basic Steps:
installing
    "babel-cli": "6.26.0",
    "babel-preset-flow": "6.23.0",
    "babel-register": "6.26.0",
    "istanbul": "1.0.0-alpha.2",

Adding options
nodeExec: require.resolve('.bin/babel-node'),
and
mochaOptions: ['--require', 'babel-register', ...
and .babelrc
{
  "presets": ["flow"]
}

Somehow the coverage doesn't get picked up correctly It seems.

hello @simlu, from a ease-of-use perspective, I'd recommend against using this grunt plugin in new projects and using something more up-to-date. for example, a combination of ava and nyc (or even nyc and mocha) is as simple as nyc ava or nyc mocha

@pocesar That's what I'm doing now. Maybe also time to abandon grunt all together.... Thanks for the reply!


PreTag Blink Fast the Beep LED - Test case pre req check fix needed

The beep FRU is not available in Palmetto /org/openbmc/control/led/beep. So the test case ended failure..
So this needs to be fix as pre-req check if this FRU is available or not then proceed.
The sample psuedo code would look like as per Chris Austen input..
Get Inventory List
[arguments] ${args}
${list} = Get Inventory Fru Type List ${OPENBMC_MODEL} ${args}
: FOR ${element} IN @{list}
\ ${d} = Get From Dictionary ${SYSTEM_INFO} ${element}
\ Run Keyword If ${d['present']} == True Verify Present Properties ${args}
Verify Present Properties
[arguments] ${args}
${data} = create dictionary data=@{EMPTY}
${resp} = OpenBMC Post Request /org/openbmc/sensor/led/${args} data=${data}
should be equal as strings ${resp.status_code} ${HTTP_OK}
${json} = to json ${resp.content}
should be equal as integers ${json['data']} 0
This needs to be rework in general way to skip the test if the LED is not available...

code is ready but waiting for this new repo to open up for code drop

Pushing the code for review..
The LEDs for Beep, Identify and hearbeat are restructured how it was running earlier.
Now with this change if the LED doesnt exist it wont even test the subsequent LED test set..


Update strawberry-graphql package to 0.203.3

Codecov Report

Merging #332 (635c242) into main (3b5c21a) will not change coverage.
The diff coverage is n/a.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@           Coverage Diff           @@
##             main     #332   +/-   ##
=======================================
  Coverage   72.92%   72.92%
=======================================
  Files         178      178
  Lines        8531     8531
  Branches     1369     1369
=======================================
  Hits         6221     6221
  Misses       2128     2128
  Partials      182      182

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Follow speed limits for slotcar robots

New feature implementation
Implemented feature
This PR allows simulated robots to respect speed limits specified in path requests, requires https://github.com/open-rmf/rmf_internal_msgs/pull/28 and https://github.com/open-rmf/rmf_ros2/pull/132
Implementation description
A new std::optional<double> max_speed is added to UpdateResult in slotcar_common, if a speed limit was specified in the message (specifically if it was > 0.0) the minimum between vehicle nominal speed and speed limit will be the navigation speed of the vehicle, before it was fixed at being the maximum robot speed at all times.
Since we need to store more information than a simple pose for differential drive robots' trajectories (i.e. speed limit), a new struct DiffDriveTrajectory was added and a minor refactoring took place to use it throughout the code.

Added checks for speed limit being greater than 0 https://github.com/open-rmf/rmf_simulation/pull/56/commits/02dc918ed81380f28c2b5b2269ad5f6b8c456771.

I also feel that we should revisit functions like calculate_control_signals and calculate_joint_control_signals in the future and remove the use of default arguments in the next major versions release, as those arguments are used wholly by our downstream packages so far.

This can be done, so the idea is that all the arguments must be specified at all times instead of relying on default arguments? In that case I think some refactoring is also in order to reduce the number of input arguments to these functions, maybe by packing more motion parameters in structs

Closed via #71


Schema default value does not create an attribute automatically, if null

OrientDB Version, operating system, or hardware.

v2.2.13

Operating System

Linux

Hi guys,
I think it's an arguable issue, but logically it seems that current schema behavior fornull default values has a problem.
Consider the following:
create class Test extends V

create property Test.myString string
alter property Test.myString default null

create property Test.myDate datetime
alter property Test.myDate default null

Then run insert into Test content {}
From my perspective what should happen is:

A Test record is created
Attributes myString and myDate must be present, but set to null from the very beginning.

What actually happens: the attributes myString and myDate are not created. Is that an expected behavior?
Concrete example that causes an issue at this point:
I am using PhpOrient project. If I am referencing a record's attribute (e.g.  $rec["myDate"]) and that attribute does not exist, an error is thrown. So while from the schema point of view I am expecting to have an attribute created for me automatically, what happens instead is that I am still forced to check whether attribute exists first and only then I can use it. Other libraries and their users might be experiencing the same effect.
Please let me know what you think.

Setting the default value to NULL means to disable the default value. OrientDB doesn't support NULL as default value, sorry.

@lvca thanks for the explanation! I see now


Don't send data with 'complete' WebSocket message

According to the protocol, the GQL_COMPLETE message does not contain any payload, however async-graphql currently sends the final data with the complete message instead of with the last GQL_DATA message.

I fixed this problem in version v1.17.8, thank you! 😁


Kint_Renderer_Rich

It throws an error when trying to start the server:
[Symfony\Component\Debug\Exception\FatalThrowableError]
  Class 'Conner\Kint\Kint_Renderer_Rich' not found

It seems that you forgot to import the Kint_Renderer_Rich Class in KintServiceProvider.php

Confirmed

same problem
"Class 'Conner\Kint\Kint_Renderer_Rich' not found"

simply adding use Kint_Renderer_Rich; at the top of the KintServiceProvider makes the problem go away

Sorry everyone. Resolved.


problem entering recovery

Hello ,
I write to you because we were trying to enter recoverymode from STATE_SAFE_OFF_2  with the following code:
cout << "**robotState = " << drfl.get_robot_state() << endl; drfl.set_robot_control(ROBOT_CONTROL::CONTROL_RECOVERY_SAFE_OFF);
int count = 0;
while(drfl.get_robot_state() == ROBOT_STATE::STATE_SAFE_OFF || drfl.get_robot_state() == ROBOT_STATE::STATE_SAFE_OFF2)
{
usleep(1000000);
cout << "robotState = " << drfl.get_robot_state() << ", " << count << endl;
if (count++ > 20)
{
count = 0;
drfl.set_robot_control(ROBOT_CONTROL::CONTROL_RECOVERY_SAFE_OFF);
}
}
bool recoveryServoOn = drfl.set_robot_control(ROBOT_CONTROL::CONTROL_RESET_SAFE_OFF);
while(robotState != 8) // finchè il robot non si è riacceso (nello stato di recovery) aspetta
{
usleep(500000);
}
But it seem not to enter recovery and change robotState.

Hi,
We had the same problem in V.10. When our robot is in collision and then the protective stop signal gets active, we can't bring our cobot in protective stop. However when we bring the robot in recovery without making the protective stop signal active, this is possible. We only have this problem when using an API..


3280 fix tables tests

@juanignaciosl @Kartones can you please review?
What's in this PR:

Delete many tests that referenced an old endpoint used for tables listing: v1_tables_url
Delete unused vizzjson endpoint (I think Alejandro did the very same thing)
Recover some E2E tests that can still be useful
Some fixes for stuff I presumably broke while doing the UserTable refactor
Finally add the suite back to the Makefile

Frontend tests were OK :+1: (details)

+1

:+1:

Frontend tests were OK :+1: (details)


Update stable version in repo from 0.0.9 to 0.0.10

Think about update stable version to 0.0.10
Version: stable=0.0.9 (49 days old) => latest=0.0.10 (15 days old)
Installs: stable=431 (62.92%), latest=81 (11.82%), total=685
Click to edit

This issue seems to be outdated
This issue suggests to update the stable version of this adapter to 0.0.10 but the current stable version is already 0.0.13.
So this issue should be closed.


Reduce some log verbosity in rfc2136 provider

Please reduce log level for output axfr is disabled

Hi,
I would like to take this. This will be good opportunity to setup, get familiar, and start contributing.
Thanks

Closing as #932 was merged.


[QUESTION] How to change label of stencil palette?

Hello, I'm new to mxGraph and tried to start experimenting from the graphEditor example. I'm trying to add a custom stencil palette by copy pasting the pattern for other palette from the Sidebar.js. But it shows as undefined instead.

So I tried to find the phrase "BPMN General" from the whole file but didn't find any. Where is the stencil palette label defined and how to change it?

I think mxResources is like a dictionary of translations - does it work if you put a direct string to it like e.g. this.addStencilPalette('customLabelzz', 'customLabelzz', dir + '/custom_stencil.xml', ...? If that works I think it should also be possible to define the translation strings in an external file if you'd like to use that system.

Please have a look at the addBpmnPalette, it calls the addPaletteFunctions function to set the title.
See https://github.com/jgraph/mxgraph/blob/v4.2.2/javascript/examples/grapheditor/www/js/Sidebar.js#L2478
	this.addPaletteFunctions('bpmn', 'BPMN ' + mxResources.get('general'), false, fns);

No activity for more than a year, so closing.
If you need more details, please open a discussion.


Active job application tab while evaluating internship application

Describe the task
Whenever we evaluate an internship application, the  Job application tab becomes active, while the internship tab should be active

This issue is completed. Closing it.


Non-deterministic failures without diagnostic

Description
I get one of these errors every so often when running docc:
error: 'docc convert' invocation failed with a nonzero exit code: '6'
error: 'docc convert' invocation failed with a nonzero exit code: '11'
error: 'docc convert' invocation failed with a nonzero exit code: '64'

The command I'm using is:
        set -eo pipefail
        for TARGET in CLI Driver FrontEnd Core IR CodeGenLLVM Utils TestUtils HyloModule GenerateHyloFileTests; do
          mkdir -p _site/docc/"$TARGET"
          swift package --allow-writing-to-directory ./_site \
          generate-documentation \
          --target "$TARGET" \
          --output-path _site/docc/"$TARGET" \
          --experimental-documentation-coverage --level brief \
          --enable-inherited-docs \
          --transform-for-static-hosting \
          --hosting-base-path Hylo/docc/"$TARGET"
        done

I'm using it on this project.  An example is here.
Checklist

[X] If possible, I've reproduced the issue using the main branch of this package.
[X] This issue hasn't been addressed in an existing GitHub issue.

Expected Behavior
Successful completion always.
Actual behavior
No response
Steps To Reproduce
With this project, follow the README instructions for building, and then run the command I gave in my description.
Swift-DocC Version Information
Whatever is installed with Xcode 14.3.1 I presume
Swift Compiler Version Information
swift-driver version: 1.75.2 Apple Swift version 5.8.1 (swiftlang-<IP_ADDRESS>.5 clang-14<IP_ADDRESS>.100)
Target: arm64-apple-macosx13.0

I think this was my problem, not docc's.  Will open a new one if that turns out to be wrong.  Sorry for the bad report!

I guess I was mistaken about being mistaken :(.  Still happening.

Is it always the same target that fails? Both examples look like the same "exit code: '6'" failure for the Core target.
To avoid needing to install and configure LLVM; would it be possible to attach a zip of the symbol graph files from that target (or all of the targets)? You can see the file path of the symbol graph files if you pass --verbose to the generate-documentation command.

Is it always the same target that fails? Both examples look like the same "exit code: '6'" failure for the Core target.

I didn't notice, sorry.

would it be possible to attach a zip of the symbol graph files

Voila: extracted-symbols.zip

The crash that I could reproduce is caused by a concurrency issue with the experimental documentation coverage feature. I think that could explain why the errors only occur sometimes and why the exit code isn't always the same.


Ros2 foxy

Issue #, if available:
Description of changes:
Update to support ROS2 Foxy.  create_publisher was only one function that needed an update, as there has been a new required parameter added to it.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

ros2_foxy will be kept as a separate branch. Closing this PR


Valid tile with partial Petal 1 on night of 20241130 (tile 23523, exposure 266219)

Petal 1 for tile 23523 has very low EFFTIME on the completion night of 20241130 due to the petal being missing entirely for exposure 263681 on the night of 20241118.
I'm pretty sure these cases are OK for LSS (as all the targets on the low EFFTIME petal are marked QAFIBERSTATUS > 0) and we can pass tile 23523 as good. But it's the weekend so I'm marking the tile as unsure just in case. I'll obtain an opinion from the wider group tomorrow.

Agreed, missing one petal for known reasons -> good.  I think we change our mind at 4 petals.

Great — that was my recollection but I wanted to be careful on a holiday Sunday. I'll update the QA for this tile soon.

I updated the QA for tile 23523 to none. @LNapolitano will mark this tile as good during normal tile QA tomorrow.

Lucas marked this tile as good, so I'm closing the ticket.


Reconnecting to memcache fails on run time

de.javakaffee.web.msm.MemcachedSessionService loadFromMemcached
WARNING: Could not load session with id 95A5E60EE19533ECBF1888D8526CECE4 from memcached.
xxxxxxxxx/xx.x.xxx.xxx:11211 active: true, authed: true, last read: 7,103 ms ago }
at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1240)
at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1257)
at de.javakaffee.web.msm.MemcachedSessionService.loadFromMemcached(MemcachedSessionService.java:1092)
at de.javakaffee.web.msm.MemcachedSessionService.findSession(MemcachedSessionService.java:585)
at de.javakaffee.web.msm.MemcachedBackupSessionManager.findSession(MemcachedBackupSessionManager.java:216)
at org.apache.catalina.connector.Request.doGetSession(Request.java:2981)
at org.apache.catalina.connector.Request.getSession(Request.java:2378)
at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:897)
--
INFO net.spy.memcached.MemcachedConnection:  Reconnecting due to exception on {QA sa=xxxxxxxxx/xx.x.xxx.xxx:11211, #Rops=6, #Wops=0, #iq=0, topRop=Cmd: get Keys: 95A5E60EE19533ECBF1888D8526CECE4Exp: 0, topWop=null, toWrite=0, interested=1}
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:384)
at net.spy.memcached.MemcachedConnection.handleReads(MemcachedConnection.java:784)
at net.spy.memcached.MemcachedConnection.handleReadsAndWrites(MemcachedConnection.java:684)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:647)
at net.spy.memcached.MemcachedConnection.handleIO(MemcachedConnection.java:418)
at net.spy.memcached.MemcachedConnection.run(MemcachedConnection.java:1401)

WARN net.spy.memcached.MemcachedConnection:  Closing, and reopening {QA sa=xxxxxxxxx/xx.x.xxx.xxx:11211, #Rops=6, #Wops=0, #iq=0, topRop=Cmd: get Keys: 95A5E60EE19533ECBF1888D8526CECE4Exp: 0, topWop=null, toWrite=0, interested=1}, attempt 0.
net.spy.memcached.protocol.ascii.AsciiMemcachedNodeImpl:  Discarding partially completed op: Cmd: get Keys: 95A5E60EE19533ECBF1888D8526CECE4Exp: 0
WARN net.spy.memcached.protocol.ascii.AsciiMemcachedNodeImpl:  Discarding partially completed op: Cmd: get Keys: 95A5E60EE19533ECBF1888D8526CECE4Exp: 0

WARNING: Could not load session with id 95A5E60EE19533ECBF1888D8526CECE4 from memcached.
java.util.concurrent.CancellationException: Cancelled
at net.spy.memcached.internal.OperationFuture.get(OperationFuture.java:177)
at net.spy.memcached.internal.GetFuture.get(GetFuture.java:69)
at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1230)
at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1257)
at de.javakaffee.web.msm.MemcachedSessionService.loadFromMemcached(MemcachedSessionService.java:1092)
at de.javakaffee.web.msm.MemcachedSessionService.findSession(MemcachedSessionService.java:585)
at de.javakaffee.web.msm.MemcachedBackupSessionManager.findSession(MemcachedBackupSessionManager.java:216)
at org.apache.catalina.connector.Request.doGetSession(Request.java:2981)
at org.apache.catalina.connector.Request.getSession(Request.java:2378)

WARN net.spy.memcached.MemcachedConnection:  Could not redistribute to another node, retrying primary node for 95A5E60EE19533ECBF1888D8526CECE4.
WARN net.spy.memcached.MemcachedConnection:  Could not redistribute to another node, retrying primary node for 95A5E60EE19533ECBF1888D8526CECE4.
WARN net.spy.memcached.MemcachedConnection:  Could not redistribute to another node, retrying primary node for 95A5E60EE19533ECBF1888D8526CECE4.
de.javakaffee.web.msm.MemcachedSessionService loadFromMemcached
WARNING: Could not load session with id 95A5E60EE19533ECBF1888D8526CECE4 from memcached.
net.spy.memcached.OperationTimeoutException: Timeout waiting for value: waited 1,000 ms. Node status: Connection Status { xxxxxxxxx/xx.x.xxx.xxx:11211 active: false, authed: true, last read: 14,666 ms ago }
at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1240)
at net.spy.memcached.MemcachedClient.get(MemcachedClient.java:1257)
at de.javakaffee.web.msm.MemcachedSessionService.loadFromMemcached(MemcachedSessionService.java:1092)
I am storing tomcat session in memcache and using spymemcache library for that.
There was an issue on running server where it could not found session from memcache --> Got some exception --> tried to reconnect to memcache automatically --> when started the memcache flag - active was false.
What could be the reason for this issue ?
Below are the configurations on context file

WEB-INF/web.xml

I got the similar error, but in my case, I got the port number wrong. 11211 is the default port which memcached server is running at. So seems like you got it right. Are you able to ping your server name or ip address manually? The error is obvisous, the net.spy.memcached.MemcachedConnection is unable to make connections.

Hey twu027,
Thanks for reply.
This issue occurred on live environment and as you can see there was an exception and then it automatically tried to close and open the connection, it could not able to connect to memcache later after the connection was closed. Also if you can see it returned "active = false" when tried to reconnect again.
When I have restarted the tomcat server, it re-established the connection and started working fine again. So I am confused, is there any configuration which is not allowing to connect it again when connection is closed automatically ?

Not sure about that. So it is working in your environment now after you restart tomcat server? Do you have to restart tomcat server regularly to make this work?
There are people using this library but seems like there isn't much support from the creator anymore.
I couldn't even get it working at all. My connection issue is gone after I set the port number correctly, not my issue is some of the classes in my app doesn't implement the serializable interface so I can't even serialize the session attributes which is bizarre.

As a wild guess, maybe the ip address had changed (in case a host name was configured), so that inetaddress dns caching caused the issue?
Is this issue still relevant at all, or shall we close it?


Update to use Go modules and work with tinygo 0.22.0

Update to work with tinygo 0.22.0, which uses Go modules by default, and has some pygamer "machine" package API changes.
Changes:

Add go.mod and go.sum files, declaring module name "github.com/jypelle/xship".
Update all import paths to use module name instead of relative imports.
Update calls to machine.ADC.Configure() to work with the tinygo 0.22.0 "machine" API.
Run go fmt on all .go source files.

Thx @matthewdale  ! Didn't know that tinygo was finally managing modules


Parse Error: Invalid header value char

Hi, I am getting this error when testing a URL under CDN/WAF (imperva).
Can it be related to the following?
https://simonbarker.me/blog/2020-09-24:solve-node-header-error-parser-strictness

Duplicate of https://github.com/louislam/uptime-kuma/issues/23


Update label id for chipseeker to match others

Otherwise, getting the following error:
$ make TOOLSET=usegalaxy.org lint
find ./usegalaxy.org -name '*.yml' | grep '^\./[^/]*/' | xargs -n 1 -P 8 python scripts/fix-lockfile.py
tool: {'owner': 'rnateam', 'name': 'chipseeker'}; unlocked: {'tool_panel_section_id': 'chip_seq', 'tools': [{'owner': 'rnateam', 'name': 'chipseeker'}]}
Traceback (most recent call last):
  File "scripts/fix-lockfile.py", line 77, in <module>
    update_file(args.fn.name)
  File "scripts/fix-lockfile.py", line 58, in update_file
    raise e
KeyError: 'tool_panel_section_label'
make: *** [lint] Error 1

Unfortunately this won't work, full details in #9. We do need to fix the script for this, however.


Liu yunqing patch 1

It looks like you are attempting to push a change to the template repository, for help with deploying your own website please start a new thread in the discussions.


cleaning large caches can take an extremely long time.

if walking the filesystem takes ~1 hour, it's possible that the cache won't ever finish being cleaned.
cc: @hillsp

We don't stop cache cleaning runs after an hour, so the cache should continue being cleaned.
We do currently do bad things in this case, which https://github.com/pagespeed/mod_pagespeed/issues/1337 tracks and I have a CL for under review.


Common CSS between patterns are getting duplicated

Using V1.5.0, we are currently generating a css file per IDL Pattern (Content Block or Section).  This CSS file is generated using the IDL's pattern's required SCSS file.  In Production, these css files would be aggregated into a single file and loaded by the rendered page.
However it looks like each of these generated CSS files has lots of underlying shared styles with each other.  So in taking the current approach, that aggregated css file would be much larger than it has to be.  This may affect page performance in Production.  So the ask is for DDS to find/provide a way to separate the common css styles needed by all IDL patterns, from the pattern specific css styles.

@kexuibm Changing this to a question.  This may have implications for other adopters of patterns beyond Drupal.  We'll discuss with team.  Have you seen page performance issues already?

@kexuibm I'm also wondering if this is something happening with the artifact creation setup in webpack rather than what's in the library. Typically, webpack handles the code duplication when creating one or many artifacts, and ensures that all the code necessary has everything it needs, whether if it's a single artifact, multiple artifacts, dynamic imports, etc. We might have to review what your webpack configuration setup is and take it from there, but from the IBM.com Library side, the scss imports are all referencing the necessary dependencies discretely in each of their respective pattern/component level scss.
cc: @kennylam @annawen1 @ljcarot @photodow @RaphaelAmadeu

@kexuibm I'm also wondering if this is something happening with the artifact creation setup in webpack rather than what's in the library. Typically, webpack handles the code duplication when creating one or many artifacts, and ensures that all the code necessary has everything it needs, whether if it's a single artifact, multiple artifacts, dynamic imports, etc. We might have to review what your webpack configuration setup is and take it from there, but from the IBM.com Library side, the scss imports are all referencing the necessary dependencies discretely in each of their respective pattern/component level scss.
cc: @kennylam @annawen1 @ljcarot @photodow @RaphaelAmadeu

@jeffchew  Yes, we've been using the scss imports per pattern/subpattern from the story book readme documentation for each pattern.  I think the issue is that we are generating separate css files one for each pattern.  One idea that we've been investigating is to simply create a webpack configuration that combines all of the scss imports from all the patterns in learn page to see if that removes duplicates classes.


Change logo in horzion

The betacloud logo should be replace with the not yet created osism logo.

At the moment we simply use the OpenStack logo. In the future kolla-ansible should be able to handle a custom logo.


Add .editorconfig and docker-fast-compile.sh

Description

.editorconfig: It is used to help maintain some consistency in the  coding style
docker-compile-fast.sh: It is used to reduce the compilation time of multilingual courses. However, it is still recommended to use the docker-compile.sh script before publishing the course in A+.

Testing
The .editorconfig was tested in vim and VS Code.
The docker-fast-compile.sh was tested in the integrated terminal of VS Code and in a ubuntu terminal.
Have you updated the README or other relevant documentation?
We will need to add some description of the docker-fast-compile.sh in the course manual. Nonetheless, a description of these two new files was added in the new https://github.com/apluslms/aplus-course-template, which may be enough.
Is it Done?

[ ] I (developer) have created unit tests and the tests pass
[ ] I (developer) have created functional tests (Selenium tests) if applicable
[X] I (developer) have tested the changes manually
[ ] Reviewer has finished the code review
[ ] After the review, the developer has made changes accordingly
[ ] Customer/Teacher has accepted the implementation of the feature

I took the https://github.com/apluslms/a-plus/blob/master/.editorconfig file as an example in order to keep consistency across projects. I basically added the settings for the  rst files

P.s. I created a new upstream source to the correct place: https://apluslms.github.io/contribute/styleguides/  . It is done in a way that the common part could be updated from the upstream easily (i.e. local changes are at the end). Also I included a bit more generic info about parameters and plugins, but I think this repo could have more VSCode linking, as it's the succested editor currently. However, I would say no human reads this file normally.

I was thinking that you would copy the upstream editorconfig file and add overrides to the end if this course-templates needs to change some of the rules. Project-specific comments in the beginning can be added after the comments from the upstream.

Ups! I did not realise that @raphendyr did changes to the file, and it seems like I did not read your comments thoroughly :man_facepalming:. Therefore, there is not much to add :). I only rewrite the comments. If you think the changes should be updated in the upstream let me know, and I will create a PR for those changes. On the contrary, if you think that the comments are better in the current state I will stick to those.


⚠ PrivateHD has degraded performance

In 6d831a2, PrivateHD ($PHD) experienced degraded performance:

HTTP code: 200
Response time: 24245 ms

Resolved: PrivateHD performance has improved in 477f630.


Adds Slack channel and fixes Calendar link

Added the public relations Slack channel
Remove the Trello's Calendar view because its link doesn't work

thanks @fapires!


[Web] Is there any way to include routes in web page addresses？

I wanted to develop a web program using Avalonia, but when I switched pages and refreshed the web page, the page went back to its original state, this is clearly wrong. I find Avalonia Web don't have router in address , therefore, the web page state cannot be stored.
Is there any way to fix this？Thanks！

You can use regular javascript API for this using javascript interop:
https://learn.microsoft.com/en-us/aspnet/core/client-side/dotnet-interop?view=aspnetcore-7.0
i.e. in main.js:
const dotnetRuntime = await dotnet.create();
dotnetRuntime.setModuleImports("main.js", {
    window: {
        history: {
            replaceState: (title, href) => globalThis.window.history.replaceState(null, title, href)
        },
        location: {
            href: () => globalThis.window.location.href
        }
    }
});

in code:
public partial static class Web
{
    [JSImport("window.history.replaceState", "main.js")]
    internal static partial void ReplaceHistoryState(string title, string href);

    [JSImport("window.location.href", "main.js")]
    internal static partial string Href();
}

Now you can use Web.Href() and Web.ReplaceHistoryState() for navigation and routing.


Changed all text with localization scripts 't('.....')

In easy-email-core, imported { t } from I18nManager, and marked texts as t('xxx').
In easy-email-editor and easy-email-extension, marked texts as t('xxx').
Created "en" and "tr" locale json files.

In easy-email-core, imported { t } from I18nManager, and marked texts as t('xxx'). In easy-email-editor and easy-email-extension, marked texts as t('xxx'). Created "en" and "tr" locale json files.

Thank @turkdemir
I have checked your PR, please update .

Hi @m-Ryan , i wrote my comments, after your answers; i will send my final request.

Hi @turkdemir ,
I have already replied, if you still don't understand something, feel free to ask questions.

Done. I hope it is ok.

Done. I hope it is ok.

Merged to testing, please pull and check


Update sbt-sonatype to 3.0

Updates org.xerial.sbt:sbt-sonatype from 2.6 to 3.0.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention @scala-steward in the comments below.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.xerial.sbt", artifactId = "sbt-sonatype" } ]

Codecov Report

Merging #81 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #81   +/-   ##
=======================================
  Coverage   51.63%   51.63%
=======================================
  Files          29       29
  Lines         244      244
=======================================
  Hits          126      126
  Misses        118      118

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cba157f...39ae91e. Read the comment docs.


Sharing bugs

[x] When sharing an adventure, redirect to the app on mobile
[x] Be able to share a profile

All done


Consider removing usage of 'fs.existsSync'

exists and existsSync are deprecated (see https://github.com/nodejs/node/issues/103 and https://github.com/nodejs/node/issues/1592) and it seems that the "right" thing to do is a try/catch with a call to stat/statSync or access/accessSync.
In sys.ts, we use fs.existsSync for fileExists, directoryExists (and thus createDirectory), and readFile.

Based on the discussion in 1592 I don't think existsSync is going anywhere

I think we're going to need an fsExistsSyncExistsSync.

It's still there!


request for conversation generated from search results

if some user searches for the same need and request conversations from the search results,
the user who owns that need gets multiple conversation requests with the same title, which can be irritating for the user if he wants to distinguish the different requests.

Maybe we should think about letting the owner user 'tag' the different requests.

old ui -> close


build: Change base Docker image from Alpine to Debian

Change base Docker image from alpine 3.16 (as an too old system) into Debian - use bookworm-slim image. (see our issue too https://github.com/apache/kvrocks/issues/2344)
As we can't upgrade alpine, for continuous use latest rocksdb release and fix some mysterious (for me, see https://github.com/apache/kvrocks/pull/2327) bugs with jemalloc in alpine, I propagate an new Docker, based on latest Debian slim.
Basically, nothing to change from original, just change an image and use apt instead of apk. Added only an apt clean command for cleanup system after upgrade.
Result image has an ~ 120Mb size instead of ~40 Mb in alpine.

Thanks to @thbley about the some fix and optimizing from https://github.com/apache/kvrocks/issues/2344

Result image has an ~ 120Mb size instead of ~40 Mb in alpine. I think that its OK.

@aleksraiden Thanks for taking care of this.

Some troubles in healthcheck on docker, try to fix today or tomorrow

As we can't upgrade alpine
Hmm is there any issue to upgrade alpine?

And also, do you have any suggestion to select between debian and ubuntu?

And also, do you have any suggestion to select between debian and ubuntu?

To me both are quite good and work great in production for many companies.
Ubuntu lts versions have longer support lifetime, more recent versions of packages (e.g. gcc, openssl), Debian more stable, less recent versions. Key for alpine is size and security, but no glibc.
for package version comparison:
https://packages.ubuntu.com/
https://www.debian.org/distrib/packages
https://pkgs.alpinelinux.org/packages

Yeah, sorry, I have not any real experience with ubuntu, but over 10 y use Debian on all our servers, so use it as a matured server-side os.
In other hands, the idea of multiple images (like a community or non-official, in separate directory etc) with other base images my Ok and I can support this feature.
In my mind, ideally - alpine 3.20+ but we cant switch due to some dependency. So - Debian looks like good solution at now.

As we can't upgrade alpine

Hmm is there any issue for us about upgrading alpine?

With @torwig we have a lot of internal tests and a core point is libexecinfo dependency


sqlsrv extension modifies the locale settings and possibly breaks other software

+## PHP Driver version or file name
ExtensionVer => 5.6.1
+## SQL Server version
not relevant
+## Client operating system
Debian 10
+## PHP version
7.1.33
+## Microsoft ODBC Driver version
not relevant
+## Table schema
not relevant
+## Problem description
The PHP Extension seems to override the LC_ options. This triggered a Problem in Magento 1 Shops. All prices were printed way to high. But this is not a fault of magento. I dont think a PHP extension should modify the locale settings at all.
Testing the output of localeconv with sqlsrv enabled:
/opt/php-7.1/bin/php  -r "print_r(localeconv());"
Array
(
    [decimal_point] => ,
    [thousands_sep] => .
    [int_curr_symbol] => EUR
    [currency_symbol] => €
    [mon_decimal_point] => ,
    [mon_thousands_sep] => .
    [positive_sign] =>
    [negative_sign] => -
    [int_frac_digits] => 2
    [frac_digits] => 2
    [p_cs_precedes] => 0
    [p_sep_by_space] => 1
    [n_cs_precedes] => 0
    [n_sep_by_space] => 1
    [p_sign_posn] => 1
    [n_sign_posn] => 1
    [grouping] => Array
        (
            [0] => 3
            [1] => 3
        )

    [mon_grouping] => Array
        (
            [0] => 3
            [1] => 3
        )

)

Testing the output of localeconv with sqlsrv disabled :
/opt/php-7.1/bin/php  -r "print_r(localeconv());"
Array
(
    [decimal_point] => .
    [thousands_sep] =>
    [int_curr_symbol] =>
    [currency_symbol] =>
    [mon_decimal_point] =>
    [mon_thousands_sep] =>
    [positive_sign] =>
    [negative_sign] =>
    [int_frac_digits] => 127
    [frac_digits] => 127
    [p_cs_precedes] => 127
    [p_sep_by_space] => 127
    [n_cs_precedes] => 127
    [n_sep_by_space] => 127
    [p_sign_posn] => 127
    [n_sign_posn] => 127
    [grouping] => Array
        (
        )

    [mon_grouping] => Array
        (
        )

)

+## Expected behavior and actual behavior

The extension should not modify locale config

Hi @marvinhinz what's the output when you run locale? What are your default LC_ options?
For example, in my env, this is what I have:
root@b5112ea0f810:/tmp# locale
LANG=en_US.UTF-8
LANGUAGE=en_US:en
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=en_US.UTF-8
root@b5112ea0f810:/tmp# php --ri sqlsrv

sqlsrv

sqlsrv support => enabled
ExtensionVer => 5.6.1

Directive => Local Value => Master Value
sqlsrv.WarningsReturnAsErrors => On => On
sqlsrv.LogSeverity => 0 => 0
sqlsrv.LogSubsystems => 0 => 0
sqlsrv.ClientBufferMaxKBSize => 10240 => 10240
root@b5112ea0f810:/tmp# php  -r "print_r(localeconv());"
Array
(
    [decimal_point] => .
    [thousands_sep] => ,
    [int_curr_symbol] => USD
    [currency_symbol] => $
    [mon_decimal_point] => .
    [mon_thousands_sep] => ,
    [positive_sign] =>
    [negative_sign] => -
    [int_frac_digits] => 2
    [frac_digits] => 2
    [p_cs_precedes] => 1
    [p_sep_by_space] => 0
    [n_cs_precedes] => 1
    [n_sep_by_space] => 0
    [p_sign_posn] => 1
    [n_sign_posn] => 1
    [grouping] => Array
        (
            [0] => 3
            [1] => 3
        )

    [mon_grouping] => Array
        (
            [0] => 3
            [1] => 3
        )

)
root@b5112ea0f810:/tmp#

$ locale
LANG=de_DE.UTF-8
LANGUAGE=
LC_CTYPE="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_PAPER="de_DE.UTF-8"
LC_NAME="de_DE.UTF-8"
LC_ADDRESS="de_DE.UTF-8"
LC_TELEPHONE="de_DE.UTF-8"
LC_MEASUREMENT="de_DE.UTF-8"
LC_IDENTIFICATION="de_DE.UTF-8"
LC_ALL=

$ /opt/php-7.1/bin/php --ri sqlsrv
sqlsrv

sqlsrv support => enabled
ExtensionVer => 5.6.1

Directive => Local Value => Master Value
sqlsrv.WarningsReturnAsErrors => On => On
sqlsrv.LogSeverity => 0 => 0
sqlsrv.LogSubsystems => 0 => 0
sqlsrv.ClientBufferMaxKBSize => 10240 => 10240

HI @marvinhinz the driver currently sets the application locale to the system one, which in your case is de_DE.UTF-8. In other words, the driver essentially does this:
php -r "setlocale(LC_ALL, ''); print_r(localeconv());"

Hi @yitam
Okay this is strange, to workaround our price bug, we had to add setlocale(LC_ALL,"C"); in magento core files.

It's because the driver currently sets the application locale to the system one, which in your case is de_DE.UTF-8. In other words, the driver essentially does this:
php -r "setlocale(LC_ALL, ''); print_r(localeconv());"

@yitam but this behaviour has some very unexpected and hard to debug side effects, which breaks other code. This should not happen.
As you know, setlocale(LC_ALL, 0) can be used to retrieve the current locale settings. With sqlserv extension loaded, we get this output:
$ php -r "var_dump(setlocale(LC_ALL, 0));"
string(11) "de_DE.UTF-8"

Without the extension the output looks quite different:
$ php -r "var_dump(setlocale(LC_ALL, 0));"
string(170) "LC_CTYPE=de_DE.UTF-8;LC_NUMERIC=C;LC_TIME=C;LC_COLLATE=C;LC_MONETARY=C;LC_MESSAGES=C;LC_PAPER=C;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=C;LC_IDENTIFICATION=C"

And as @marvinhinz has already pointed out, Magento (version 1) that uses the Zend Framework's locale component (see https://framework.zend.com/manual/1.12/en/zend.locale.functions.html) struggels with the setting done by sqlsrv extension.
Can you tell me, why the extension changes the locale settings?

Hi @marvinhinz, glad to hear you found a workaround.
Hi @naitsirch, please be reminded that when calling setlocale(LC_ALL, 0), if not all categories have the same values then a string containing all the locales will be returned. That explains why you see a long string of length 170, because some have the default C locale.
To prove my point, try the following:
$ export LANG=C

$ php -r "var_dump(setlocale(LC_ALL, 0));"
string(1) "C"

Or you can try something like this (with sqlsrv disabled):
$ export LANG='en_US.UTF-8'

$ php -r "var_dump(setlocale(LC_ALL, 0));"
string(170) "LC_CTYPE=en_US.UTF-8;LC_NUMERIC=C;LC_TIME=C;LC_COLLATE=C;LC_MONETARY=C;LC_MESSAGES=C;LC_PAPER=C;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=C;LC_IDENTIFICATION=C"

$ php -r "setlocale(LC_ALL, 'en_US.UTF-8'); var_dump(setlocale(LC_ALL, 0));"
string(11) "en_US.UTF-8"

$ php -r "setlocale(LC_ALL, 'en_US.UTF-8'); var_dump(setlocale(LC_MESSAGES, 0));"
string(11) "en_US.UTF-8"

$ php -r "setlocale(LC_ALL, 'en_US.UTF-8'); var_dump(setlocale(LC_NUMERIC, 0));"
string(11) "en_US.UTF-8"

We had users in the past who wanted sqlsrv or pdo_sqlsrv drivers to work with their non-UTF-8 locale(s) in their Linux env. Hence, the values of their locale environment variables are used.
That being said, we may consider providing a configurable option for users. I can mark this issue as a feature request if you like.

Thanks for the explanation. An option to define this behavior would be the best solution I think. The question is, what is the default behavior?
IMHO the activation of an extension should not change the behavior of other parts of the language or library per default. If the user definitely want that and defines that via config file or code, it's okay.
Another question: Can you tell me where I can find the code in the sqlsrv extension that changes the locale setting? I tried to find it but wasn't successful.

hi @naitsirch , you can find it here
We will discuss how / whether to make this configurable, but in order not to introduce a breaking change the default might likely be the way it is now.

@yitam Thanks for pointing me to the right place :-)
I do understand your BC concerns. Maybe an idea to achieve my proposed idea with developer and administrator friendly upgrade path:

Introduce configuration in next 5.x release and and trigger a deprecation message if the configuration is not set. But keep the default as it is now. This will make sure that everyone set the config the way they need it.
Then in 6.0 the default behavior can be changed to not changing locale settings without explicit config setting.

That's the way BC breaks are handled in projects like Symfony or PHP core.

Thanks @naitsirch. We will keep your suggestions in mind when we decide our next course of action.

Hi @marvinhinz @naitsirch
We just released 5.8.0 and please check the latest documentation concerning this issue

Looks good! Thank you

Great! Thanks for the confirmation @naitsirch. Closing this issue now. Please feel free to reopen if necessary.

In my opinion, this issue is not solved.
Consider the following scenario.
I have my developer machine set to my national locale (diacritics, etc.), PHP application sets it's own locale settings independently on host machine. After installing the extension, application is broken due to forced locale settings, and the only way how to fix it is to reconfigure the system (LC_ variables are used by other applications too).
I suggest adding a directive to set specific locale or remove this magic setting inside completely.

@grossmannmartin I don't quite understand the issue. Please elaborate.
Did you add the driver setting in the php.ini as explained in the documentation?

@yitam Sorry, I try to describe the issue better.

My notebook is set to Czech locale. I do not have set LC_* variables.

Bootstrap of my application contains following

setlocale(LC_CTYPE, 'en_US.utf8');
setlocale(LC_NUMERIC, 'en_US.utf8');

var_dump(localeconv());

output is
array:18 [
  "decimal_point" => "."
  "thousands_sep" => ""
  "int_curr_symbol" => ""
  "currency_symbol" => ""
...

I enable sqlsrv extension

output of the code above is

array:18 [
  "decimal_point" => ","
  "thousands_sep" => " "
  "int_curr_symbol" => "CZK "
  "currency_symbol" => "Kč"

I set sqlsrv.SetLocaleInfo under [sqlsrv] section to any value (0, 1, 2)

no effect

I have to set environment variable LC_CTYPE to en_US.utf8

now the output is as it should be

But any other software depending on this variable break Czech language (git for example)
I have to alter my system settings to make the extension work properly even when I set the locale explicitly in the application.

@grossmannmartin did you happen to have installed pdo_sqlsrv as well? If so, you will need to set locale info for pdo_sqlsrv to 0. If not, please create a new issue and reference this one and we will help you from there.

@yitam Thank you :) it was really a problem related to pdo_sqlsrv.
After setting pdo_sqlsrv.set_locale_info = 0 in my php.ini file everything works like a charm.
Sorry for the fuss. You rock 👍


FreeBSD: support networking (with runj.ext.json)

runj now supports networking via runj.ext.json.
https://github.com/samuelkarp/runj/blob/ec642aa7887be2a4968f20d24b54eed687895a65/docs/oci.md
{
  "network": {
    "ipv4": {
      "mode": "inherit"
    }
  }
}

Probably, nerdctl run --net=bridge can be mapped to "mode": "new", and --net=host can be mapped to "mode": "inherit".

For "mode":"new" there's some additional work to be done:

runj will need to gain support for specifying a vnet should be created
runj will need to gain support for passing an interface to the vnet
nerdctl (or a CNI plugin?) would need to create an epair and bridge and configure them appropriately

runj now has support for vnet.  I've written a bit about what steps can be used for enabling networking in runj here: https://samuel.karp.dev/blog/2022/12/docker-style-networking-for-freebsd-jails-with-runj/.


instuctions for docker-compose not working

I have tried to run the demo following the docker-compose instructions and the registry log reports this error while I interact with the registry web UI:
registry1    | 2020-07-01 16:48:33,697 ERROR [io.api.reg.res.RegistryExceptionMapper] (executor-thread-1) StreamsMetadata is currently unavailable. This can occur during rebalance operations. Store-name: storage-store: org.apache.kafka.streams.errors.InvalidStateStoreException: StreamsMetadata is currently unavailable. This can occur during rebalance operations. Store-name: storage-store
on top of that, it seems that no event is generated to feed the demo, is one step missing?

Can you try this with Registry’s upstream master?
(temp building your own Docker image(s))
On Thu, 2 Jul 2020 at 15:13, Andrea Tarocchi<EMAIL_ADDRESS>wrote:

I have tried to run the demo following the docker-compose instructions and
the registry log reports this error while I interact with the registry web
UI:
registry1 | 2020-07-01 16:48:33,697 ERROR
[io.api.reg.res.RegistryExceptionMapper] (executor-thread-1)
StreamsMetadata is currently unavailable. This can occur during rebalance
operations. Store-name: storage-store:
org.apache.kafka.streams.errors.InvalidStateStoreException: StreamsMetadata
is currently unavailable. This can occur during rebalance operations.
Store-name: storage-store
on top of that, it seems that no event is generated to feed the demo, is
one step missing?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Apicurio/apicurio-registry-demo/issues/8, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AACRA6FC3ZQYHNDYDFB5AE3RZSBWVANCNFSM4OO4B2QQ
.


i can't see any pokemon

i don't understand which is the correct form that i should write in the "server" space of the app...
I tried:

http:// <IP_ADDRESS>
https:// <IP_ADDRESS>
http:// <IP_ADDRESS>:5000

but no one works; what am I doing wrong?

When you enter "http://<IP_ADDRESS>:5000" on safari do you see a map ?

no, safari can't open any page

I am a noob to this
How can i lunch server with "-H <IP_ADDRESS>" ?

Have you set up the server downloaded from https://github.com/AHAAAAAAA/PokemonGo-Map?

Sure, it run in background at this moment

I assume you're able to view the map on your PC then. Is your iPhone connected to the same network as your PC? You could also try setting up a forwarding address through a service like ngrok.com or forwardhq.com, that way you don't have to be connected to the same local network.

i tried also ngrok.com but when i put the "web interface" parameter in the app it doesn't work too

You can see that it's forwarding to localhost on port 80, while your server runs on localhost port 5000. Can you check if you can change this?

How can i change this?

You're close, it seems you started ngrok with a command similar to ngrok http 80, you should start it with ngrok http 5000. After that you can see the random http://subdomain.ngrok.io domain, just below the line you circled. That's the address you need to enter in the app.

You are GREAT!! now it works! Thank you so much for your patience :D

Enjoy!

Sure! thank you again


Pyinstaller Windows exe is flagged as false positive by different anti-viruses

Reported by testers.

Avast

Edge browser shows "Download failed Virus detected"

Apparently it's related to https://github.com/pyinstaller/pyinstaller/issues/5932 and other Pyinstaller's antivirus-false-positives issues.
A possible solution is to use cx_Freeze >= 6.7 as suggested here.

Switched to cx_Freeze in cca4c97e352c691d172b3ba9895ded1309a67162


update module to be compatible with stdlib 9.x

some function like "has_key" have been removed in stdlib 9. Need to clean it up.

my bad, version 4.14.0 already replaced "has_key" function. I guess we only need to update metadata?

already done !

already done !


./gitea dump doesn't allows to specify backup destination

I like gitea dump because it allows to backup everything into a single file, but I find it difficult to automate (weekly cron job) because it's doesn't allows to specify the exact destination of the backup (to copy it elsewhere for example).
That leaves us with the (not pretty) option to parse the output to get the zip name.. something like gitea dump --destination /path/to/zip would be great.
Sorry not a go developer, if not I will simply send a PR for this (yeps it's a lame excuse, I know =)

./gitea dump 2>&1 >/dev/null | awk 'END {print $NF}'
(just in case somebody wonders how to get it)

I think WIP PR #1637 will implement this?

@lafriks Not exactly:
   --config FILE, -c FILE   Custom configuration FILE path (default: "custom/conf/app.ini")
   --tempdir PATH, -t PATH  Temporary directory PATH (default: "/tmp")
   --target PATH            Target directory PATH to save backup archive (default: "./")
   --verbose, -v            Show process details
   --db                     Backup the database (default: true)
   --repos                  Backup repositories (default: true)
   --data                   Backup attachments and avatars (default: true)
   --custom                 Backup custom files (default: true)```

That will allow to specify where to land the zip, but still no luck to set the filename.

Should be easy enough to change --target to take a filename instead of a directory :)

Was this functionality removed?
OPTIONS:    --config value, -c value    Custom configuration file path (default: "custom/conf/app.ini")    --verbose, -v               Show process details    --tempdir value, -t value   Temporary dir path (default: "/tmp")    --database value, -d value  Specify the database SQL syntax

@Quinity to be honest IDK, ended using awk to extract the dump filename but a --target or something similar will be much more appreciated.

I recently tried tackling this using cron on my debian machine, but I had no luck. However, I found great sucess using a systemd service/timer and setting the WorkingDirectory to wherever you like.
The service file:
[Unit]
Description=Gitea Dump

[Service]
Type=simple
User=git
Group=git
WorkingDirectory=/var/gitea/backup
ExecStart=/usr/local/bin/gitea dump -c /etc/gitea/app.ini
Environment=USER=git HOME=/home/git GITEA_WORK_DIR=/var/lib/gitea

[Install]
WantedBy=multi-user.target

Timer:
[Unit]
Description=Daily Gitea Dump

[Timer]
# See systemd.time(7) for time format
OnCalendar=*-*-* 05:00:00
Persistent=true

[Install]
WantedBy=timers.target

And the output of /var/gitea/backup:
$ ls /var/gitea/backup
gitea-dump-1529562686.zip

--target is not documented in https://docs.gitea.io/en-us/backup-and-restore/

So this is a documentation problem not a feature.

I don't know. Does --target allow you to specify the filename now? That was the original feature request as I understand it. At least that's what I would want for my use-case.

@lunny I was just mentioning the missing documentation, in addition to the feature request discussed here - I think it's a good opportunity to fix the documentation. I don't think --target accepts a destination filename (only a directory).
I would appreciate this change, because I run dumps from a cron job and the destination directory ends up cluttered with daily dumps.
But maybe some people prefer the old behavior - in this case there could be 2 separate options (--destdir --destfile), or we could just resort to --destfile /var/backups/gitea/gitea-dump-$(date +%s)

--target is not documented in https://docs.gitea.io/en-us/backup-and-restore/

This is not yet documented

--work-dir is also not documented (https://github.com/go-gitea/gitea/issues/9100)

it's implemented option: --file/-f

and if you type gitea dump --help you get the whole documentation of this subcomand

@6543 yes but the documentation is wrong https://docs.gitea.io/en-us/command-line/#dump
Just copy pasting the gitea dump --help output there would be better than having outdated documentation. Do you want me to send a pull request?

@nodiscc yes would be great!!!
files are in https://github.com/go-gitea/gitea/tree/master/docs


Malformed YAML Forces Safe Mode

Tried installing this theme and when restarting HA it booted into safe mode with the following error:
Found duplicate anchor 'card-mod-root'; first occurrence in "/config/themes/lcars/lcars.yaml", line 357, column 18 second occurrence in "/config/themes/lcars/lcars.yaml", line 577, column 18
I had to delete the lcars folder in /config/themes/ in order to fully boot HA

Thanks for letting me know. Sorry for the inconvenience. I just released the latest updates which do not have the duplicate card-mod-* entries. Please update and let me know if you have further issues.

New error:
Mapping values are not allowed here in "/config/themes/lcars/lcars.yaml", line 28, column 66

That's a weird one. There is no column 66 on line 28 in the yaml. I just pushed one more update just in case.

That's a weird one. There is no column 66 on line 28 in the yaml. I just pushed one more update just in case.

No change.  Same error.

What file is that? That's not lcars.yaml.

What file is that? That's not lcars.yaml.

Maybe I'm downloading it wrong from Github.  Stand by...

You should be able to install it from HACS by adding the repo in HACS.

Ok, I was right-clicking on the YAML and "Save Link As..." instead of downloading the source code.  My bad.  That seems to have worked, no errors when calling frontend.reload_themes
Thank you.

Yay!