Operator is stuck in a "deploying" phase loop when internal deployment fails indefinitely

What happened?
When deploying an Integration using the Camel K Operator that fails a deployment due a either a missing dependency or perhaps an underlying API incompatibility, the Integration will be stuck in the Deploying phase.
The issue here being that the Integration "phase" never goes into an Error state even if the cause of the deployment failure is not self-healable (i.e. API incompatibility as an example).
I have two examples of when/where this occurs:

Deploying an Integration to Camel K Operator with startupProbe enable on the Health Trait (Ref) when using Knative serving v1 API which DOES NOT support startupProbes (Ref)

│   Type     Reason                       Age               From                            Message                                                │
│   ----     ------                       ----              ----                            -------                                                │
│   Warning  IntegrationError             1s (x11 over 6s)  camel-k-integration-controller  Cannot reconcile Integration template-connector-1: err │
│ or executing post actions: error during apply resource: gc-generic/template-connector-1: failed to create typed patch object (gc-generic/templat │
│ e-connector-1; serving.knative.dev/v1, Kind=Service): .spec.template.spec.containers[0].startupProbe: field not declared in schema

Deploying an Integration to Camel K Operator with a missing dependency (e.g. ConfigMap)

│   Type     Reason            Age                     From                            Message                                                     │
│   ----     ------            ----                    ----                            -------                                                     │
│   Warning  IntegrationError  2m46s (x288 over 3d3h)  camel-k-integration-controller  Cannot reconcile Integration template-connector-1: error du │
│ ring trait customization: master trait configuration failed: ConfigMap "template-connector-1-openapi-000" not found

In both cases The Camel K Operator reports the above issues as a warning type of event when describing the deployment. Not sure if this should rather be classified as an error instead, or perhaps at least move to error after X number of retries/backoff attempts?
I believe its crucial that an Integration should "eventually" end up an error state if the issue cannot be self-healed as per the above example use-cases.
Steps to reproduce

Setup a K8s cluster with the following

Camel K Operator v2.1.0
Knative service v1.12.0

Deploy a Camel K Integration with the Health Trait enabled (health.enabled=true), and the startupProbe enabled (health.startup-probe-enabled=true)

Relevant log output
│   Last Init Timestamp:  2024-01-18T12:48:11Z                                                                                │
│   Observed Generation:  1                                                                                │
│   Phase:                Deploying                                                                                │
│   Platform:             camel-k                                                                                │
│   Profile:              Knative                                                                                │
│   Runtime Provider:     quarkus                                                                                │
│   Runtime Version:      3.2.0                                                                                │
│   Version:              2.1.0                                                                                │
│ Events:                                                                                │
│   Type     Reason                       Age               From                            Message                                                │
│   ----     ------                       ----              ----                            -------                                                │
│   Normal   IntegrationConditionChanged  9s                camel-k-integration-controller  Condition "IntegrationPlatformAvailable" is "True" for │
│  Integration template-connector-1: camel-k/camel-k                                                                                │
│   Normal   IntegrationPhaseUpdated      9s                camel-k-integration-controller  Integration "template-connector-1" in phase "Initializ │
│ ation"                                                                                │
│   Normal   IntegrationPhaseUpdated      6s (x2 over 9s)   camel-k-integration-controller  Integration "template-connector-1" in phase "Building  │
│ Kit"                                                                                │
│   Normal   IntegrationConditionChanged  6s                camel-k-integration-controller  Condition "IntegrationKitAvailable" is "True" for Inte │
│ gration template-connector-1: kit-cmkhrk8ve68c73e60i30                                                                                │
│   Normal   IntegrationPhaseUpdated      6s                camel-k-integration-controller  Integration "template-connector-1" in phase "Deploying │
│ "                                                                                │
│   Warning  IntegrationError             1s (x11 over 6s)  camel-k-integration-controller  Cannot reconcile Integration template-connector-1: err │
│ or executing post actions: error during apply resource: gc-generic/template-connector-1: failed to create typed patch object (gc-generic/templat │
│ e-connector-1; serving.knative.dev/v1, Kind=Service): .spec.template.spec.containers[0].startupProbe: field not declared in schema

│   conditions:                                                                                │
│   - firstTruthyTime: "2024-01-18T12:48:11Z"                                                                                │
│     lastTransitionTime: "2024-01-18T12:48:11Z"                                                                                │
│     lastUpdateTime: "2024-01-18T12:48:11Z"                                                                                │
│     message: camel-k/camel-k                                                                                │
│     reason: IntegrationPlatformAvailable                                                                                │
│     status: "True"                                                                                │
│     type: IntegrationPlatformAvailable                                                                                │
│   - firstTruthyTime: "2024-01-18T12:48:14Z"                                                                                │
│     lastTransitionTime: "2024-01-18T12:48:14Z"                                                                                │
│     lastUpdateTime: "2024-01-18T12:48:14Z"                                                                                │
│     message: kit-cmkhrk8ve68c73e60i30                                                                                │
│     reason: IntegrationKitAvailable                                                                                │
│     status: "True"                                                                                │
│     type: IntegrationKitAvailable

Camel K version
2.1.0

Thanks for reporting, I can confirm this behavior. We will have a look at this.

As a workaround, can you use the integration with health disabled ?

As a workaround, can you use the integration with health disabled ?

Hey @claudio4j,
Thanks so much for validating the issue.
We can just disable the startupProbe (traits.health.startupProbeEnabled: false)...
However, that is not really the main issue in my opinion.
Its more so that the Integration is stuck in a "deploying" loop indefinitely, and never ends up in an error state.
Just to confirm, is the above described "deploying" behaviour on such deployment failure scenarios intended?

is the above described "deploying" behaviour on such deployment failure scenarios intended?

Not on purpose. As you noted above, the camel-k-operator tries to set a field startupProbe on a knative service, which is not supported and the error is not handled, leaving the Integration object with the wrong status.
In the knative-service trait we have to handle this scenario (health enabled) and not set that field. We have to document this case and have testing in place.

I can take a look at this.

@realMartinez are you actively working on this?

@realMartinez are you actively working on this?

I have been working on another issue, I have put thison hold for now

Okey. Removed the assignment as I may have a look at this.


[COMMUNITY] Update maturity model

Why are the changes needed?

This PR proposes to update the Kyuubi project maturity model to reflect the current project status.
How was this patch tested?

[ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[ ] Add screenshots for manual tests if appropriate

[ ] Run test locally before make a pull request

Codecov Report

Merging #3743 (5b61c62) into master (f21ad83) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master    #3743   +/-   ##
=========================================
  Coverage     52.57%   52.57%
  Complexity       13       13
=========================================
  Files           492      492
  Lines         27705    27705
  Branches       3830     3830
=========================================
  Hits          14565    14565
- Misses        11753    11755    +2
+ Partials       1387     1385    -2

Impacted Files
Coverage Δ

...n/scala/org/apache/kyuubi/engine/ProcBuilder.scala
80.74% <0.00%> (-0.63%)
:arrow_down:

...he/kyuubi/ha/client/etcd/EtcdDiscoveryClient.scala
67.64% <0.00%> (-0.59%)
:arrow_down:

...g/apache/kyuubi/operation/BatchJobSubmission.scala
74.64% <0.00%> (ø)

...che/kyuubi/server/KyuubiTHttpFrontendService.scala
60.54% <0.00%> (+0.68%)
:arrow_up:

...rg/apache/kyuubi/ctl/cmd/log/LogBatchCommand.scala
65.57% <0.00%> (+1.63%)
:arrow_up:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

I second to comments from our mentor, can you elaborate a bit more what&why can we update the model, especially add some solid evidence, not only says that update .... reflect the current project status. which is clear to everyone

@yaooqinn @WillemJiang I supplied examples to explain why I think the project reached such a standard.

Thanks all for your review and approval, the success of Kyuubi is inseparable from all of you! Merging to master.


feat: allow collapsing of chat card descriptions

I goofed up the format. Gonna recreate on a clean branch with main pulled first.


WIP Audio

This is currently broken. With the Raylib.log statements left in, it exits with a 'file does not exist'-like error message; with  commented out, we get a panic with:
Roc crashed with: voided tag constructor is unreachable

We make it to the first loadSound after the RocStr-by-reference changes. From messing with gdb and roc src a little, the unreachable tag equivalence is Ok & Err.

If I replace init and render's error types with _, then the crash moves to unreachable Sound.load, but I can see from the rust-side logs that we 'should' be returning Ok 0.


DatePicker onChange not being triggered on change

Description
I am trying to implement the DatePicker, but it seems that the Date picked is not triggering the value change.
If I add a state to the "value" prop and then a function to the "onChange" prop that will only set the state according to the value, and then create a useEffect that will console that state each time it changes, I expect it to console a value when I select one using the Calendar.
Instead nothing happens.
I tried the same simple implementation on a pure html input, and a Select component from the lib, and both acts as expected.
The Datepicker component is acting as it never triggers the value change.
In another application of mine, more complex than example repo, I am using React Hook Form to manage a form that should receive a value from the Datepicker. Every time I submit the form, the value from Datepicker is undefined, even though I chose one on Calendar.
Link to Reproduction
https://github.com/gtonussi/ark-test/blob/main/src/app/page.tsx
Steps to reproduce

Clone repo
Install all packages (npm install)
Run the repo (npm run dev)
Pick any date and search the console for the value
No value will be consoled

Ark UI Version
0.13.1
Framework

[X] React
[ ] Solid
[ ] Vue

Browser
Google Chrome 93
Additional Information
No response

Thanks for taking your time to report this issue. A fix will be in 0.15.0


docs: clean up individual changelogs

TIL that the root changelog hasn't been updated since we switched to a monorepo, so I updated it to point to the individual changelogs. I also cleaned those up, as there were a few extraneous lines from the 1.0 release.

:tada: This PR is included in version 1.0.3 :tada:
The release is available on npm package (@latest dist-tag)
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.0.1 :tada:
The release is available on npm package (@latest dist-tag)
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.0.1 :tada:
The release is available on npm package (@latest dist-tag)
Your semantic-release bot :package::rocket:


Can't add multiple Command triggered Actions to a stage.

Note: I don't know if this is a glitch or a feature.
What is your Quests version / build number (do not say "latest")?
3.7.6-b5
Describe the problem and how to replicate it. Any console errors to post using Pastebin.com?
Console Errors: None.
How to replicate?:
Firstly, I made a new Quest, I made a new stage and added a goal (Talk to "NPC"), then I went to the actions menu of the stage and added 4 "Command Action" Actions. When I save the Quest and start editing it again, every "Command Action" Actions I added except for the last one dissapear.
If applicable, can you provide an example quest from quests.yml using Pastebin.com?
No example, but I can tell you that the "Command Action" Actions I add do not save into the Quest.yml file and when I try to add the multiple Actions into the file manually they dissapear (Except for of course the last one added) after starting the server.

Hi legit, Please try this newer version, with this version I could insert several commands in a stage which are also accepted https://ci.codemc.io/job/PikaMug/job/Quests/38/

Hi legit, Please try this newer version, with this version I could insert several commands in a stage which are also accepted https://ci.codemc.io/job/PikaMug/job/Quests/38/

I'll try that thanks!

It may happen that you have to delete the complete folder of quests because some changes have been made in the files


DocsArchive-<IP_ADDRESS>Auto archive content

Please don't merge this PR until target archive repo PR https://github.com/OPS-E2E-PPE/docs-archive-test-target/pull/834 is merged into live branch.
Auto archive content to https://github.com/opstest2/docs-archive-test-target.git

Docs Build status updates of commit d7a9941:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error-GitBranchDeletedOrForcePushed] Cannot sync git repo to specified commit because branch Release_Archive_master_2021-05-24-02-44-12 has been deleted or has been force pushed

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Not found: ValidatedGraphConfig Initialization failed.When I use handtrackinggpu.aar for Android，print：No registered calculator found

I build hand tracking aar or handTrackingGpu.aar，use it in Android error,print:
com.google.mediapipe.framework.MediaPipeException: not found: ValidatedGraphConfig Initialization failed.
No registered object with name: MultiHandDetectionSubgraph; Unable to find Calculator "MultiHandDetectionSubgraph"
No registered object with name: MultiHandLandmarkSubgraph; Unable to find Calculator "MultiHandLandmarkSubgraph"
No registered object with name: MultiHandRendererSubgraph; Unable to find Calculator "MultiHandRendererSubgraph"

i have set
load("//mediapipe/java/com/google/mediapipe:mediapipe_aar.bzl", "mediapipe_aar")
mediapipe_aar(
name = "my_handtrackinggpu_aar",
calculators = [
"//mediapipe/graphs/hand_tracking:mobile_calculators",
"//mediapipe/calculators/core:pass_through_calculator",
"//mediapipe/calculators/core:constant_side_packet_calculator",
"//mediapipe/calculators/core:flow_limiter_calculator",
"//mediapipe/graphs/hand_tracking/subgraphs:hand_renderer_gpu",
"//mediapipe/modules/hand_landmark:hand_landmark_tracking_gpu",
],
)

Which mediapipe graph is used with the aar?

Hi @chensisi0730 ,
There is a separate target for "multi-hand" subgraphs, ""multi_hand_landmark_gpu".  This needs to be included in the dependencies for the AAR target.

Hi @chensisi0730 , There is a separate target for "multi-hand" subgraphs, ""multi_hand_landmark_gpu". This needs to be included in the dependencies for the AAR target.
could you tell me  configeration of build file


infer quant --ref and --index

As reference fasta should be identifiable from the input metadata table (by species name), we should support quant --ref "infer",  which automatically detects the reference fasta in the default reference directory (I think we don't have such dir yet, so let's say amalgkit_out/reference/). File names should be like Arabidopsis_thaliana_ANYANNOTATIONHERE.fasta.gz. quant --index "infer" should also be supported similarly.

Yeah, I thought about something like this. This would also tie in toa amalgkit sanity functionality we talked about in another thread.

Will you work on this?

Sure! I actually already have code for something very close to this.

https://github.com/kfuku52/amalgkit/issues/33
and
https://github.com/kfuku52/amalgkit/issues/72


form storage + hidden input make your hidden value disappear

Hi,
context: I use form-storage and I populate my form using formStoreData function
If I check what F7 store, I have my data, it's ok. But my I change a value in a text input, my hidden inputs fall back to "".
Is it normal ?

OK I found that hidden inputs fall back to "" when there are not String

Ok, i guess we can close it. Reopen if it is still actual and how to replicate it


Terms, Types and Expressions

In (Py)Iceberg we have a hierarchy that works very well. In this issue, I'll try to explain it, and also convince y'all to use it in iceberg-rust as well. Disclaimer, I'm an OOP guy, so probably there are some things that don't make sense in Rust. I don't think we even can port the whole hierarchy, since Rust is not OOP.
The important traits are (directly translated from Python):
trait Bound {
    fn invert(&self) -> Bound;
}

trait Unbound {
    fn bind(&self, schema &impl Schema, case_sensitive &bool) -> Unbound;
}

(This excludes Term, Reference, BooleanExpression, maybe we should call Unbound as UnboundBooleanExpression, and Bound as BoundBooleanExpression, it is up to you. In the end, naming things is the hardest thing in computer science).
This is implemented by operations such as,

Unary predicates: IsNull, NotNull, IsNaN, NotNaN
Set predicates: In, NotIn
Literal predicates: EqualTo, NotEqualTo, LessThan, LessThanOrEqual, GreaterThan, GreaterThanOrEqual
Negation: Not
Compositions: And, Or
Literal: AlwaysTrue, AlwaysFalse

The inverse method is important later on to rewrite Not(...) operations. Not(EqualTo("UserId", "123")), can be rewritten to NotEqualTo("UserID", 123). Similar for Not can be rewritten: !(A and B) == !A or !B.
All the operations come in a bound and unbound one. The EqualTo is in the public API, and once it is bound to a schema, it will a BoundEqualTo. Binding is important since let's say that we have an expression: UserID = '123', then we want to convert this at bind time to UserID = 123 because the UserID is a date field in this case. Since Iceberg is lazy, every file written can have a different schema, and binding to each of the schemas has some advantages:

If you promote a column along the way from i32 to i64, the will UserID will also be promoted to an i64 when binding to a schema with a newer file.
If you add a new column, but this column isn't written in an older file, then it will be converted to an AlwaysFalse().
When an IsNull("UserID") is bound to a UserID INTEGER NOT NULL column, then this will also convert into an AlwaysFalse().
Optional: In PyIceberg we have optimizations, that In("UserID", {123}) is rewritten to EqualTo("UserID" == 123) since there is only one literal.

Hi, @Fokko Thanks for this great writing. If I understand correctly, this is quite similar to expression evaluation/optimization in a database query engine.  Usually, a database engine has its own expression optimization system, and table format needs to provide statistics to query optimizer. Since currently we haven't decided yet how to integrate with other query engines, I would suggest implementing this later after we have finished core specs such as catalog, table, transaction, metadata, etc.

@liurenjie1024 Yes, provides the input for the evaluation/optimization. I agree that we should not implement an optimizer in iceberg-rust, and I think we should also get the primitives in place. Many databases don't support id-based field resolution, and Iceberg has a lot of metadata at a file level. What I would suggest is that iceberg-rust does do the pruning of the unrelated files, rather than doing this in the database, but let's defer this discussion.

@Fokko Thanks for the explanation. I agree that providing pruning in iceberg-rust would benefit the community and other query engines.

Implementing expressions feels like a huge task which also has to be maintained later on. I'm not sure about Databend or RisingWave but for Datafusion it would be sufficient to have functionality like:
// returns min value of every manifest file for the given column
fn manifest_min_values(table: &Table, column_name: &str) -> Vec<Value>
fn manifest_max_values(table: &Table, column_name: &str) -> Vec<Value>

// returns min value of every data file for the given column
fn datafile_min_values(table: &Table, column_name: &str) -> Vec<Value>
fn datafile_max_values(table: &Table, column_name: &str) -> Vec<Value>

or even better with arrow:
fn manifest_min_values(table: &Table, column_name: &str) -> ArrayRef
fn manifest_max_values(table: &Table, column_name: &str) -> ArrayRef

fn datafile_min_values(table: &Table, column_name: &str) -> ArrayRef
fn datafile_max_values(table: &Table, column_name: &str) -> ArrayRef

Yes, expression system is not small effort, and we can postpone discussion about it later.

Implementing expressions feels like a huge task which also has to be maintained later on. I'm not sure about Databend or RisingWave but for Datafusion it would be sufficient to have functionality like:

fn manifest_min_values(table: &Table, column_name: &str) -> Vec<Value>
I'm curious abort what use case need to use this? 🤔

Implementing expressions feels like a huge task which also has to be maintained later on. I'm not sure about Databend or RisingWave but for Datafusion it would be sufficient to have functionality like:

fn manifest_min_values(table: &Table, column_name: &str) -> Vec<Value>
I'm curious abort what use case need to use this? 🤔

I think it's used in planning

Datafusion has the trait PruningStatistics that you can implement for a file/container. It can be used with a pruning predicate to check whether an expression could evaluate to true for at least one row in the file/container. This way you don't have to evaluate the expression evaluation yourself.

As DataFusion already implemented mature expression system and evaluation framework, I'm wondering if it is possibly or it is better option to reuse it in iceberg-rust instead re-implementing another expression + evaluation etc.?


Added .env.local as an environment for docker-compose configuration

Fixed docker-compose environment by adding .env.local as a env configuration.
LIIKUNTA-180

Trying to start the service with compose now.


[Bug] routine load 导入数据失败，物化视图统计字段不存在

Search before asking

[X] I had searched in the issues and found no similar issues.

Version
2.0.3
What's Wrong?
物化视图根据 _source_id 字段进行聚合计数
所有的数据都有该_source_id字段， 运行一段时间后， routineload 自动停止
报错
errCode = 2, detailMessage = column has no source field, column=mv__source_id
实际上所有数据都是有该_source_id字段的。
临时解决 只能删除物化视图后 重启routineload, 然后重建物化视图后正常
十天半个月就会出现一次
What You Expected?
服务正常
How to Reproduce?
No response
Anything Else?
No response
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

能提供能复现的步骤吗？