Query: AQL: bad allocation (exception location: D:\hw24-windows\oskar\work\ArangoDB\arangod\Aql\ExecutionBlockImpl.cpp:368). Please report this error to arangodb.com (while executing)

My Environment

ArangoDB Version:        3.8.1
Storage Engine:             RocksDB
Deployment Mode:       Single Server
Deployment Strategy:   Manual Start
Configuration:               default settings
Infrastructure:               own
Operating System:        Windows 10
Total RAM in your machine:        64 GB
Disks in use:        1x M2 NVME 512 GB
Used Package:              N/A

Component, Query & Data
Affected feature:
AQL query using web interface (also tried running query directly on arangosh)
AQL query (if applicable):
RETURN LENGTH(
    OUTERSECTION((FOR c IN call_log RETURN c._id), FLATTEN(FOR p IN phones FILTER LENGTH(p.callLogs) > 0 RETURN p.callLogs))
)

AQL explain and/or profile (if applicable):
Query String (155 chars, cacheable: true):
 RETURN LENGTH(
     OUTERSECTION((FOR c IN call_log RETURN c._id), FLATTEN(FOR p IN phones FILTER LENGTH(p.callLogs) > 0 RETURN p.callLogs))
 )

Execution plan:
 Id   NodeType                       Est.   Comment
  1   SingletonNode                     1   * ROOT
 19   SubqueryStartNode                 1     - LET #4 = ( /* subquery begin */
  8   EnumerateCollectionNode      621402       - FOR p IN phones   /* full collection scan, projections: `callLogs` */   FILTER (LENGTH(p.`callLogs`) > 0)   /* early pruning */
 11   CalculationNode              621402         - LET #10 = p.`callLogs`   /* attribute expression */   /* collections used: p : phones */
 20   SubqueryEndNode                   1         - RETURN  #10 ) /* subquery end */
 17   SubqueryStartNode                 1     - LET #1 = ( /* subquery begin */
 16   IndexNode                 202273369       - FOR c IN call_log   /* primary index scan, index only, projections: `_id` */
  4   CalculationNode           202273369         - LET #6 = c.`_id`   /* attribute expression */   /* collections used: c : call_log */
 18   SubqueryEndNode                   1         - RETURN  #6 ) /* subquery end */
 14   CalculationNode                   1     - LET #12 = LENGTH(OUTERSECTION(#1, FLATTEN(#4)))   /* simple expression */
 15   ReturnNode                        1     - RETURN #12

Indexes used:
 By   Name      Type      Collection   Unique   Sparse   Selectivity   Fields       Ranges
 16   primary   primary   call_log   true     false       100.00 %   [ `_key` ]   *

Functions used:
 Name           Deterministic   Cacheable   Uses V8
 FLATTEN        true            true        false
 OUTERSECTION   true            true        false
 LENGTH         true            true        false

Optimization rules applied:
 Id   RuleName
  1   move-calculations-up
  2   move-filters-up
  3   move-calculations-up-2
  4   move-filters-up-2
  5   move-calculations-down
  6   reduce-extraction-to-projection
  7   move-filters-into-enumerate
  8   splice-subqueries

Optimization rules with highest execution times:
 RuleName                                    Duration [s]
 use-indexes                                      0.00003
 reduce-extraction-to-projection                  0.00002
 splice-subqueries                                0.00001
 move-calculations-up                             0.00001
 move-calculations-down                           0.00001

41 rule(s) executed, 1 plan(s) created

Dataset:
{
  "_class": "com.app.model.CallLog",
  "loggedAt": "2020-10-19T13:31:12.437Z",
  "callStart": false
}

{
  "_class": "com.app.model.Phone",
  "createdAt": "2021-09-17T10:22:10.852Z",
  "number": "xxxxxxxxxxxxxxxx",
  "callLogs": [ "call_log/1327824417", "call_log/1327834800", ........... ]
}

Size of your Dataset on disk:
11 GB
The call_log table is really big and has around 224,080,537 doc(s), so the phone table is also big as there are references to that.
Steps to reproduce
Problem:
Query: AQL: bad allocation (exception location: D:\hw24-windows\oskar\work\ArangoDB\arangod\Aql\ExecutionBlockImpl.cpp:368). Please report this error to arangodb.com (while executing) (exception location: D:\hw24-windows\oskar\work\ArangoDB\arangod\RestHandler\RestCursorHandler.cpp:290). Please report this error to arangodb.com
This error is thrown when I try to execute this really slow query. Thing is I don't even have a D:\ unit on the computer running it.
Expected result:
This query should return the call_log entries count that are not in any list of the numbers. (orphan call logs)
Just as curiosity, here's a similar query being executed, but only counting items
RETURN {
    callLogs: LENGTH(FOR c IN call_log RETURN c._id),
    phoneCallLogs: LENGTH(FLATTEN(FOR p IN phones FILTER LENGTH(p.callLogs) > 0 RETURN p.callLogs))
}

The result is
107.832 seconds to execute
{
    "callLogs": 202273369,
    "phoneCallLogs": 202273352
}

It means there are 17 orphan call_log entries that I need to get it's ids to erase

Hi @goedderz let me ask you one more question
Considering the same entities from above "call_log" and "phones"
{
  "_class": "com.app.model.CallLog",
  "loggedAt": "2020-10-19T13:31:12.437Z",
  "callStart": false
}

{
  "_class": "com.app.model.Phone",
  "createdAt": "2021-09-17T10:22:10.852Z",
  "number": "xxxxxxxxxxxxxxxx",
  "callLogs": [ "call_log/1327824417", "call_log/1327834800", ........... ]
}

I need to list callLogs for a given phone sorted in descendant order (latest records first) and I'm using pagination to query data.
FOR p IN phones
FOR l IN call_log
FILTER p.number == "xxxxxxxxxx"
FILTER l._id IN p.callLogs[*]
SORT l.loggedAt DESC
LIMIT 0, 100
RETURN l

After arangodb3 service is restarted one single query like this takes around 10 seconds to execute in the first time.
As I have many running at the same time, they start accumulating, arangodb3 service locks and freezes for minutes (yesterday more than 30 minutes) and stops responding to new queries. Only after a lot of time (minutes, sometimes hours) when data is cached (50 GB of RAM used) the queries run smoothly.
The problem seems to be the sorting of "cl.loggedAt". So I changed the query a bit and tried to sort by "_key" instead. With this query time dropped from 10 to 4 seconds. But it's still a problem.
FOR p IN phones
FOR l IN call_log
FILTER p.number == "xxxxxxxxxx"
FILTER l._id IN p.callLogs[*]
SORT l._key DESC
LIMIT 0, 100
RETURN l

Here's the query explanation
Query String (141 chars, cacheable: true):
 FOR p IN phones FOR l IN call_log FILTER p.number == "xxxxxxxxxxxx" FILTER l._id IN
 p.callLogs[*] SORT l._key DESC LIMIT 0, 100 RETURN l

Execution plan:
 Id   NodeType          Est.   Comment
  1   SingletonNode        1   * ROOT
 13   IndexNode            1     - FOR p IN phones   /* hash index scan, projections: `callLogs` */
 12   IndexNode            9       - FOR l IN call_log LET #6 = l.`_key`   /* primary index scan */    /* with late materialization */
  9   SortNode             9         - SORT #6 DESC   /* sorting strategy: standard */
 10   LimitNode            9         - LIMIT 0, 100
 14   MaterializeNode      9         - MATERIALIZE l
 11   ReturnNode           9         - RETURN l

Indexes used:
 By   Name                      Type      Collection   Unique   Sparse   Selectivity   Fields         Ranges
 13   idx_1668782269401137152   hash      phones       true     false       100.00 %   [ `number` ]   (p.`number` == "xxxxxxxxxxxx")
 12   primary                   primary   call_log   true     false       100.00 %   [ `_key` ]     (l.`_id` IN p.`callLogs[*]`)

Optimization rules applied:
 Id   RuleName
  1   move-calculations-up
  2   move-filters-up
  3   move-calculations-up-2
  4   move-filters-up-2
  5   use-indexes
  6   remove-filter-covered-by-index
  7   remove-unnecessary-calculations-2
  8   reduce-extraction-to-projection
  9   late-document-materialization

Optimization rules with highest execution times:
 RuleName                                    Duration [s]
 use-indexes                                      0.00005
 remove-filter-covered-by-index                   0.00001
 move-calculations-up                             0.00001
 late-document-materialization                    0.00001
 reduce-extraction-to-projection                  0.00001

42 rule(s) executed, 1 plan(s) created

"call_log" table has only the default primary index "_key"
"phone" table has 3 indexes, the default _key, a hash index on "number" and a persistent index on "callLogs'
Why isn't the "p.callLogs" index being used in the query explanation?
Are there any other ways of optimizing that?
Thanks

Hi @brunoaduarte!
First let me point out that it's always really helpful having a profile of the query which performance you want to improve. Could you provide one in your next comment?
The main thing that makes this query tricky is this:
You cannot use one index for lookups (e.g. for l._id IN p.callLogs[*]), and another one for sorting. You'd have to be able to use a single index for both lookup and sorting. An example where this would be possible is an index ["value", "timestamp"] with a query like FOR d IN col FILTER d.value == @value SORT d.timestamp. But note that FOR d IN col FILTER d.value IN @values SORT d.timestamp would not work (or only if @values contains only a single value). This is because the index is sorted lexicographically, first by value, then by timestamp. Therefore FOR d IN col FILTER d.value IN @values SORT d.value, d.timestamp would still work. And note that SORT d.value, d.timestamp and SORT d.timestamp yield the same results if all d.value are the same, which is why the first example also works.
Therefore, as the index cannot be used for sorting in your example, all documents have to be read, sorted, and then only a few of them will be returned. I don't see an obvious way how that could be completely avoided.
One reason why I reckon the _key example is faster is that it's part of the index the FOR node is reading from. This means the whole document doesn't yet have to be read before sorting (_key is read from the index instead, see FOR l IN call_log LET #6 = l.\_key`in the explain output), and only the top 100 have to be read later (see theMATERIALIZEnode in the explain output). As that isn't the case when sorting byloggedAt`, all documents have to be read already before sorting.
You might be able to get that improvement by adding an index on [_key, loggedAt]. In case the optimizer isn't able to figure out for itself that this is a good idea, try to supply an index hint: https://www.arangodb.com/docs/stable/aql/operations-for.html#indexhint
Best,
Tobias

You might be able to get that improvement by adding an index on [_key, loggedAt]. In case the optimizer isn't able to figure out for itself that this is a good idea, try to supply an index hint: https://www.arangodb.com/docs/stable/aql/operations-for.html#indexhint
Best,
Tobias

Hi @goedderz , thanks for your reply! Sure. Here are the profiles for both cases
Sorting by "loggedAt"
Query String (147 chars, cacheable: false):
 FOR p IN phones FOR l IN call_log FILTER p.number == "xxxxxxx" FILTER l._id IN
 p.callLogs SORT l.loggedAt DESC LIMIT 0, 100 RETURN l

Execution plan:
 Id   NodeType          Calls    Items   Runtime [s]   Comment
  1   SingletonNode         1        1       0.00000   * ROOT
 13   IndexNode             1        1       0.00786     - FOR p IN phones   /* hash index scan, projections: `callLogs` */
 12   IndexNode           181   180609      13.32903       - FOR l IN call_log   /* primary index scan */
  8   CalculationNode     181   180609       0.05977         - LET #6 = l.`loggedAt`   /* attribute expression */   /* collections used: l : call_log */
  9   SortNode              1      100       0.24297         - SORT #6 DESC   /* sorting strategy: standard */
 10   LimitNode             1      100       0.00001         - LIMIT 0, 100
 11   ReturnNode            1      100       0.00001         - RETURN l

Indexes used:
 By   Name                      Type      Collection   Unique   Sparse   Selectivity   Fields         Ranges
 13   idx_1668782269401137152   hash      phones       true     false       100.00 %   [ `number` ]   (p.`number` == "xxxxxxx")
 12   primary                   primary   call_log   true     false       100.00 %   [ `_key` ]     (l.`_id` IN p.`callLogs`)

Optimization rules applied:
 Id   RuleName
  1   move-calculations-up
  2   move-filters-up
  3   move-calculations-up-2
  4   move-filters-up-2
  5   use-indexes
  6   remove-filter-covered-by-index
  7   remove-unnecessary-calculations-2
  8   reduce-extraction-to-projection

Query Statistics:
 Writes Exec   Writes Ign   Scan Full   Scan Index   Filtered   Peak Mem [b]   Exec Time [s]
           0            0           0       180610          0       54067200        13.64019

Query Profile:
 Query Stage           Duration [s]
 initializing               0.00001
 parsing                    0.00004
 optimizing ast             0.00001
 loading collections        0.00001
 instantiating plan         0.00005
 optimizing plan            0.00037
 executing                 13.63970
 finalizing                 0.00003

Sorting by _key
Query String (143 chars, cacheable: false):
 FOR p IN phones FOR l IN call_log FILTER p.number == "xxxxx" FILTER l._id IN
 p.callLogs SORT l._key DESC LIMIT 0, 100 RETURN l

Execution plan:
 Id   NodeType          Calls    Items   Runtime [s]   Comment
  1   SingletonNode         1        1       0.00001   * ROOT
 13   IndexNode             1        1       0.00645     - FOR p IN phones   /* hash index scan, projections: `callLogs` */
 12   IndexNode           181   180609       3.29125       - FOR l IN call_log LET #6 = l.`_key`   /* primary index scan */    /* with late materialization */
  9   SortNode              1      100       0.17029         - SORT #6 DESC   /* sorting strategy: standard */
 10   LimitNode             1      100       0.00001         - LIMIT 0, 100
 14   MaterializeNode       1      100       0.00163         - MATERIALIZE l
 11   ReturnNode            1      100       0.00001         - RETURN l

Indexes used:
 By   Name                      Type      Collection   Unique   Sparse   Selectivity   Fields         Ranges
 13   idx_1668782269401137152   hash      phones       true     false       100.00 %   [ `number` ]   (p.`number` == "xxxxx")
 12   primary                   primary   call_log   true     false       100.00 %   [ `_key` ]     (l.`_id` IN p.`callLogs`)

Optimization rules applied:
 Id   RuleName
  1   move-calculations-up
  2   move-filters-up
  3   move-calculations-up-2
  4   move-filters-up-2
  5   use-indexes
  6   remove-filter-covered-by-index
  7   remove-unnecessary-calculations-2
  8   reduce-extraction-to-projection
  9   late-document-materialization

Query Statistics:
 Writes Exec   Writes Ign   Scan Full   Scan Index   Filtered   Peak Mem [b]   Exec Time [s]
           0            0           0       180610          0       25493504         3.47005

Query Profile:
 Query Stage           Duration [s]
 initializing               0.00000
 parsing                    0.00004
 optimizing ast             0.00001
 loading collections        0.00001
 instantiating plan         0.00003
 optimizing plan            0.00029
 executing                  3.46966
 finalizing                 0.00003

I've managed to improve the query a little bit by running it like this
Query String (565 chars, cacheable: false):
 LET shIds = FLATTEN(FOR p IN phones FILTER p.number == "xxxxx" RETURN p.callLogs)
 FOR l IN call_log FILTER l._id IN shIds SORT l._key DESC LIMIT 0, 100 RETURN l

Execution plan:
 Id   NodeType            Calls   Items   Runtime [s]   Comment
  1   SingletonNode           1       1       0.00000   * ROOT
 19   SubqueryStartNode       1       2       0.00001     - LET #1 = ( /* subquery begin */
 18   IndexNode               1       2       0.00729       - FOR p IN phones   /* hash index scan, projections: `callLogs` */
  6   CalculationNode         1       2       0.00101         - LET #7 = p.`callLogs`   /* attribute expression */   /* collections used: p : phones */
 20   SubqueryEndNode         1       1       0.00054         - RETURN  #7 ) /* subquery end */
 17   IndexNode               1     100       0.31862     - FOR l IN call_log   /* reverse primary index scan */
 15   LimitNode               1     100       0.00001       - LIMIT 0, 100
 16   ReturnNode              1     100       0.00001       - RETURN l

Indexes used:
 By   Name                      Type      Collection   Unique   Sparse   Selectivity   Fields         Ranges
 18   idx_1668782269401137152   hash      phones       true     false       100.00 %   [ `number` ]   (p.`number` == "xxxxx")
 17   primary                   primary   call_log     true     false       100.00 %   [ `_key` ]     (l.`_id` IN FLATTEN(#1))

Optimization rules applied:
 Id   RuleName
  1   move-calculations-up
  2   move-filters-up
  3   move-calculations-up-2
  4   move-filters-up-2
  5   use-indexes
  6   remove-filter-covered-by-index
  7   use-index-for-sort
  8   remove-unnecessary-calculations-2
  9   reduce-extraction-to-projection
 10   splice-subqueries

Query Statistics:
 Writes Exec   Writes Ign   Scan Full   Scan Index   Filtered   Peak Mem [b]   Exec Time [s]
           0            0           0          101          0       22315008         0.32795

Query Profile:
 Query Stage           Duration [s]
 initializing               0.00000
 parsing                    0.00005
 optimizing ast             0.00001
 loading collections        0.00001
 instantiating plan         0.00003
 optimizing plan            0.00032
 executing                  0.32750
 finalizing                 0.00003

I could not use the index created with [ _key, loggedAt ]
ID | Type | Unique | Sparse | Extras | Selectivity Est. | Fields | Name
1018833048 | persistent | false | false | deduplicate: false, estimates: true | 99.99% | _key, loggedAt | idx_1716566713295699968

FOR p IN phones FOR l IN call_log OPTIONS { indexHint: "idx_1716566713295699968", forceIndexHint: true } FILTER p.number == "xxxxx" FILTER l._id IN p.callLogs SORT l.loggedAt DESC LIMIT 0, 100 RETURN l`

Query: AQL: could not use index hint to serve query; {"indexHint":{"forced":true,"type":"simple","hint":["idx_1716566713295699968"]}} (while optimizing plan)

Hi @brunoaduarte,
I can't see on the first glance why the index shouldn't be usable. You might want to open another bug report for this, and provide the query and available indexes on participating collections.
Apart from that, I don't see much to improve given the current data model.
If you're able and willing to change the data model, you could change the way you save the relation between phones and call_log. For example by saving a list of related phones at every call_log, either instead of the other way round, or in addition to if necessary for other queries. Then the query could be written as
FOR l in call_log
  FILTER @phone IN l.phones
  SORT l.loggedAt
  LIMIT 100
  RETURN l

. An (array) index on call_log with the fields [ "phones[*]", "loggedAt" ] should then be able to satisfy filter and provide the value loggedAt.
However, I don't expect a huge performance gain, probably in the ballpark of the Sorting by _key query.
Another option would be to keep the callLogs array in the phones documents always sorted by the corresponding loggedAt values, so you don't have to SORT in the first place.
Best,
Tobias

P.S.: You might also want to consider using an edge collection to model the relation between phones and call_log. If you also add the loggedAt timestamp of the corresponding call_log to each edge, a vertex centric index on [_to, loggedAt] could be used for both lookup and sort. This should improve the runtime significantly.

Since ArangoDB 3.8 is meanwhile EOL and several hints have been given how to work around this I'm closing this issue as solved.


White flash in the model-viewer before reloading the page

Description
When the page reloads in the moment before it, model-viewer fills with white background (Like flash before reloading).
Version

model-viewer: v3.0.0

Browser Affected

[x] Chrome, version: 120.0.6099.71
[x] Edge
[ ] Firefox
[ ] IE
[ ] Safari

OS

[ ] Android
[ ] iOS
[ ] Linux
[ ] MacOS
[x] Windows

AR

[ ] WebXR
[ ] SceneViewer
[ ] QuickLook

Can you add a link and repro steps? Ideally a screen recording? Does it work as expected on other devices/browsers/MV versions?

Hi! Sorry for late answer. Yes, you can try it on me website https://snl4.ch/
I used this code:
<model-viewer src="/components/models/model.glb" ios-src="/components/models/model.usdz" alt="Show" touch-action="none" disable-pan camera-controls disable-zoom camera-orbit="90deg 80deg 100%" orbit-sensitivity="0.5" interaction-prompt="none" loading="eager"></model-viewer>

https://github.com/google/model-viewer/assets/149927173/4bb29748-d583-49e7-9821-5f257752b56f
I tried a normal reboot, Ctrl + F5
But it can be noted that if you reboot very often, the white flash won't appear, or if you reboot after a long time, it won't do that either. But in normal cases this happens. I haven't tried it on other browsers

I can't repro it on my mac, but our default poster image is white, so you might be seeing that. I would highly recommend you use a poster-image (you can generate seamless ones on our editor) as that will always give a better experience, especially on slow internet, and it'll probably fix this problem as well.


RTC.ccp has error: 2nd "GetDate(" needs changing to "SetDate("

https://github.com/m5stack/M5StickC-Plus/blob/master/src/RTC.cpp as at 9Oct22 has an error in it:

There are TWO "void RTC::GetDate(RTC_DateTypeDef* RTC_DateStruct)",
the lower of which needs changing to "void RTC::SetDate(RTC_DateTypeDef* RTC_DateStruct)"

Note that " Fixed Get/Set Data->Date #40 " does NOT appear to have fixed this error in RTC.ccp"...

Fixed,0dc7e13


refs 217: added option for change spoken text or dialog option to gibberish

added gibberish generator
https://github.com/carenalgas/popochiu/assets/22888465/04cb617d-f63e-478d-a92c-46daf4f8f15d

cool today I'll try to make some changes :)

Hey @mefjak . Sorry for dissappearing for so long. Do you think it is possible that you put this branch up do date with the last changes in develop so we can merge it before the release of Popochiu 2? If you don't have the time to do so, I can try to help.

no problem :) i try to update it :)

@mapedorr  sorry for not keeping up-to-date. I did try to update it now. But I am still having difficult time to properly test it :)

No worries. If it is fine for you, we can help with the testing.

If it's ok please do - but if you got time can you describe your workflow? I think I didnset up everything but always got some problems with git or testing 😅

Hey @mefjak ! I made a small update on your branch to fix a bug, but appart from that, looks like it is ready to be merged. I'll make a couple of additional tests tomorrow just to double check everything is fine.

Cool, thank you very much


npm packages are not automatically restored

Hi Guys,
In my solution, gulpfile.js and package.json are together located in a solution folder, same level as sln file, and they are under the source control as well. the problem right now is, whenever i get latest version of these files, and let say they have been updated by a team member. the npm packages are not automatically updated until i save the package.json manually. is it expected?

@yechongbo the restore will not happen until the .json file is in fact changed.

@spboyer, thanks for the clarification. but it seems it works when they are at project level.

interesting, sorry missed the "location" portion.  Create a new issue in  Tooling repo -> https://github.com/aspnet/tooling/issues


fix MemoryStore broken due to SDK update (79)

Following up existing open Pull and fixing the comments
#79

@hxquangnhat FYI


:memo: [doc] Added example for s3 textract

I was struggling when using the library and I hope this simple example helps somebody in the future.
Best wishes,
Jing Hui P.

I'm not sure what would be a good home for it but the generated code is not the place at least. Tonight, this file will be re-generated so even if we were to merge this, it'd be lost the night after.
May I consider either a EXAMPLES.md or an examples/textract.ex file to be added instead? :-)

Rebased on latest master just to make sure the CI still passes. Locally it works just fine but want the CI to agree with me prior to merging!
I'm really sorry for the delay 🤐

@enkr1 examples/* do not get published to hexdocs but I'll open an issue to look into that. Having it be part of the repo is a good start at least IMHO👌


add paperhash parameter

@mzarozinski this is the way to call it:
openreview.get_notes(paperhash = 'kim|interpretable_3d_human_action_analysis_with_temporal_convolutional_networks')

Be sure that this index is in the database:
notesDb.ensureHashIndex('content.paperhash');

done!


BUG: Theme fails to install properly on KDE Plasma 5.24.5

Attempted to install through the Plasma Store method, which returned the following error message:
Download of "Catppuccin Colors" failed, error: SETTINGS invalid number of concurrent streams"
Attempted to install from this repository by following the instructions. This returned this error message:
installing...
qt.network.http2: connection error: SETTINGS invalid number of concurrent streams
qt.network.http2: stream 1 finished with error: "SETTINGS invalid number of concurrent streams"
kf.newstuff.core: "SETTINGS invalid number of concurrent streams"
kf.newstuff.core: "Download of \"Catppuccin Colors\" failed, error: SETTINGS invalid number of concurrent streams"
kns error: KNSCore::InstallationError "Download of \"Catppuccin Colors\" failed, error: SETTINGS invalid number of concurrent streams" QVariant(Invalid)
Error: Installation of /home/mr/Downloads/KDE/kde-store-archives/global-theme/catppuccin.tar.gz failed: Could not install dependency: 'kns://colorschemes.knsrc/api.kde-look.org/1690362'

Here is full system information:
Operating System: Manjaro Linux
KDE Plasma Version: 5.24.5
KDE Frameworks Version: 5.94.0
Qt Version: 5.15.4
Kernel Version: 5.18.3-1-MANJARO (64-bit)
Graphics Platform: X11
Processors: 8 × Intel® Core™ i7-3770 CPU @ 3.40GHz
Memory: 31.3 GiB of RAM
Graphics Processor: NVIDIA GeForce GTX 745/PCIe/SSE2

Seems to just be KDE issue.


Add anonymous synthetic types for Go struct generation

This creates synthetic types for where we need anonymous members in generated Go structs.

@ostromart: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

gencheck_operator
30d39dc845494e2eb30cc0b9cdf1c5bc3e31a2ae
link
/test gencheck_operator

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

gencheck_operator
42b0d21f8c10dcde385d31ae80bf3440ae932b40
link
/test gencheck_operator

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

gencheck_operator
1cb3322402581059a0ea94561f7a75fdaaf6b1a3
link
/test gencheck_operator

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

gencheck_operator
1cb3322402581059a0ea94561f7a75fdaaf6b1a3
link
/test gencheck_operator

lint_operator
1cb3322402581059a0ea94561f7a75fdaaf6b1a3
link
/test lint_operator

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

gencheck_operator
1cb3322402581059a0ea94561f7a75fdaaf6b1a3
link
/test gencheck_operator

lint_operator
1cb3322402581059a0ea94561f7a75fdaaf6b1a3
link
/test lint_operator

mesh_operator
1cb3322402581059a0ea94561f7a75fdaaf6b1a3
link
/test mesh_operator

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

gencheck_operator
1cb3322402581059a0ea94561f7a75fdaaf6b1a3
link
/test gencheck_operator

lint_operator
1cb3322402581059a0ea94561f7a75fdaaf6b1a3
link
/test lint_operator

mesh_operator
1cb3322402581059a0ea94561f7a75fdaaf6b1a3
link
/test mesh_operator

mandiff_operator
1cb3322402581059a0ea94561f7a75fdaaf6b1a3
link
/test mandiff_operator

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

gencheck_operator
1cb3322402581059a0ea94561f7a75fdaaf6b1a3
link
/test gencheck_operator

lint_operator
1cb3322402581059a0ea94561f7a75fdaaf6b1a3
link
/test lint_operator

mesh_operator
1cb3322402581059a0ea94561f7a75fdaaf6b1a3
link
/test mesh_operator

mandiff_operator
1cb3322402581059a0ea94561f7a75fdaaf6b1a3
link
/test mandiff_operator

test_operator
1cb3322402581059a0ea94561f7a75fdaaf6b1a3
link
/test test_operator

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ostromart: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

lint_operator
1cb3322402581059a0ea94561f7a75fdaaf6b1a3
link
/test lint_operator

mesh_operator
1cb3322402581059a0ea94561f7a75fdaaf6b1a3
link
/test mesh_operator

mandiff_operator
1cb3322402581059a0ea94561f7a75fdaaf6b1a3
link
/test mandiff_operator

test_operator
1cb3322402581059a0ea94561f7a75fdaaf6b1a3
link
/test test_operator

gencheck_operator
e1ab7f740578e0dae10adcfcaef3b54da3f226ae
link
/test gencheck_operator

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

$ make clean gen
-> modified:   pkg/apis/istio/v1alpha1/values_types.pb.go
(this is due to fixup_structs not running and running the substitution on gogo proto include)
$ touch pkg/apis/istio/v1alpha1/values_types.proto
$ make gen
-> modified file disappears i.e. substitution runs correctly.
Any ideas what could be causing this @geeknoid ?

@ostromart That's strange. Go ahead and ignore the job failure (it's not required) for now, and once this change goes in I'll try to figure out what's going wrong. It's probably something silly.

Hey guys, it's probably a bit late for this comment, but why are we using .proto files instead of k8s/code-generator go-to-protobuf?  I think using that would solve a lot of problems, as the type would be defined in go, as it's used by the controller, while making protobuf available for use with the apiserver.

@rcernich We have infrastructure built around using proto as source of truth (docs, validation, schema generation, etc). I don't want operator to be an outlier as compared to the rest of Istio.

@geeknoid, understood.  However, these types are specific to the operator, which only runs in k8s, thus not the same.  I think it's fine to have what is essentially the 'spec' portion in istio/api, but have the wrapping k8s type fully declared in this project, or at least in istio/client-go.  That said, the type used by the operator will have a status field, which is currently not supported with the k8s api generation toolset (i.e. kubetype-gen).  I think we're being pedantic forcing the k8s type into istio/api.

There's an ongoing discussion elsewhere about ongoing problems and history with using proto as a source of truth.
With considerable effort, I think we've managed to make it work. It's certainly not a perfect solution. I propose revisiting this discussion after 1.4 - this is a very deep and involved topic with lots of surprises and caveats where the rubber hits the road.
@sdake @rcernich could one of you guys please approve this for 1.4? We should create an issue to tackle the bigger issue of source of truth, use of jsonpb, gogo etc. and fix this or at least agree on the direction at the start of 1.5.

I'll defer to @sdake, as I think he has a PR dealing with removing gogo libraries.  That said, I'm okay with this.  If it's not going to disrupt that effort, let me know and I'll approve this.


Make Travis CI be able to run on forks

When developing a feature it often happens during development that I do not want to create a pull request straight away but do some testing in my fork and when successful I will create a PR.
Unfortunately currently travis doesn't run on a fork due to invalid paths. This PR fixes this so a Fork can run the test unit as well.

👍


Upgrade Tether.app to v1.3.0

Thank you for the contribution. It will be merged in a different pull request due to some simple recommended alterations. Your contribution is still included (and still credited to you), with the appropriate modifications, when applicable.
Changes:

Facilitated future updates.


First release + more mature repo standards

Summary
Preparations for the first release.
Proposal
Minimal list of prerequisites:

CHANGELOG
README update
release not necessarily aligned with milestones, e.g., this release fill fall between milestones 5 and 6.

For Admin Use

[ ] Not duplicate issue
[ ] Appropriate labels applied
[ ] Appropriate contributors tagged
[ ] Contributor assigned/self-assigned

release not necessarily aligned with milestones, e.g., this release fill fall between milestones 5 and 6.

Yes, this was on purpose. Milestones are mid-month to allow for release preparation by end of month. But now with releases every every month not sure it makes sense anymore.


chore: bdd-spring-1575977693 to 0.0.1

chore: Promote bdd-spring-1575977693 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Feature: Copy to clipboard

Nice component, I think it would be pretty neat to have the option of a copy to clipboard option baked in

Now you can make wrapper above this component and use copy package like vue-clipboard2, why not?)


Illegal instruction (core dumped) while importing hnswlib on an AWS Batch Job

The code works locally and on an EC2 instance. But while running it as a batch job, I keep getting this error: Illegal instruction (core dumped) python3 ./jarvis_e.py $ENV because of this command in my docker file "RUN pip install hnswlib"

Hi @AkhilKothari7, I am not sure what is the issue.
You can login into the last runnable docker container and try to debug it from there.


npm start fail

After git pull and 'sudo npm install' I now get this error when running 'npm start':
npm start

><EMAIL_ADDRESS>> electron main.js

sh: 1: electron: not found

npm ERR<EMAIL_ADDRESS>start: `electron main.js`
npm ERR! Exit status 127
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>start script.
npm ERR! This is most likely a problem with the OpenBazaar package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     electron main.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls OpenBazaar
npm ERR! There is likely additional logging output above.
npm ERR! System Linux 3.2.0-92-generic-pae
npm ERR! command "/usr/local/bin/node" "/usr/local/bin/npm" "start"
npm ERR! cwd /home/nondescript/OpenBazaar-Client
npm ERR! node -v v0.10.32
npm ERR! npm -v 1.4.28
npm ERR! code ELIFECYCLE
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     /home/nondescript/OpenBazaar-Client/npm-debug.log
npm ERR! not ok code 0

Solved by running:
 sudo npm install electron-prebuilt


Repo URL in the marketplace is wrong

Error 404 on https://github.com/sandipchitale/vscode-sigs-view.git
Needs to be updated to this repo.

Fixed by renaming repository.


Not able to run with llama.cpp on MTL platform with various errors

I'm experiencing various issues when using on MTL platform. No successes.
Below are some of the errors:

Crashed on MTL when offload all layers. Error code  :: Unknow Native API error -999
Online video playback streaming at 1080P has playback distortion when running inferencing
Received following error when using server.exe mode

command:
llama.cpp\server.exe -m "C:\llm\models\llama-2-7b-chat.Q4_K_M.gguf" -ngl 10

app side:
ReadError: [WinError 10054] An existing connection was forcibly closed by the remote host

llama.cpp side:

GGML_ASSERT: C:\Users\Administrator\actions-runner\bigdl-core-cpp-release\_work\llm.cpp\llm.cpp\llama-cpp-bigdl\ggml.c:14154: ne1 == N
GGML_ASSERT: C:\Users\Administrator\actions-runner\bigdl-core-cpp-release\_work\llm.cpp\llm.cpp\llama-cpp-bigdl\ggml.c:14154: ne1 == N

Received following error when running main with interactive mode:

command:
main.exe -m "C:\llm\models\llama-2-7b-chat.Q4_K_M.gguf" -n 256 --repeat_penalty 1.0 --color -i -r "User:" -f prompts/chat-with-bob.txt

 Transcript of a dialog, where the User interacts with an Assistant named Bob. Bob is helpful, kind, honest, good at writing, and never fails to answer the User's requests immediately and with precision.

User: Hello, Bob.
Bob: Hello. How may I help you today?
User: Please tell me the largest city in Europe.
Bob: Sure. The largest city in Europe is Moscow, the capital of Russia.
User:how many states in USA and list all of them and the founding date
BobGGML_ASSERT: C:\Users\Administrator\actions-runner\bigdl-core-cpp-release\_work\llm.cpp\llm.cpp\llama-cpp-bigdl\ggml.c:14154: ne1 == N
GGML_ASSERT: C:\Users\Administrator\actions-runner\bigdl-core-cpp-release\_work\llm.cpp\llm.cpp\llama-cpp-bigdl\ggml.c:14154: ne1 == N

System information

OS Version:                10.0.22631 N/A Build 22631
OS Manufacturer:           Microsoft Corporation
OS Configuration:          Standalone Workstation
OS Build Type:             Multiprocessor Free
System Model:              HP Spectre x360 2-in-1 Laptop 14-eu0xxx
System Type:               x64-based PC
Processor(s):              1 Processor(s) Installed.
                           [01]: Intel64 Family 6 Model 170 Stepping 4 GenuineIntel ~1400 Mhz
BIOS Version:              AMI F.05, 12/26/2023
System Locale:             en-us;English (United States)
Input Locale:              en-us;English (United States)
Time Zone:                 (UTC-08:00) Pacific Time (US & Canada)
Total Physical Memory:     15,912 MB
Available Physical Memory: 2,151 MB
Virtual Memory: Max Size:  31,784 MB
Virtual Memory: Available: 10,302 MB
Virtual Memory: In Use:    21,482 MB

GPU:
Adapter Type	Intel(R) Arc(TM) Graphics     Family, Intel Corporation compatible
Adapter Description	Intel(R) Arc(TM) Graphics
Adapter RAM	128.00 MB (134,217,728 bytes)
Driver Version	<IP_ADDRESS>35

oneAPI
:: initializing oneAPI environment...
   Initializing Visual Studio command-line environment...
   Visual Studio version 17.9.3 environment configured.
   "C:\Program Files\Microsoft Visual Studio\2022\Community\"
   Visual Studio command-line environment initialized for: 'x64'
:  advisor -- latest
:  compiler -- latest
:  dal -- latest
:  debugger -- latest
:  dev-utilities -- latest
:  dnnl -- latest
:  dpcpp-ct -- latest
:  dpl -- latest
:  ipp -- latest
:  ippcp -- latest
:  mkl -- latest
:  tbb -- latest
:  vtune -- latest
:: oneAPI environment initialized ::

llama.cpp log
llm_load_print_meta: format           = GGUF V2
llm_load_print_meta: arch             = llama
llm_load_print_meta: vocab type       = SPM
llm_load_print_meta: n_vocab          = 32000
llm_load_print_meta: n_merges         = 0
llm_load_print_meta: n_ctx_train      = 4096
llm_load_print_meta: n_embd           = 4096
llm_load_print_meta: n_head           = 32
llm_load_print_meta: n_head_kv        = 32
llm_load_print_meta: n_layer          = 32
llm_load_print_meta: n_rot            = 128
llm_load_print_meta: n_embd_head_k    = 128
llm_load_print_meta: n_embd_head_v    = 128
llm_load_print_meta: n_gqa            = 1
llm_load_print_meta: n_embd_k_gqa     = 4096
llm_load_print_meta: n_embd_v_gqa     = 4096
llm_load_print_meta: f_norm_eps       = 0.0e+00
llm_load_print_meta: f_norm_rms_eps   = 1.0e-06
llm_load_print_meta: f_clamp_kqv      = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: f_logit_scale    = 0.0e+00
llm_load_print_meta: n_ff             = 11008
llm_load_print_meta: n_expert         = 0
llm_load_print_meta: n_expert_used    = 0
llm_load_print_meta: causal attn      = 1
llm_load_print_meta: pooling type     = 0
llm_load_print_meta: rope type        = 0
llm_load_print_meta: rope scaling     = linear
llm_load_print_meta: freq_base_train  = 10000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_yarn_orig_ctx  = 4096
llm_load_print_meta: rope_finetuned   = unknown
llm_load_print_meta: ssm_d_conv       = 0
llm_load_print_meta: ssm_d_inner      = 0
llm_load_print_meta: ssm_d_state      = 0
llm_load_print_meta: ssm_dt_rank      = 0
llm_load_print_meta: model type       = 7B
llm_load_print_meta: model ftype      = Q4_K - Medium
llm_load_print_meta: model params     = 6.74 B
llm_load_print_meta: model size       = 3.80 GiB (4.84 BPW)
llm_load_print_meta: general.name     = LLaMA v2
llm_load_print_meta: BOS token        = 1 '<s>'
llm_load_print_meta: EOS token        = 2 '</s>'
llm_load_print_meta: UNK token        = 0 '<unk>'
llm_load_print_meta: LF token         = 13 '<0x0A>'
[SYCL] call ggml_init_sycl
ggml_init_sycl: GGML_SYCL_DEBUG: 0
ggml_init_sycl: GGML_SYCL_F16: no
found 5 SYCL devices:
|  |                  |                                             |Compute   |Max compute|Max work|Max sub|               |
|ID|       Device Type|                                         Name|capability|units      |group   |group  |Global mem size|
|--|------------------|---------------------------------------------|----------|-----------|--------|-------|---------------|
| 0|[level_zero:gpu:0]|                    Intel(R) Arc(TM) Graphics|       1.3|        128|    1024|     32|<PHONE_NUMBER>|
| 1|    [opencl:gpu:0]|                    Intel(R) Arc(TM) Graphics|       3.0|        128|    1024|     32|<PHONE_NUMBER>|
| 2|    [opencl:cpu:0]|               Intel(R) Core(TM) Ultra 7 155H|       3.0|         22|    8192|     64|<PHONE_NUMBER>|
| 3|    [opencl:cpu:1]|               Intel(R) Core(TM) Ultra 7 155H|       3.0|         22|    8192|     64|<PHONE_NUMBER>|
| 4|    [opencl:acc:0]|               Intel(R) FPGA Emulation Device|       1.2|         22|67108864|     64|<PHONE_NUMBER>|
ggml_backend_sycl_set_mul_device_mode: true
detect 1 SYCL GPUs: [0] with top Max compute units:128
llm_load_tensors: ggml ctx size =    0.22 MiB
llm_load_tensors: offloading 10 repeating layers to GPU
llm_load_tensors: offloaded 10/33 layers to GPU
llm_load_tensors:      SYCL0 buffer size =  1192.42 MiB
llm_load_tensors:        CPU buffer size =  3891.24 MiB
.................................................................................................
llama_new_context_with_model: n_ctx      = 512
llama_new_context_with_model: n_batch    = 512
llama_new_context_with_model: n_ubatch   = 512
llama_new_context_with_model: freq_base  = 10000.0
llama_new_context_with_model: freq_scale = 1
llama_kv_cache_init:      SYCL0 KV buffer size =    80.00 MiB
llama_kv_cache_init:  SYCL_Host KV buffer size =   176.00 MiB
llama_new_context_with_model: KV self size  =  256.00 MiB, K (f16):  128.00 MiB, V (f16):  128.00 MiB
llama_new_context_with_model:  SYCL_Host  output buffer size =     0.24 MiB
llama_new_context_with_model:      SYCL0 compute buffer size =   173.04 MiB
llama_new_context_with_model:  SYCL_Host compute buffer size =    17.01 MiB
llama_new_context_with_model: graph nodes  = 1062
llama_new_context_with_model: graph splits = 246

Running llama.cpp + sycl built directly works as expected:

Hi @hpxiong ,
The two errors you received when using server.exe mode or interactive mode is triggered by only offloading partial model layers to GPU, our latest code (ipex-llm[cpp] > 2.5.0b20240401) has bug with this case, we will try to fix it.
One workaround for this case is you can try with ipex-llm[cpp]==2.5.0b20240401, which has not introduced this bug.
As for "Crashed on MTL when offload all layers", we have not seen related error before.
Could you please provide us more info about this, like detailed error log, your system info, iGPU memory/driver info,  oneapi version etc ?

@rnwang04
Thank you for your reply.  I actually do not observe crash when using -ngl 999 while previously I was using -ngl -1. Not sure if this made a difference or not.  But at least this is working for me now and LLM chat is working.
However, I still have problem using this for RAG with LangChain based implementation. The model will not return anything readable. While I can see properly data source was returned from vector database but not the answer. Do you have any suggestions? Thanks!

Hi @hpxiong ,
Do you mean you are using our llama.cpp for RAG with LangChain based implementation and meet some problems?
If so, we have not tried this, could you please provide us with some source code so that we can reproduce your code?

@rnwang04
Yes, I do have a project here: https://github.com/intel-sandbox/hpxiong-dbot.git. Maybe we can setup a quick offline chat? You should be able to find me on outlook using my GitHub id. Thank you.

Hi @hpxiong
I have reproduced this issue with your RAG project.
After some debugging, I found the garbage output is caused by in your rag case, template for llama is missing.
Just change ./server -m ./llama-2-7b-chat.Q4_K_M.gguf -ngl 999 --port 8016 --host <IP_ADDRESS> to
./server -m ./llama-2-7b-chat.Q4_K_M.gguf -ngl 999 --port 8016 --host <IP_ADDRESS> -v --chat-template llama2 can solve this issue.
Below is my test result:
IPEX-LLM.txt

By the way, we also have a RAG example based on torch backend (https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/chatchat_quickstart.html) for your reference : )

@rnwang04
Thank you very much for your effort. You are right, the RAG problem is due to the mismatching prompt template.


libswoc: Replace ts::IpAddrSet with swoc::IPRangeSet

Due to complaints by particularly noisy PMC member, the simple IP address set logic was moved from the ATS codebase to libswoc. This removes the ATS variant and updates code that depends on it.

[approve ci fedora]


Knowledge management strategy (editing)   Q2

$100 each to

Sucre
Tevo

on delivery of an edited draft, on or before 2nd August 2024

Just for reference, you'll find the draft strategy from your writing team in the relevant issue https://github.com/SingularityNet-Ambassador-Program/AI-Ethics-Workgroup/issues/16
Here's a direct link to it https://docs.google.com/document/d/1yo9MuYSA_4ZOkMH2KJibUu6C7tlzHFe9e4YC7mvVl6g/edit?usp=sharing

Here's the edited version: https://docs.google.com/document/d/1yo9MuYSA_4ZOkMH2KJibUu6C7tlzHFe9e4YC7mvVl6g/edit?pli=1&disco=AAABSxTuDfU

If you didn't receive payments for this today, please know that payments will be made next Tuesday. The reason is because an error was made when sending the tasks for payment on Dework. The sign $ was used instead of USD to place the fund request. Apologies for this.


Incorrect DateTime format

I think the BuiltinNumberFormat for numFmtId= 14 should follow the short date format (CultureInfo.CurrentCulture.DateTimeFormat.ShortDatePattern) of the system instead of hardcoded "d/m/yyyy". As noted in "Format Cells" menu in Excel, date formats that begin with an asterisk (*) respond to changes in regional date and time settings that are specified for the operating system.

Any follow up on this issue?

Hi,
You are probably right this could be improved, however I don't think anybody is working on it.
A solution could be something like:

add a new CultureInfo option to the configuration object (default InvariantCulture)
change IWorksheet.GetNumberFormatString to take a CultureInfo parameter
change BuiltinNumberFormat.GetBuiltinNumberFormat() to take a CultureInfo parameter
implement a converter from .NET DateTimeFormat patterns to Excel NumberFormat patterns
determine which built-in formats respond to changes in regional date and time settings, then
check for these formats in GetBuiltinNumberFormat and return the converted format string


Make Clicking The White On "OFF" State Work (Possibly disable dragging)

OK so tested on my own and also here: http://www.bootstrap-switch.org/examples.html
When the button is on OFF state and you click on the white side it won't be ON only if the checkbox is ON state clicking on white works. This is so annoying sometimes you keep clicking and damn you must click on OFF?

@voidale i am unable to reproduce the bug. what i do so far is:

click on label (white side)
the switch always change state

can you please tell me which browser, browser version and OS are you using?

@LostCrew  this is super weird, I used google chrome, It didn't work but all of a sudden it did work. Must be something that can cause this issue but I can't really tell what it was.

@voidale no problem, feel free to reopen if it happens again.


[Enum Request] Table "Document Entry" - Document Type should be an enum

...we need to extend it.

This field cannot be enum, as it used for different document types. Please use integer.


Avoid creating stage pod using controller

All functionality is described here - https://github.com/fusor/mig-controller/issues/330

When annotating pods, we no longer need to skip stage pods since they no longer exist on the source cluster.  https://github.com/fusor/mig-controller/blob/3b51d5e4bf59c0d22a7ed7b38c3cc73b24cfe354/pkg/controller/migmigration/annotation.go#L211

Please ensure that the Task unit tests have been updated.

Dependent on: https://github.com/fusor/openshift-migration-plugin/pull/37


a simple UI thing

What happened?
when changing view from large display to small display , the name of the nft should also change
Please reproduce in steps
visit any profile and change the size
Expected Behavior
I've expected ...
What browsers are you seeing the problem on?
Chrome
Are you logged in?
Yes
At which address did you encounter bug?
nft.kodadot.xyz / kodadot.xyz
Screenshots

Relevant log output
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

when changing view from large display to small display , the name of the nft should also change

I don't get it or I'm over-worked today.
If you can be more verbose it would help me to understand what you mean.

@yangwao
the problem is when we use the option to show the nfts in small size
the name of the artwork stays big, and that makes it impossible to read as shown in this pic

.
the text showing the name of the nft should also scale down

@yangwao do u understand it now ? Or should i make a draft using Photoshop?
And  can u change the name of the issue to '' scaling of text '' ?

@yangwao do u understand it now ? Or should i make a draft using Photoshop?
And  can u change the name of the issue to '' scaling of text '' ?

Yeah more descriptive issue name way better.
Ah I got it. So text should be smaller on smaller layout
@roiLeo may check on that 👀

@yangwao i'm learning lol ^^

@yangwao No more bounties ? lol im no where near ''funded'' for a new laptop 😂

@yangwao this problem still exist on: https://beta.kodadot.xyz/

@yangwao this problem still exist on: https://beta.kodadot.xyz/

This will be added soon 👍

great ^^


Integration Tests - Bug: Assert hit! Tried to dispose DisposableManager twice! (times x-1)

If I do some Integration tests with the Unity Test Tools in combimation with Zenject...
After all (or selected) tests passed successfully Zenject throws the following exception... (times x where x is amaount of tests run - 1)
ZenjectException: Assert hit! Tried to dispose DisposableManager twice!
ModestTree.Assert.Throw (System.String message) (at Assets/Zenject/Source/Internal/Assert.cs:245)
ModestTree.Assert.That (Boolean condition, System.String message, System.Object[] parameters) (at Assets/Zenject/Source/Internal/Assert.cs:233)
Zenject.DisposableManager.Dispose () (at Assets/Zenject/Source/Runtime/DisposableManager.cs:52)
Zenject.MonoKernel.OnApplicationQuit () (at Assets/Zenject/Source/Runtime/Kernels/MonoKernel.cs:71)

Can you share the tests you're using to reproduce this?  I'm not seeing this issue.  The sample project for example, runs all of its integration tests fine

Yes I can upload it tomorrow.
Basicly, For every test a new Scene Context is initialized because I saw that all single bound classes are shared between the tests otherwise. And I don't want that.
I have a Script "BeforeTest" MonoBehaviour component which is attached to the Integration Test and which will initialize the context before each test.

I'm assuming this is fixed but if not let me know here


The (ambiguous) app's responsibility

In the paragraph there's ambiguity as it relates to the notion of "the app"

The blazor.server.js script establishes the client connection. It's the app's responsibility to persist and restore app state as required (for example, in the event of a lost network connection). The blazor.server.js script is served from an embedded resource in the ASP.NET Core shared framework.

Would love to know... is it the server-side app's responsibility, or is the blazor.server.js script delivering an application that manages app state?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 542f38c4-1502-f739-372a-80c86c377491
Version Independent ID: 72c2a844-ffb2-4799-2c7a-a7a73d3b6833
Content: ASP.NET Core Blazor hosting models
Content Source: aspnetcore/blazor/hosting-models.md
Product: aspnet-core
Technology: aspnetcore-blazor
GitHub Login: @guardrex
Microsoft Alias: riande

Hello @em-jones ... I'll do some work on the text this week to clarify.
The blazor.server.js script doesn't deliver the app ... it establishes the client connection from the client-side. If the connection is lost, it's the client-side app that must deal with it. By default, a client-side app will attempt to re-establish the connection. If it fails on its own, it offers the user a button to retry. The behaviors are customizable (explained in another topic). For example, you can register a callback for a lost connection and then try to reconnect in your own code with your own logic. Those aspects are explained in the Fundamentals > Additional scenarios topic.
I'll improve the language in the section of this topic. Thanks for writing in about it.

I super appreciate you taking your time to respond and look into this. Especially considering this was on a Saturday. In case there was any doubt, anybody who didn't already think the team behind blazor really cared about this project would likely be really hard to impress. Thanks for your hard work!


Publish-DbaDacPackage doesn't know the parameter -Option

Before submitting a bug report:

[x] Ensure you are able to reproduce it on the latest released version (we release often)
[x] Verified this bug is not already reported in an issue
[x] Verified errors are not related to permissions
[x] Can reproduce in a clean PowerShell session (clean = powershell -NoProfile)

Steps to Reproduce
$options = New-DbaDacOption -Type Dacpac -Action Publish
$options.DeployOptions.DropObjectsNotInSource = $true
Publish-DbaDacPackage -SqlInstance mySqlServer -Database myDatabase -Options $options -Path C:\TEMP\myDatabase.dacpac

Expected Behavior
Deployment of database
Actual Behavior

in English: Parameter name "-Option" wasn't found
Environmental data

PowerShell:
PSVersion                      5.1.16299.666
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.16299.666
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>

SQL Server: Microsoft SQL Server 2017 (RTM-CU11) (KB4462262) - 14.0.3038.14 (X64)   Sep 14 2018 13:53:44   Copyright (C) 2017 Microsoft Corporation  Standard Edition (64-bit) on Windows Server Standard 10.0  (Build 16299: ) (Hypervisor)

Thank you - does it work with -DacOption ? if so, we can update the help

And add a parameter alias

Yes -DacOption seems to work


More types support

This adds support for more kinds of types.

Hey @da-x, thanks for the changes on here! I had just a couple notable points to make but it's still mergeable, so I'll go ahead and do so


Getting nodes from another scene not working!

Godot version:
3.2.2.stable.official
OS/device including version:
Windows 10 Home 1909
Issue description:
I can't get a node from another scene, get_node("/root/node") and get_tree().get_root().get_node("node") don't work.

Steps to reproduce:
Create a scene. Create another scene with script. Write in _ready() get_node("/root/node")orget_tree().get_root().get_node("node")`.
Minimal reproduction project:
get node from other scene bug.zip
Start the Node2D scene.

I suspect this might be a slight misunderstanding / user error. Mind if you attach a minimal reproduction project?

A bit complicated, isn't it? I think this should be possible.

Docs could probably mention that you can't access the new scene immediately.
btw related: https://github.com/godotengine/godot-proposals/issues/921


Add test for nested batch accumulator

cc @allenlavoie
Currently investigating why fails.

What's the error / stack trace?

What's the error / stack trace?

AssertionError: Tuples differ: (2,) != ()

First tuple contains 1 additional elements.
First extra element 0:
2

- (2,)
+ () : Shape mismatch: expected (2,), got () with contents -0.16209069.

The problem is with using tape.gradient afterwards. Results are as expected with acc.jvp (tf.Tensor([-0.0841471  -0.16829419], shape=(2,), dtype=float32)

A scalar seems like the right shape to me, since it's a scalar (primals) passed as the second argument to tape.gradient. Note that tape.gradient(y, x) is equivalent to tape.gradient(tf.reduce_sum(y), x) (where the reduce_sum is implicitly watched on the tape). If you want a full Jacobian there's another method for that.

So the jvp function for the op runs, but something is passing a scalar? It could be related to the tf.function special casing for forwardprop. Maybe this placeholder? https://github.com/tensorflow/tensorflow/blob/247e9bd050e68fa4b055fe6c99144def3fde4e81/tensorflow/python/eager/function.py#L1005
To be clear the inner ForwardAccumulator is making a tf.function, and so the outer accumulator runs forwardprop on that function rather than op-by-op. But the test looks fine to me (until the assertions at least; is the whole thing linear in the tangent?).

@abhichou4 Can you please address Ubuntu Sanity errors? Thanks!


Update to allow native Apple Silicon

This addresses #12, requires dependencies update and deployment target update, since now ARM64 simulator exclude should not be required

Hi @olejnjak! Thank you for bringing this to our attention. The reason we excluded the arm64 architecture for simulator initially was because the Google IMA SDK did not support it. Since then, it looks as though M1 support was released in v3.16.3 on 4/19/2022. I am trying to confirm if the Google IMA tvOS SDK also supports the M1. If so, we will be able to drop our exclusion. Thank you for your patience and we will have an update for you soon!

Hellow @nidhik thanks for response, it's been two weeks, any news? :-)

Hi @olejnjak! Sorry for the delay on this. I have been testing using my branch here: https://github.com/muxinc/mux-stats-google-ima/pull/14. But for some reason, the Example is still not building on the M1 mac. It continues to fail with the following, implying that the arm64 architecture for the simulator is not present.
ld: building for iOS Simulator, but linking in dylib built for iOS, file '/Users/echong/mux-repos/mux-stats-google-ima/Example/Pods/GoogleAds-IMA-iOS-SDK/GoogleInteractiveMediaAds.framework/GoogleInteractiveMediaAds' for architecture arm64

When you tested out your fork, were you able to run on an M1 simulator? Did you have to do anything additional besides removing the exclusions in the podspec?

Well your branch does not have updated pods, it still uses Google IMA 3.13.0, according to your Podfile.lock
https://github.com/muxinc/mux-stats-google-ima/blob/961f90bea4f51ad89e1454fad6430b0440b29d2f/Example/Podfile.lock#L3

@olejnjak you're totally right, once I pulled in v3.16.3 the build worked. Working through the Apple TV test next

@olejnjak it doesn't look like the Google IMA tvOS SDK has been updated to support the M1 macs. We pulled in the latest v4.6.1 and it was still a .framework (rather than a .xcframework) and building it resulted in an error about trying to pull in a dylib built for the device rather than the tvOS simulator.
So I think I'll remove the exclusion for iOS and keep it for tvOS. Does that sound good to you?

Yes.

This was released in v0.8.0


Update README.md

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 45.06%. Comparing base (b20edc1) to head (bda727d).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #112   +/-   ##
=======================================
  Coverage   45.06%   45.06%
=======================================
  Files          39       39
  Lines        1782     1782
=======================================
  Hits          803      803
  Misses        979      979

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Commit https://github.com/HumanSignal/infra/commit/9001e209236c8a52637a573c4feccc4ad3052025 is created.

Commit https://github.com/HumanSignal/infra/commit/b9e4c8c240c832147f1a054cd968bc8123a4d395 is created.


Add SepReader.Cols.ParseToArray/ToStrings/ToStringsArray, Improve Debuggability

Verify Parse and similar works on .NET 8 where string : ISpanParsable, but also introduce ToStrings/ToStringsArray for convenience
SepReader DebuggerDisplay will show source for reader via internal SepReader.Info
SepReader.Row adds DebuggerTypeProxy to show each col when expanding in debug view

Codecov Report
Patch coverage is 100.00% of modified lines.

Files Changed
Coverage

src/Sep/SepReader.Col.cs
100.00%

src/Sep/SepReader.Cols.cs
100.00%

src/Sep/SepReader.Row.cs
100.00%

src/Sep/SepReader.cs
100.00%

src/Sep/SepReaderExtensions.cs
100.00%

:loudspeaker: Thoughts on this report? Let us know!.

Example in Visual Studio

Codecov Report
Patch coverage is 100.00% of modified lines.

Files Changed
Coverage

src/Sep/SepReader.Col.cs
100.00%

src/Sep/SepReader.Cols.cs
100.00%

src/Sep/SepReader.Row.cs
100.00%

src/Sep/SepReader.cs
100.00%

src/Sep/SepReaderExtensions.cs
100.00%

:loudspeaker: Thoughts on this report? Let us know!.


allow arbitrary metric prefixes (v0.16.x)

(cherry picked from commit 69360e984884902e60c0ffafe4c73a18702896ad)

Why do you need this?

This commit fixes the metric prefix issue with Stackdriver Monitoring (https://github.com/census-instrumentation/opencensus-java/issues/1433). The bug is just released in v0.16.0 so we think it makes sense to include it in this patch release.
/cc @mayurkale22

It is only in the exporter so I am not that concerned.


fix(ngTransclude): fix case where ngTransclude attribute value equals its key

Some preprocessors such as Jade will automatically provide a value for an attribute
rather than leave it empty. E.g. <div ng-transclude="ng-transclude">.
In these situations we still want to use the default transclusion slot.
Closes #12934

LGTM

:+1: thanks!


Add support for description for devices settings policy

devices settings policy

This is dependent on pr in cloudflare-go
https://github.com/cloudflare/cloudflare-go/pull/1294

acceptance tests all passing
$ TEST="./internal/sdkv2provider" TESTARGS='-run "^(TestAccCloudflareDeviceSettingsPolicy_|TestAccCloudflareTeamsAccountConfigurationBasic)" -count 1' make testacc
TF_ACC=1 go test ./internal/sdkv2provider -v -run "^TestAccCloudflareTeamsAccountConfigurationBasic" -count 1 -timeout 120m -parallel 1
=== RUN   TestAccCloudflareDeviceSettingsPolicy_Create
--- PASS: TestAccCloudflareDeviceSettingsPolicy_Create (37.76s)
=== RUN   TestAccCloudflareTeamsAccountConfigurationBasic
--- PASS: TestAccCloudflareTeamsAccountConfigurationBasic (20.09s)
PASS
ok  	github.com/cloudflare/terraform-provider-cloudflare/internal/sdkv2provider	57.854s


Discord link is invalid

https://discord.gg/cyK3Hjm is invalid... You should replace with a permanent link.

The repo has moved to https://github.com/shardlab/discordrb


npm audit failing on lodash vulnerability

Since this week, npm audit is flagging a vulnerability in lodash.
https://npmjs.com/advisories/1523
Lodash has been asked by many to merge PR's to fix the issue, but seems to be ignoring and has been for a long time. Hopes for a fix seems slim, based on them closing the issue:
https://github.com/lodash/lodash/issues/4744
Perhaps it's time to consider dropping lodash from styled-components? Or to fork and fix?
In our case this is now flagged by security and could become a matter of having to refactor our entire portal to drop styled-components.

lodash is only a dependency of babel-plugin-styled-components, so it never ends up in your application if only styled-components is used. I don't think babel-plugin-styled-components actually needs lodash and I'm not sure when it was added, but feel free to open an issue over on the other repository: https://github.com/styled-components/babel-plugin-styled-components
But that being said, this doesn't affect your application and you'll need to always research first where any potential vulnerabilities come from. This one is even a "prototype pollution" vulnerability, which means that in all likelihood it has no effect on your application at all, since we're not talking about anything that'd flow into it. Furthermore it's only in a rarely used utility: zipObjectDeep

@kitten we're seeing security vulnerabilities found by the BlackDuck due to babel-plugin-styled-components being defined as regular dependency of styled-components and as a result we've got lodash bundled in our app, shouldn't it rather be a dev dependency so it does not end up in the library bundle together with lodash?


Ingest run_constraints on other llvm packages from repodata patches

Closes #141 (hopefully)
Still a draft because I'm not sure about some things (will comment in the diff).

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

Only https://github.com/conda-forge/llvmdev-feedstock/pull/142/commits/9217917b29f611fd5d78be59992a998dc9e554c7 is worth reviewing...

I'll need to resolve the merge conflicts at the cli. Will do later.


drop php 5.3-5.5 supporting

http://php.net/supported-versions.php

Hi,
Since you are using declare(strict_types = 1);, you already support only PHP 7.x.
When I tried to compose require the plugin on PHP 5.6, it resulted in a list of errors:
Warning: Unsupported declare 'strict_types' in D:\projects\cms-composer\cms\vendor\octolab\cleaner\src\Plugin.php on line 3

Warning: Unsupported declare 'strict_types' in D:\projects\cms-composer\cms\vendor\octolab\cleaner\src\Config\PluginConfig.php on line 3

Warning: Unsupported declare 'strict_types' in D:\projects\cms-composer\cms\vendor\octolab\cleaner\src\Util\FileCleaner.php on line 3

Warning: Unsupported declare 'strict_types' in D:\projects\cms-composer\cms\vendor\octolab\cleaner\src\Util\CleanerInterface.php on line 1

Warning: Unsupported declare 'strict_types' in D:\projects\cms-composer\cms\vendor\octolab\cleaner\src\Util\WeightMatcher.php on line 3

Warning: Unsupported declare 'strict_types' in D:\projects\cms-composer\cms\vendor\octolab\cleaner\src\Util\MatcherInterface.php on line 1

Warning: Unsupported declare 'strict_types' in D:\projects\cms-composer\cms\vendor\octolab\cleaner\src\Util\CategoryNormalizer.php on line 3

Warning: Unsupported declare 'strict_types' in D:\projects\cms-composer\cms\vendor\octolab\cleaner\src\Util\NormalizerInterface.php on line 1
Plugin installation failed, rolling back
  - Removing octolab/cleaner (dev-master)

Could you make your plugin compatible with at least PHP 5.6, please?

fixed by f9625224910e2dc0cab3a283fd639bba18624025


feat: find all import paths from source

Summary
Adding function that returns absolute paths of files imported stemming from a source file. The path resolver used is ember-cli-addon-aware-resolver
Test
Added unit test

Thanks @spham92 this is looking good. All my feedback is polish at this point.

released as 1.2.0


.pic

hello.
I'm not a programmer, how do you start this program?
I have windows 10.

Try https://github.com/ogamespec/PicDecoder/releases/tag/picdecoder-10

I'm sorry I don't understand much about programming.
The program PicDecord 10 does not open the .pic files of the pc game "Darklands", they are very old files and maybe that is why (1992).

I only checked on Magic the Gathering, I haven't looked at the others.

Ok, thanks
Could you make another one for darklands?

I don't have an opportunity right now, maybe someone else can. Well, or you learn programming and do it - it will be a good motivation.

ok. Thanks


✨Provide a script to take a snapshot of KubeStellar deployment

Summary
Provide a new script to take a snapshot of KubeStellar deployment: kubestellar-snapshot.sh
Usage: ./kubestellar-snapshot.sh [--kubeconfig|-K <filename>] [--context|-C <name>] [--verbose|-V] [-X]
--kubeconfig|-K <filename> use the specified kubeconfig
--context|-C <name>        use the specified context
--logs|-L                  save the logs of the pods
--yaml|-Y                  save the YAML of the resources
--verbose|-V               output extra information
--version|-v               print out the script version
-X                         enable verbose execution of the script for debugging

Example of usage after running test/e2c/run_test.sh:
$ ./kubestellar-snapshot.sh -V -Y -L
KubeStellar Snapshot v0.10

Script run on 2024-11-07_23:21:53
Checking script dependencies:
✔ kubectl version v1.31.1 at /usr/local/bin/kubectl
✔ helm version v3.16.1 at /usr/sbin/helm
✔ jq version jq-1.6 at /usr/bin/jq
Using kubeconfig(s): /home/vagrant/.kube/config
Using contexts(s): cluster1, cluster2, its1, kind-kubeflex, wds1

KubeStellar:
- Helm chart ks-core (v0.1.0) in context kind-kubeflex

KubeFlex:
- kubeflex-system namespace in context kind-kubeflex
- controller-manager: version=0.7.1, pod=kubeflex-controller-manager-54c7fcb78c-gmgr4, status=Running
- postgres-postgresql-0: pod=postgres-postgresql-0, status=Running

Control Planes:
- its1: type=vcluster, pch=its, context=kind-kubeflex, namespace=its1-system
  - Post Create Hook: pod=its-vsmt2, status=Succeeded
  - Status addon: pod=addon-status-controller-d8864f4fd-m57xq-x-open-clust-fbedb9af6a, status=Running
- wds1: type=k8s, pch=wds, context=kind-kubeflex, namespace=wds1-system
  - KubeStellar controller: version=bcda2ef6a, pod=kubestellar-controller-manager-85c5c954db-lv5lm namespace=wds1-system, status=Running
  - Transport controller: version=bcda2ef6a, pod=transport-controller-697d88bf55-b4ffg namespace=wds1-system, status=Running

Managed Clusters:
- cluster1 in its1: accepted=True, joined=True, available=True, synced=True
  {
    "location-group": "edge",
    "name": "cluster1",
    "region": "east"
  }
- cluster2 in its1: accepted=True, joined=True, available=True, synced=True
  {
    "location-group": "edge",
    "name": "cluster2",
    "region": "west"
  }

Binding Policies:
- nginx-bindingpolicy in control plane wds1
  {
    "clusterSelectors": [
      {
        "matchLabels": {
          "location-group": "edge"
        }
      }
    ],
    "downsync": [
      {
        "objectSelectors": [
          {
            "matchLabels": {
              "app.kubernetes.io/name": "nginx"
            }
          }
        ]
      }
    ]
  }

Manifest Works:
- nginx-bindingpolicy-wds1-0 in cp=its1, namespace=cluster1: wds1 --> nginx-bindingpolicy --> cluster1
  {
    "apiVersion": "v1",
    "kind": "Namespace",
    "name": "nginx"
  }
- nginx-bindingpolicy-wds1-1 in cp=its1, namespace=cluster1: wds1 --> nginx-bindingpolicy --> cluster1
  {
    "apiVersion": "apps/v1",
    "kind": "Deployment",
    "name": "nginx-deployment"
  }
- nginx-bindingpolicy-wds1-0 in cp=its1, namespace=cluster2: wds1 --> nginx-bindingpolicy --> cluster2
  {
    "apiVersion": "v1",
    "kind": "Namespace",
    "name": "nginx"
  }
- nginx-bindingpolicy-wds1-1 in cp=its1, namespace=cluster2: wds1 --> nginx-bindingpolicy --> cluster2
  {
    "apiVersion": "apps/v1",
    "kind": "Deployment",
    "name": "nginx-deployment"
  }

Saving logs and/or YAML to ./kubestellar-snapshot.tar.gz

Here are the file saved:
$ tree /tmp/kubestellar-snapshot/
/tmp/kubestellar-snapshot/
├── its1
│   ├── cp.yaml
│   ├── its-job-clusteradm.log
│   ├── its-job-status-addon.log
│   ├── its-job.yaml
│   ├── managed-clusters
│   │   ├── cluster1.yaml
│   │   └── cluster2.yaml
│   ├── manifest-works
│   │   ├── cluster1
│   │   │   ├── nginx-bindingpolicy-wds1-0.yaml
│   │   │   └── nginx-bindingpolicy-wds1-1.yaml
│   │   └── cluster2
│   │       ├── nginx-bindingpolicy-wds1-0.yaml
│   │       └── nginx-bindingpolicy-wds1-1.yaml
│   ├── status-addon.log
│   └── status-addon.yaml
├── kubeflex
│   ├── kubeflex-controller.log
│   └── postgresql.log
├── kubestellar
│   ├── its-pch.yaml
│   └── wds-pch.yaml
└── wds1
    ├── binding-politcies
    │   └── nginx-bindingpolicy.yaml
    ├── cp.yaml
    ├── kubestellar-controller.log
    ├── kubestellar-controller.yaml
    ├── transport-controller.log
    └── transport-controller.yam

Related issue(s)
#2508
Fixes #

Some comments on what is exhibited in the opening comment here.

Some of the format is almost YAML. Should it be actual YAML? Remember that YAML is flexible. For example, an array of strings can be represented by [ "string1", "string2", "and", "so", "on" ].
Other parts of the output are almost tabular. I am also a big fan of tabular output. For example, the dump of ManifestWork contents could, I think, be very nicely represented by a table with columns for: ITS name, ManifestWork name, WEC name (equals namespace of the ManifestWork), WDS name, Binding name (equals BindingPolicy name but I think we should take Bindings as central), Kind, apiVersion, namespace of the workload object, name of the workload object.
I really think that Binding objects deserve at least as much attention as ManifestWork objects. The Binding objects are closer to the user and provide an important bit of feedback (without entanglement with other concerns): how the predicates in the BindingPolicy matched up against the available WECs and workload objects.
OTOH, for "must gather" functionality, I think we want comprehensive JSON or YAML (and I would prefer JSON) rather than stuff that is pretty and shortened for easy consumption. kubectl get -o json --show-managed-fields.

mspreitz@mjs13 kubestellar % echo $PWD
/Users/mspreitz/go/src/github.com/kubestellar/kubestellar

mspreitz@mjs13 kubestellar % tr <<<"$PWD" ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz
/users/mspreitz/go/src/github.com/kubestellar/kubestellar

Adding set -x to my copy of the script reveals where the time is going:
+ contexts=($(k config get-contexts --no-headers -o name))
++ k config get-contexts --no-headers -o name
++ KUBECONFIG=/Users/mspreitz/.kube/config
++ kubectl config get-contexts --no-headers -o name
+ [[ -z /api-prod-dipc-res-ibm-com:6443/4A5349897@W3IDRealm cluster1 cluster2 cp1 default/api-epoc2e-ete14-res-ibm-com:6443/kube:admin<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>ep1 imbs1 imbs2 its1 its2 kind-kubeflex ks-edge-cluster1 ks-edge-cluster2 kscore kscore-stage<EMAIL_ADDRESS>rancher-desktop tsi-kube01/c20o1lbd0mhbu12dbd0g wds1 wds2 ]]
+ echov 'Using contexts(s): '
+ :
+ for context in '"${contexts[@]}"'
+ k --context /api-prod-dipc-res-ibm-com:6443/4A5349897@W3IDRealm get pods -A

There is a long pause at this point. I have a lot of obsolete contexts in my kubeconfig.

After fixing the parameter expansion problem, I tested and got surprisingly empty results.
mspreitz@mjs13 kubestellar % time scripts/kubestellar-snapshot.sh

KubeStellar:
- Helm chart ks-core (v0.1.0) in context kind-kubeflex
  - Secret=sh.helm.release.v1.ks-core.v1

KubeFlex:
- kubeflex-system namespace in context kind-kubeflex
- controller-manager: version=0.7.1, pod=kubeflex-controller-manager-76f8db9767-m5dds, status=running
- postgres-postgresql-0: pod=postgres-postgresql-0, status=running

Control Planes:
- its1: type=vcluster, pch=its, context=kind-kubeflex, namespace=its1-system
  - Post Create Hook: pod=its-j5bjm, ns=its1-system, status=succeeded
  - Status addon: pod=addon-status-controller-d8864f4fd-wl9l5-x-open-clust-a298b3c05d, ns=its1-system, status=running
- wds1: type=k8s, pch=wds, context=kind-kubeflex, namespace=wds1-system
  - KubeStellar controller: version=a6f73c3db, pod=kubestellar-controller-manager-dc95df885-6vccs namespace=wds1-system, status=running
  - Transport controller: version=a6f73c3db, pod=transport-controller-d6994657b-nznhw namespace=wds1-system, status=running

Managed Clusters:

Binding Policies:

Manifest Works:

Work Statuses:
scripts/kubestellar-snapshot.sh  4.78s user 2.57s system 1% cpu 6:40.00 total

But just looking with kubectl finds stuff:
mspreitz@mjs13 kubestellar % kubectl --context wds1 get bindings.control
NAME                  AGE
nginx-bindingpolicy   4h59m

mspreitz@mjs13 kubestellar % kubectl --context its1 get managedclusters
NAME       HUB ACCEPTED   MANAGED CLUSTER URLS                  JOINED   AVAILABLE   AGE
cluster2   true           https://cluster2-control-plane:6443   True     True        4h59m
cluster1   true           https://cluster1-control-plane:6443   True     True        4h59m

mspreitz@mjs13 kubestellar % kubectl --context its1 get manifestworks -A
NAMESPACE   NAME                          AGE
cluster1    nginx-bindingpolicy-wds1-0    4h59m
cluster1    nginx-bindingpolicy-wds1-1    4h59m
cluster2    nginx-bindingpolicy-wds1-0    4h59m
cluster2    nginx-bindingpolicy-wds1-1    4h59m
cluster1    addon-addon-status-deploy-0   4h59m
cluster2    addon-addon-status-deploy-0   4h59m

mspreitz@mjs13 kubestellar % kubectl --context its1 get WorkStatus -A
NAMESPACE   NAME                                                                            AGE
cluster1    edd79b63-cc53-4fbe-9891-b1efe299314c-v1-namespace--nginx                        4h59m
cluster2    c670f1a6-1c8d-4aaa-bf96-eca98bdfa1a7-v1-namespace--nginx                        4h59m
cluster1    0742a339-c2d0-4e92-aee6-23a813b713cc-appsv1-deployment-nginx-nginx-deployment   4h59m
cluster2    cde59121-962e-459f-a8c4-948e510bcc48-appsv1-deployment-nginx-nginx-deployment   4h59m

I did a little more investigating of why the script does not find any managed clusters and so on. Allowing the error messages to show up, I see error messages indicative of completely missing/wrong kubeconfig. I am reminded of a previous PR where you also tried the kubectl --kubeconfig <(command) thing and it completely failed for me. I think that eventually the PR was changed to work a different way.

I tested on my laptop, MacOS 15.1, bash 3.2.57. It took a few minutes before outputting anything at all.
mspreitz@mjs13 kubestellar % scripts/kubestellar-snapshot.sh

KubeStellar:
- Helm chart ks-core (v0.1.0) in context kind-kubeflex
  - Secret=sh.helm.release.v1.ks-core.v1

KubeFlex:
- kubeflex-system namespace in context kind-kubeflex
- controller-manager: version=0.7.1, pod=kubeflex-controller-manager-76f8db9767-m5dds, status=scripts/kubestellar-snapshot.sh: line 100: ${status,,}: bad substitution

mspreitz@mjs13 kubestellar % git rev-parse HEAD
4abed364465eb50bb8e37a9b48fb71a3e149a948

It looks like the script relies on a feature than is not in my bash:
bash-3.2$ echo "${PWD,,}"
bash: ${PWD,,}: bad substitution

I moved to use tr to handle old bash versions

I did a little more investigating of why the script does not find any managed clusters and so on. After editing the script to allow the error messages to show up, I see error messages indicative of completely missing/wrong kubeconfig. I vaguely recall that there was a previous PR where you also tried the kubectl --kubeconfig <(command) thing and it completely failed for me. I think that eventually the PR was changed to work a different way.

The only alternative would be to dump the in-memory kubeconfig contet to a tmp file and use KUBECONFIG env variable to point to it... It's more work, less secure
Can you investigate your source of this specific issue?

FYI, Apple ships an old version of bash because of a copyright(left) dispute with the authors of bash; I do not expect a modern bash from Apple any time soon. While Mac users can install variant software, that is inherently dodgy; I would not recommend that we require this.
(I share your dismay. It means that we can not have nice things like: associative arrays, @Q in parameter expansion.)

Here is a simple demonstration of the problem (producing the same error messages that I got from testing this PR's script after making it not discard the error messages).
bash-3.2$ kubectl --kubeconfig ~/.kube/config get ns
NAME                                  STATUS   AGE
default                               Active   27h
kube-node-lease                       Active   27h
kube-public                           Active   27h
kube-system                           Active   27h
local-path-storage                    Active   27h
nginx                                 Active   27h
open-cluster-management               Active   27h
open-cluster-management-agent         Active   27h
open-cluster-management-agent-addon   Active   27h

bash-3.2$ kubectl --kubeconfig <(cat ~/.kube/config) get ns
E1115 15:37:02.330638    9646 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp <IP_ADDRESS>:8080: connect: connection refused
E1115 15:37:02.331242    9646 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp <IP_ADDRESS>:8080: connect: connection refused
E1115 15:37:02.332225    9646 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp <IP_ADDRESS>:8080: connect: connection refused
E1115 15:37:02.332702    9646 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp <IP_ADDRESS>:8080: connect: connection refused
E1115 15:37:02.333825    9646 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp <IP_ADDRESS>:8080: connect: connection refused
The connection to the server localhost:8080 was refused - did you specify the right host or port?

Not much help here:
bash-3.2$ set -x
bash-3.2$ kubectl -v=10 --kubeconfig <(cat ~/.kube/config) get ns
+ kubectl -v=10 --kubeconfig /dev/fd/63 get ns
++ cat /Users/mspreitz/.kube/config
I1115 15:40:48.578262   10161 loader.go:395] Config loaded from file:  /dev/fd/63
I1115 15:40:48.578488   10161 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json;g=apidiscovery.k8s.io;v=v2beta1;as=APIGroupDiscoveryList,application/json" -H "User-Agent: kubectl1.29.2/v1.29.2 (darwin/arm64) kubernetes/4b8e819" 'http://localhost:8080/api?timeout=32s'
...

This script shows only expected behavior. I infer that the way that the file is read makes the difference.
#!/usr/bin/env bash

set -x

echo Args are "<$@>"
echo arg 1 is "<$1>"
cat $1 > copy-of-1

Here is using it:
bash-3.2$ ./exam.sh ~/.kube/config
+ echo Args are '</Users/mspreitz/.kube/config>'
Args are </Users/mspreitz/.kube/config>
+ echo arg 1 is '</Users/mspreitz/.kube/config>'
arg 1 is </Users/mspreitz/.kube/config>
+ cat /Users/mspreitz/.kube/config

bash-3.2$ ls -l copy-of-1
-rw-r--r--@ 1 mspreitz  staff  77170 Nov 15 15:49 copy-of-1

bash-3.2$ ls -l ~/.kube/config
-rw-------@ 1 mspreitz  staff  77170 Nov 14 11:51 /Users/mspreitz/.kube/config

bash-3.2$ diff copy-of-1 ~/.kube/config
bash-3.2$

@MikeSpreitzer what is your kubectl version?
 kubectl version
Client Version: v1.31.1
Kustomize Version: v5.4.2
Server Version: v1.31.0

I will investigate the --kubeconfig <(command) issue. I also wonder (without having tried to study the script in detail) why it is necessary, when the user already has a kubeconfig file.

The script does not require kflxe or contexts for any CP.
Once it locates KubeStellar Helm chart, it will track down the CPs and other resouces, hence the need to deal with in memory kubeconfigs

Here is a simple demonstration of the problem (producing the same error messages that I got from testing this PR's script after making it not discard the error messages).
bash-3.2$ kubectl --kubeconfig ~/.kube/config get ns
NAME                                  STATUS   AGE
default                               Active   27h
kube-node-lease                       Active   27h
kube-public                           Active   27h
kube-system                           Active   27h
local-path-storage                    Active   27h
nginx                                 Active   27h
open-cluster-management               Active   27h
open-cluster-management-agent         Active   27h
open-cluster-management-agent-addon   Active   27h

bash-3.2$ kubectl --kubeconfig <(cat ~/.kube/config) get ns
E1115 15:37:02.330638    9646 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp <IP_ADDRESS>:8080: connect: connection refused
E1115 15:37:02.331242    9646 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp <IP_ADDRESS>:8080: connect: connection refused
E1115 15:37:02.332225    9646 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp <IP_ADDRESS>:8080: connect: connection refused
E1115 15:37:02.332702    9646 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp <IP_ADDRESS>:8080: connect: connection refused
E1115 15:37:02.333825    9646 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp <IP_ADDRESS>:8080: connect: connection refused
The connection to the server localhost:8080 was refused - did you specify the right host or port?

What about this:
kubectl --kubeconfig <(cat ~/.kube/config) config view --flatten > /tmp/dump.txt

diff /tmp/dump.txt ~/.kube/config

for me there is no difference observed

mspreitz@mjs13 kubestellar % kubectl version --client
Client Version: v1.29.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3

mspreitz@mjs13 kubestellar % kubectl --kubeconfig <(cat ~/.kube/config) config view --flatten > /tmp/dump.txt

mspreitz@mjs13 kubestellar % wc /tmp/dump.txt
       7      14     106 /tmp/dump.txt

mspreitz@mjs13 kubestellar % wc ~/.kube/config
     539    1067   77170 /Users/mspreitz/.kube/config

Here is a demonstration of keeping temp files private to the user. (Remember that the user's kubeconfig, which is (at best) private to the user, gives access to anything this script finds.)
bash-3.2$ umask 077

bash-3.2$ mktemp -p . -d
./tmp.c3ifC8QXym

bash-3.2$ date > tmp.c3ifC8QXym/date

bash-3.2$ ls -l
total 0
drwx------  3 mspreitz  staff  96 Nov 16 14:04 tmp.c3ifC8QXym

bash-3.2$ ls -l tmp.c3ifC8QXym/
total 8
-rw-------  1 mspreitz  staff  29 Nov 16 14:04 date

It tested the current rev of this PR, commit bb2b5843a50b9891047f83debb645ac6f108ccc3.
I first built a system with test/e2e/run-test.sh --released. Then I ran the snapshot script. I have attached a typscript of running the script. Unless I missed something, it worked as intended.
snap-trial-4.log

This script prints the .status.phase of every WorkStatus object, regardless of whether there is a phase defined in the schema of the relevant .status.


Users can't run "nix-collect-garbage -d" any more

I used to be able to run nix-collect-garbage -d as a normal user, but now I get this error message when I try:
[simons@latitude:~]$ nix-collect-garbage -d
removing old generations of profile /nix/var/nix/profiles/default
error: opening lock file ‘/nix/var/nix/profiles/default.lock’: Permission denied
Died at /run/current-system/sw/bin/nix-collect-garbage line 43.

I'm running NixOS 15.05pre57108.96d6344 (Dingo).

Works for me on the latest master....

@wkennington, I don't think so, I believe the system is set up properly. I guess what puzzles me is that the command aborts after the error. When I run the same command on a different machine (some version of NixOS), I get this:
[simons@work:~]$ nix-collect-garbage -d
removing old generations of profile /nix/var/nix/profiles/per-user/root/channels
error: opening lock file ‘/nix/var/nix/profiles/per-user/root/channels.lock’: Permission denied
removing old generations of profile /nix/var/nix/profiles/per-user/root/profile
error: opening lock file ‘/nix/var/nix/profiles/per-user/root/profile.lock’: Permission denied
removing old generations of profile /nix/var/nix/profiles/per-user/simons/profile
removing generation 7
removing generation 8
removing generation 9
removing generation 10
removing generation 11
removing generation 12
removing generation 13
removing generation 14
removing generation 15
removing old generations of profile /nix/var/nix/profiles/system
error: opening lock file ‘/nix/var/nix/profiles/system.lock’: Permission denied
finding garbage collector roots...
removing stale link from ‘/nix/var/nix/gcroots/auto/5ms46abnr1kj79x2dvx159df44w6xdz3’ to ‘/run/user/0/nix-build.6QJlu8/derivation’
removing stale link from ‘/nix/var/nix/gcroots/auto/lz3cajjnh63c21ga05mql0yywnp08fh1’ to ‘/run/user/1000/nix-build.vWnbGF/derivation’
removing stale link from ‘/nix/var/nix/gcroots/auto/wg7yhh6lrrdkbxcr14s421jdap1i5773’ to ‘/run/user/1000/nix-prefetch-url.Ua7BjZ/urls’
deleting garbage...

In other words, it's perfectly normal for the process to get that permission denied response, but its un-normal for it to abort the entire garbage collection process because of that.

Wild guess: different version of something (perl? nix?) behaves differently about failure states when removing generations ("-d"). BTW, it's probably more of a nix issue than nixpkgs/nixos.

Also noticed this today though didn't think much of it.

The issue has disappeared on my machine. I contemplated deleting the default profile, because I don't use it anyway, and so I ended up listing its contents and found some ancient package installed in there, which I probably added months ago for some temporary purposes, but then I forget to remove it again. Anyway, I removed that  package from the profile -- which is now empty -- and since then garbage collection works fine, like it used to. Now what?

Cannot be easily re-produced, so there's probably no point in keeping this issue open without anyone actually working on it.

BTW, I get the same error message, but that's expected behaviour since I don't have write access to the default profile.

@edolstra, you also get the Died at /run/current-system/sw/bin/nix-collect-garbage line 43. part?

Yes.

Interesting. I don't see that error in Nix 1.8 from nixos-unstable:
[simons@latitude:~]$ nix-collect-garbage -d
removing old generations of profile /nix/var/nix/profiles/default
error: opening lock file ‘/nix/var/nix/profiles/default.lock’: Permission denied
removing old generations of profile /nix/var/nix/profiles/per-user/root/channels
error: opening lock file ‘/nix/var/nix/profiles/per-user/root/channels.lock’: Permission denied
removing old generations of profile /nix/var/nix/profiles/per-user/root/profile
error: opening lock file ‘/nix/var/nix/profiles/per-user/root/profile.lock’: Permission denied
removing old generations of profile /nix/var/nix/profiles/per-user/simons/profile
removing old generations of profile /nix/var/nix/profiles/system
error: opening lock file ‘/nix/var/nix/profiles/system.lock’: Permission denied
finding garbage collector roots...
deleting garbage...
deleting ‘/nix/store/rrn0f1g0m51yk7plbmfnw7hmqxlz2sxr-python2.7-testtools-0.9.34.drv’
deleting ‘/nix/store/wlf92r42y8kp8q7nkahd2lyb8bykggcb-ecospat_1.1.tar.gz’
[...]

still problematic?

Since then it was rewritten from perl to c++ and some issues in that were fixed already. I've never noticed this problem so I can't say. (I don't use -d, prune links manually, and 2> /dev/null.)

I still get transient permission denied errors like the ones above. Interestingly enough, the errors simply go away when I re-run the command, so it's no big deal, but, yes, it does still happen.

Works fine here on NixOS 16.03 (and master):
user $ nix-collect-garbage -d
....
deleting ‘/nix/store/39a4qg7panfzf0358wap5by647f7d2g9-tornado-4.2.1.tar.gz’
deleting ‘/nix/store/7mylnkgyrssi9rj5ccv6s7ai8kf9rfl8-xsession.drv’
deleting ‘/nix/store/trash’
deleting unused links...
note: currently hard linking saves -64.49 MiB
1569 store paths deleted, 2023.29 MiB freed
[successful exit status]

(I don't remember if I've ever had transient errors.)
I think we should close this issue for now.

This happened to me recently, after running nix-collect-garbage -d with sudo -i.

Same

Same @asmodehn

still happens here (darwin) with :
deleting '/nix/store/trash'
12428 store paths deleted, 664.94 MiB freed
error: cannot unlink '/nix/store/trash/4mnbcnp5fb8495pjlzwrfy24c9yk122n-vscode/extension-ms-vscode-csharp-1.21.5/share/vscode/extensions/ms-vscode.csharp/.omnisharp/1.34.5/license.md': Permission denied

I guess it happens when nix/store/... is mutated outside of nix. vscode extensions are good candidates to replicate this error I think.

I guess it happens when nix/store/... is mutated outside of nix. vscode extensions are good candidates to replicate this error I think.

You're not supposed to ever do this.

I guess it happens when nix/store/... is mutated outside of nix. vscode extensions are good candidates to replicate this error I think.

You're not supposed to ever do this.

I did'nt do it directly I guess it was done by another process.

I'm facing similar problem after uninstalling vscode.
❯ nix-collect-garbage -d
removing old generations of profile /nix/var/nix/profiles/per-user/guilhermefilho/home-manager
removing generation 16
removing old generations of profile /nix/var/nix/profiles/per-user/guilhermefilho/profile
removing old generations of profile /nix/var/nix/profiles/per-user/guilhermefilho/channels
removing old generations of profile /nix/var/nix/profiles/system
removing generation 25
finding garbage collector roots...
removing stale link from '/nix/var/nix/gcroots/auto/007j1mr5szp3cp70chg757k8abnfq4y3' to '/private/var/folders/8x/tmvwknqx1679s05zf5zkdl3m0000gp/T/nix-build-5659-0/result'
removing stale link from '/nix/var/nix/gcroots/auto/lvwb7ia80vl56jf7pfk8kn2j4s8dycw6' to '/private/var/folders/8x/tmvwknqx1679s05zf5zkdl3m0000gp/T/nix-build-5139-0/result'
0 store paths deleted, 0.00 MiB freed
error: cannot unlink '/nix/store/trash/j5yl1h6inih9lm94bsplz8izilvgsyi3-vscode-1.45.0/Applications/Visual Studio Code.app/Contents/CodeResources': Permission denied

Any idea?

This issue comes back often... maybe reopen the issue as nix-collect-garbage having to have better error messages that offer actionable suggestions to the user?

I suspect that vscode was modified outside of nix but I'm not sure.
@fare I would reopen this one. As you said, maybe just give the user a hint on how to solve the issue without creating more problems (like running things as sudo) is enough.

I have a PR base on this project related to this issue.
This package brings a wrapper over vscode that is built to run in a nix-shell of a project.
It uses a unique data-dir per project, so everything is compartmentalized.
It gives the ability to:

set settings (which disable auto-updates).
declaratively define extensions and mutable extensions(that won't live in the nix store).


Update VisualStudio.gitignore

Reasons for making this change:
Visual Studio Code is part of the family now and creates its own working folder, for supporting multiple root spaces since version 1.1.8. It would be nice to keep these files out of Git.
Links to documentation supporting these rule changes:
https://code.visualstudio.com/docs/editor/multi-root-workspaces#_settings

Thanks, but please see Global/VisualStudioCode.gitignore for VSCode...

Ah, I found it was odd there was nothing for it yet. Thanks!
But how does this work? I mean, it isn't very unlikely that Visual Studio (full) and Code are used in the same project right? So, wouldn't it make sense to incorporate it in the VisualStudio.gitignore as well?

So, wouldn't it make sense to incorporate it in the VisualStudio.gitignore as well?

VisualStudio.gitignore has accrued a rich history of rules due to the myriad of technologies and extensions that it has supported over the years. I would prefer to keep VisualStudioCode.gitigore focused on just what VSCode needs and supports.
Also please note that there are a number of files under the .vscode directory that can be safely versioned.

Fair enough! Thanks!