Future of candle-transformers / long-term plans

Hi there!
Apologies for the vague issue title, but I was struggling to think of one that conveyed my sentiments.
I'm the primary maintainer of rustformers/llm, which implements several common LLM architectures atop GGML as an easy-to-use library. In some sense, it can be considered a robust, consistent and extensible Rust library-ification of llama.cpp with support for other architectures.
Recently, I've been considering winding down development on it in favour of encouraging people to use candle-transformers instead, because Candle can evolve faster than we do, supports more models, and isn't held back by the free time I/our contributors have.
My initial plan was to get llm back up to speed with the latest in llama.cpp and elsewhere, and then add support for other backends, so that Candle could be a secondary backend and (likely) become the primary backend in future. However, chasing a moving target is quite difficult, and candle-transformers already covers much of the same ground.
With that in mind, people in that discussion have raised a few issues around switching to candle-transformers and think llm is still relevant. I think it'd be simpler for the ecosystem if there was a single place for LLMs, but the concerns raised have made it harder to provide a straightforward recommendation to switch to candle-transformers.
So, here are my questions:

What's the long-term plan for candle-transformers? Vague question, I know, but will it live in this repo forever? Will it become an ecosystem unto itself like Python transformers?
Will the models share a unified interface to make it easy to swap between architectures?
Similarly, are there any plans to offer a higher-level abstraction for model inference? I'm not completely happy with our own interface, but it makes common tasks (prompt-feeding, streamed inference) easy, and harder tasks (custom inference, custom models) doable: https://github.com/rustformers/llm/blob/main/crates/llm/examples/inference.rs
Are there plans for Metal acceleration support in the near-future? I'm guessing this is tracked in #313, but I'm not sure what the timeline of that is.
Finally, do you think a library like llm is necessary? My gut feeling is that Candle (and candle-transformers) could grow to cover all of its territory, but there may still be some value in custom high-level abstractions and support for other backends. That would be obviated by Candle developing its own high-level abstractions and increasing its backend support, though.

Sorry about the wall of questions, but your input is hugely useful in figuring out our own direction. Knowing what you have planned will clarify some of the unknowns for us and let us figure out what to do next.
Thanks in advance! ❤

Another llm contributor here - one question I would have for the maintainers of this repository is whether or not they foresee the need for helpers/abstractions on top of this project that could be provided by ecosystem libraries like llm. I suppose in this specific case, the gap from Candle would need to align with the existing architecture/design of llm.
I know that @philpax and I have slightly different ways of looking at llm (and he is the primary maintainer), but from my POV it was always about Rust bindings (or perhaps a Rust implementation) of LLM quantization, friendly interfaces for inferring text from quantized models, and supporting the most ubiquitous architectures used by quantized models. How do these goals relate to the goals of this project? If they are close to being the same, I heartily agree that we should defer to this project, which is clearly going to be more well-supported. If there are "gaps" though, I would be very interested in hearing about them 👂🏻

I would love to know MPS timeline as well.

Hello,
Thanks for reaching out and for having put all this work on the llm crate.
To me the goals of llm and candle are fairly different. I can mostly talk about the candle side but the main differences to me would be the following (obviously feel free to correct me as I actually never used llm).

Candle aims at being a generic ML framework, so covers LLMs but also other architectures.
While it's optimized mostly for inference, training is still possible (backprop is supported on almost all ops).
Candle tries to be a pure Rust solution so that it can be run in the browser for example.
The scope of candle seems closer to PyTorch than to llm. Though we want candle to be really good at LLM inference including good quantization support, though it may not be as efficient as the one in llm.

One of the ideas for building candle is to drive the core libraries development by the requirements of state of the art models and that's where candle-transformers belongs. This started as being a collection of various models used to show how to use the core libraries but is becoming more and more a collection of models that can be re-used by users that do not want to re-implement the models. The scope covered is wide and it may be split at some point (e.g. stable diffusion may have its own crate, computer vision bits may move to another one etc) but that's not scheduled for the near future.
One key aspect of candle is to provide flexibility compared to llama.cpp for example. E.g. if someone wants to try quantizing any model, candle should make this easy, either by writing rust code or even by first prototyping things in Python using the candle-pyo3 layer.

With that in mind, people in that discussion have raised a few issues around switching to candle-transformers and think llm is still relevant. I think it'd be simpler for the ecosystem if there was a single place for LLMs, but the concerns raised have made it harder to provide a straightforward recommendation to switch to candle-transformers.
So, here are my questions:

What's the long-term plan for candle-transformers? Vague question, I know, but will it live in this repo forever? Will it become an ecosystem unto itself like Python transformers?

That's a bit unclear. For the close future it will be in this repo, and we will continue pushing more models into it, hopefully adding examples of missing model types such as tts (tortoise? musicgen?), multimodal (fuyu?), etc.
What I would like to see is third parties implementing models based on candle in separate repos and instead of having candle-transformers we have more of a model hub that easily lets you use models from any repos. This hasn't happened much yet, and maybe to final users, having a central repo with lots of models is more convenient? I'm pretty curious to see how this evolves over the next few months, and this will help deciding on where we bring candle-transformers next.

Will the models share a unified interface to make it easy to swap between architectures?

We haven't done much of that yet. It's fairly tricky as lots of models have various kinks that make it hard to have very generic traits. Here we try more to propose a collection of models rather than a framework for such models, this may evolve but I feel that the amount of boilerplate to switch from a model to another is not that large currently (this can be seen in the examples) and would be more in favor of making helper functions for the shared bits rather than providing a fully unified interface.
That said, it would be great if someone wanted to try building their own crate on top of candle-transformers that would provide such an interface, I would be very curious to see how it goes.

Similarly, are there any plans to offer a higher-level abstraction for model inference? I'm not completely happy with our own interface, but it makes common tasks (prompt-feeding, streamed inference) easy, and harder tasks (custom inference, custom models) doable: https://github.com/rustformers/llm/blob/main/crates/llm/examples/inference.rs

Not that much in candle-transformers, we do have helper functions for these and likely will build more. Maybe there will be a candle-llm crate someday to unify the LLM aspects a bit but here also I would certainly like to see more experimentation from the community on that before having some form of "official" crate.

Are there plans for Metal acceleration support in the near-future? I'm guessing this is tracked in Apple silicon (MPS backends) support? #313, but I'm not sure what the timeline of that is.

Yes, we do want to support metal and there has been some progresses on it. I'm not sure about the timeline neither but it's certainly something that we do want to have.

Finally, do you think a library like llm is necessary? My gut feeling is that Candle (and candle-transformers) could grow to cover all of its territory, but there may still be some value in custom high-level abstractions and support for other backends. That would be obviated by Candle developing its own high-level abstractions and increasing its backend support, though.

My feeling on this is that it's better to have multiple crates trying different designs, aim at different use cases and overall experiment different tradeoffs. On the backend side, metal and webgpu are two obvious things that would help lots of candle users and are on the todo list. For high level abstractions, this really depend on users demand and also on what the community builds. Having higher level crates built on candle would be the best, and hopefully they can be developed and maintained independently. That said if none of these appear and we get lots of users demand for such abstractions we should consider building them directly.

Thanks for the detailed response! It's clarified a lot of things, and helped me better understand the relationship between Candle and candle-transformers.

To me the goals of llm and candle are fairly different. I can mostly talk about the candle side but the main differences to me would be the following (obviously feel free to correct me as I actually never used llm).
* Candle aims at being a generic ML framework, so covers LLMs but also other architectures.

* While it's optimized mostly for inference, training is still possible (backprop is supported on almost all ops).

* Candle tries to be a pure Rust solution so that it can be run in the browser for example.
  The scope of candle seems closer to PyTorch than to `llm`. Though we want candle to be really good at LLM inference including good quantization support, though it may not be as efficient as the one in `llm`.

Yes, I'd agree with this. It would be nice to cover the additional territory that you do, but it's not a priority for us; our focus is on ensuring that LLM inference is easy and fast first and foremost. I'd be happy to see Candle as a backend some day :)

What's the long-term plan for candle-transformers? Vague question, I know, but will it live in this repo forever? Will it become an ecosystem unto itself like Python transformers?

That's a bit unclear. For the close future it will be in this repo, and we will continue pushing more models into it, hopefully adding examples of missing model types such as tts (tortoise? musicgen?), multimodal (fuyu?), etc. What I would like to see is third parties implementing models based on candle in separate repos and instead of having candle-transformers we have more of a model hub that easily lets you use models from any repos. This hasn't happened much yet, and maybe to final users, having a central repo with lots of models is more convenient? I'm pretty curious to see how this evolves over the next few months, and this will help deciding on where we bring candle-transformers next.

That makes sense; I'll be keeping an eye on things as well, sounds like an exciting frontier 🙂

Will the models share a unified interface to make it easy to swap between architectures?

We haven't done much of that yet. It's fairly tricky as lots of models have various kinks that make it hard to have very generic traits. Here we try more to propose a collection of models rather than a framework for such models, this may evolve but I feel that the amount of boilerplate to switch from a model to another is not that large currently (this can be seen in the examples) and would be more in favor of making helper functions for the shared bits rather than providing a fully unified interface. That said, it would be great if someone wanted to try building their own crate on top of candle-transformers that would provide such an interface, I would be very curious to see how it goes.

That's understandable, and I can see where the challenges come from, especially as you're implementing a diverse suite of models. I've found a unified interface to be useful from a library design perspective, as it allows people to learn one API and use it for every model, as opposed to having to look out for differences in individual models.
However, I don't think that's too much of an issue for the current implementations - they all share a pretty similar interface from what I can see. The only thing to watch out for in this regard would be ensuring they don't drift too far apart (principle of least surprise and whatnot).

Similarly, are there any plans to offer a higher-level abstraction for model inference? I'm not completely happy with our own interface, but it makes common tasks (prompt-feeding, streamed inference) easy, and harder tasks (custom inference, custom models) doable: https://github.com/rustformers/llm/blob/main/crates/llm/examples/inference.rs

Not that much in candle-transformers, we do have helper functions for these and likely will build more. Maybe there will be a candle-llm crate someday to unify the LLM aspects a bit but here also I would certainly like to see more experimentation from the community on that before having some form of "official" crate.

Yup, gotcha! That makes sense, the experimental phase should be quite interesting 🙂

Are there plans for Metal acceleration support in the near-future? I'm guessing this is tracked in Apple silicon (MPS backends) support? #313, but I'm not sure what the timeline of that is.

Yes, we do want to support metal and there has been some progresses on it. I'm not sure about the timeline neither but it's certainly something that we do want to have.

👍

Finally, do you think a library like llm is necessary? My gut feeling is that Candle (and candle-transformers) could grow to cover all of its territory, but there may still be some value in custom high-level abstractions and support for other backends. That would be obviated by Candle developing its own high-level abstractions and increasing its backend support, though.

My feeling on this is that it's better to have multiple crates trying different designs, aim at different use cases and overall experiment different tradeoffs. On the backend side, metal and webgpu are two obvious things that would help lots of candle users and are on the todo list. For high level abstractions, this really depend on users demand and also on what the community builds. Having higher level crates built on candle would be the best, and hopefully they can be developed and maintained independently. That said if none of these appear and we get lots of users demand for such abstractions we should consider building them directly.

Awesome, thank you for clarifying all of this. You're right that having multiple different takes on the problem might help us figure out what the best set of tradeoffs are, and that community demand/builds will be crucial to the growth of these ecosystems.
Based on what you've said, I'm now confident with continuing development on llm. For now, llm will cover some territory that Candle won't/can't (Metal-through-GGML acceleration, a LLM-focused API), while the Candle ecosystem can freely experiment and develop more backends and abstractions. In future, there's likely to be more overlap with either llm supporting Candle or candle-transformers growing/becoming community-maintained, but we can figure out what that looks like when we get there. Looking forward to a friendly work relationship!
All of my questions have been answered; I'm happy to close the issue, but other people might appreciate the information you've provided, so I'll leave it up to you. Thanks once again - your insight has been absolutely invaluable ❤

All of my questions have been answered; I'm happy to close the issue, but other people might appreciate the information you've provided, so I'll leave it up to you. Thanks once again - your insight has been absolutely invaluable ❤

Great very happy that this was useful and that the effort on llm will continue - having more people doing ML in Rust is good for everyone! And yes let's keep this issue open as it may give some context - though obviously things will evolve. I would be delighted if llm uses candle as a possible backend at some point - happy to help with this if I can - and don't hesitate to borrow bits from candle-transformers or candle-examples if useful.


ERC-721 with Mutable Metadata

mutable metadata - copied ERC721TL, stripped out Synergy, and replace…d with MutableMetadata.

I'm assuming you already ran tests on this contract and it passed

The test file is part of the PR

I saw the test file, just didn’t see the results.
Cheers,
Ben Strauss
Founder
@.***
@transientlabs https://twitter.com/TransientLabs
www.transientlabs.xyz
On Mon, Aug 12, 2024 at 5:04 PM Marco @.***> wrote:

I'm assuming you already ran tests on this contract and it passed
The test file is part of the PR
—
Reply to this email directly, view it on GitHub
https://github.com/Transient-Labs/tl-creator-contracts/pull/54#issuecomment-2285043484,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AWZ3TO65CN74N3G53HTVJNLZRE5QFAVCNFSM6AAAAABMK5C45GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBVGA2DGNBYGQ
.
You are receiving this because your review was requested.Message ID:
@.***>


The home page could use some style

Looking for a pull request that includes a CSS file...

Hello there! May I do this? I was thinking something like this:


Todo: where should this go?

rename to PackageExtensionCompat?
move to Compat.jl?
move to Requires.jl? Seems like a natural fit.

I decided against Requires.jl because by keeping it external, we can avoid loading Requires.jl at all on newer versions of Julia.

So going to release as its own package.


fix click events re-propogation on mobile

fixes GH-170
Previously, we did not fire a click event for touch events
on mobile unless the event target was a form field. This is
incorrect as the event was not preventDefault'd before.
As you can see on Lines 48-50 in https://github.com/emberjs/list-view/blob/0e596c9c629fa076e9089e787bb2e360b8efccb6/packages/list-view/lib/virtual_list_scroller_events.js#L48-50 (not the commit
that introduced the bug, just a hard link reference),
the "begin scroll" handler does not preventDefault
on the event if it is a form field, so we do not need
to re-propogate it in synthesizeClick.
All other tag types do need to synthesize the click
event on mobile however.
I ran the tests in the XCode simulator and they passed.

 Glad we came up with a similar fix though. :) cursor: pointer as @krisselden suggested in that PR does not make stuff like link-to etc work, so we need one of these PRs


Persistence settings should be parameterized - causes issues.

After this PR has been merged to Kubernetes https://github.com/kubernetes/kubernetes/pull/103958
There is issues with long-time running connections (with no activity).
example:
require 'k8s-ruby'

K8s::Logging.debug!
K8s::Transport.verbose!

client = K8s::Client.config(
  K8s::Config.load_file(
    File.expand_path '~/.kube/config'
  )
)

sleep_seconds = 120

client.api('v1').resource('pods', namespace: 'default').list().each do |pod|
    puts "namespace=#{pod.metadata.namespace} pod: #{pod.metadata.name} node=#{pod.spec.nodeName}"
end

puts "Now sleeping #{sleep_seconds}"

while sleep_seconds > 0
    puts "Countdown - #{sleep_seconds} SECONDS(S) LEFT!"
    sleep_seconds -= 1
    sleep 1
end
client.api('v1').resource('pods', namespace: 'default').list().each do |pod|
    puts "namespace=#{pod.metadata.namespace} pod: #{pod.metadata.name} node=#{pod.spec.nodeName}"
end

This fails with:
Traceback (most recent call last):
	11: from k8s.rb:27:in `<main>'
	10: from /var/lib/gems/2.7.0/gems/k8s-ruby-0.10.5/lib/k8s/resource_client.rb:201:in `list'
	 9: from /var/lib/gems/2.7.0/gems/k8s-ruby-0.10.5/lib/k8s/resource_client.rb:210:in `meta_list'
	 8: from /var/lib/gems/2.7.0/gems/k8s-ruby-0.10.5/lib/k8s/transport.rb:284:in `request'
	 7: from /var/lib/gems/2.7.0/gems/excon-0.89.0/lib/excon/connection.rb:282:in `request'
	 6: from /var/lib/gems/2.7.0/gems/excon-0.89.0/lib/excon/connection.rb:451:in `response'
	 5: from /var/lib/gems/2.7.0/gems/excon-0.89.0/lib/excon/middlewares/redirect_follower.rb:82:in `response_call'
	 4: from /var/lib/gems/2.7.0/gems/excon-0.89.0/lib/excon/middlewares/response_parser.rb:7:in `response_call'
	 3: from /var/lib/gems/2.7.0/gems/excon-0.89.0/lib/excon/response.rb:63:in `parse'
	 2: from /var/lib/gems/2.7.0/gems/excon-0.89.0/lib/excon/response.rb:63:in `loop'
	 1: from /var/lib/gems/2.7.0/gems/excon-0.89.0/lib/excon/response.rb:64:in `block in parse'
/var/lib/gems/2.7.0/gems/excon-0.89.0/lib/excon/socket.rb:65:in `readline': EOFError (EOFError)
	11: from k8s.rb:27:in `<main>'
	10: from /var/lib/gems/2.7.0/gems/k8s-ruby-0.10.5/lib/k8s/resource_client.rb:201:in `list'
	 9: from /var/lib/gems/2.7.0/gems/k8s-ruby-0.10.5/lib/k8s/resource_client.rb:210:in `meta_list'
	 8: from /var/lib/gems/2.7.0/gems/k8s-ruby-0.10.5/lib/k8s/transport.rb:284:in `request'
	 7: from /var/lib/gems/2.7.0/gems/excon-0.89.0/lib/excon/connection.rb:282:in `request'
	 6: from /var/lib/gems/2.7.0/gems/excon-0.89.0/lib/excon/connection.rb:451:in `response'
	 5: from /var/lib/gems/2.7.0/gems/excon-0.89.0/lib/excon/middlewares/redirect_follower.rb:82:in `response_call'
	 4: from /var/lib/gems/2.7.0/gems/excon-0.89.0/lib/excon/middlewares/response_parser.rb:7:in `response_call'
	 3: from /var/lib/gems/2.7.0/gems/excon-0.89.0/lib/excon/response.rb:63:in `parse'
	 2: from /var/lib/gems/2.7.0/gems/excon-0.89.0/lib/excon/response.rb:63:in `loop'
	 1: from /var/lib/gems/2.7.0/gems/excon-0.89.0/lib/excon/response.rb:64:in `block in parse'
/var/lib/gems/2.7.0/gems/excon-0.89.0/lib/excon/socket.rb:65:in `readline': EOFError (EOFError) (Excon::Error::Socket)

in lib/k8s/transport.rb - changing line 173 - from persistent: true to persistent: false - seems to solve the issue.
So I suggest that the persistence can be overridden when initiating the client.

I found the answer..
client = K8s::Client.config(
  K8s::Config.load_file(
    File.expand_path '~/.kube/config'
  ),
  persistent: false
)


fix(throw): Observable.throw() now returns Observable

Description: Observable.never(), Observable.empty() and .ignoreElements() were previously changed to Observable<never>; this patch complements it with Observable.throw().
Related issue (if exists): #2640 and its PR #2691

This change is now included in the throwError refactor. Closing this PR in favor of #3288


Add ES 8.x support

In addition to pinning v8 of elasticsearch-ruby, there were two other things I had to consider.
ES 8 does not allow sending a document type. I followed previous work in this repo and used Hash#compact to remove the type param.
elasticsearch-ruby v8 uses the new elastic-transport gem instead of the older elasticsearch-transport one. I added a (hopefully) backwards compatible solution via ELASTIC_TRANSPORT_CLASS.

The test suite needs to run with ES 8 on this branch but I'm not familiar enough with it to make the right change.


Remove autopin

In my Zotero library, all new added items automatically gets pinned before I get to change fields in the metadata. I looked into the hidden preferences but can only see the option for delay - is it possible to manually pin the citekeys?

Support log ID: CZW8RYY7-euc

It should turn off if you set autoPin to 0 (which is the default value, so clearing autoPin should set it to 0).

Thank you - I accidently pressed the "set to default" instead of just typing 0 and now it has disappeared from the hidden preferences

That's a bug -- a version with a fix is building now.

It came back when I restarted. Thank you!


fixed title text format

Docs Build status updates of commit 4fb2bf1:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

mixed-reality-docs/mrlearning-sharing(photon)-ch2.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


the 1.19.4 seed cracker version not working

when i try to set the version for 1.19.4 it shows only up to 1.19

when i try to set the version for 1.19.4 it shows only up to 1.19

It will still work

1.19.3 does not work on 1.19.4. Hopefully the 1.19.4 version is coming soon.

1.19.3 does not work on 1.19.4. Hopefully the 1.19.4 version is coming soon.

Works fine on 1.19.4 for me, might be some other mod you have messing with it.

EDIT: found pre-release version, works on 1.19.4


Auditlog is not implemented

I have been trying to test some seclang rule but I am getting not implemented in audit log without any response
putting
SecRule REQUEST_HEADERS:X-User-ID "@pm user123 user456" \
    "id:1002,phase:2,pass,log,msg:'User authorized',chain"
    SecRule REQUEST_URI "@streq /protected-endpoint" "t:none"

SecRule REQUEST_HEADERS:X-User-ID "!@pm user123 user456" \
    "id:1003,phase:2,deny,status:403,log,msg:'Unauthorized user',chain"
    SecRule REQUEST_URI "@streq /protected-endpoint" "t:none"

in directive
and then in request
curl -H "X-User-ID: user789" -X GET http://localhost:8080/api/hello

I make sure that api/hello is running, also tried with a deployed URL but it seems like requests are not going anywhere
What will be the best way to test it as reverse proxy with custom seclang rules
With docker I can probably use something like :
https://github.com/jcchavezs/coraza-http-wasm-traefik

Duplicate #20.


Unify tabs placement

e.g. connector overview page:

tabs are placed above the panes and the page looks empty and weird. Let's unify all the views with this approach:

broker overview page panes are located below the tabs therefore "overview" tab is no longer required

Tabs are still on top for topics:

Needs a fix.

@Haarolean considering the comment from Oleg in #2637 , can this considered to WAD for the topics and therefore closed?

@Haarolean there is still Overview tab with old layout on the Connector details page:

Should this defect be reopened?


Blackhole connector

====================
Black Hole Connector
The Black Hole connector works in similar way as /dev/null device file in Linux operating system.
SELECT statement from any of its table will return no rows and INSERT will also be mocked and no real
writes will be performed.
All information (meta data) about created tables live only in memory, so when you restart server they
will be discarded.
Example usage
Create table using blackhole connector::
CREATE TABLE blackhole.default.nation AS SELECT * from tpch.tiny.nation;

Insert data to table in blackhole connector::
INSERT INTO blackhole.default.nation SELECT * FROM tpch.tiny.nation;

Read from blackhole connector::
SELECT COUNT(*) FROM blackhole.default.nation;

Please add tests for this connector

Hey, I updated the pull request. All comments should be now solved with integration tests included. Also, I had to do a rebase to make code be pass on travis automation and so I had to push with force
(git push -f).

@martint, @electrum  Thanks for great comments ;) I think I have fixed them all. Is there anything left, what are the next steps?

I squashed the commits, made a few edits to the docs and merged it. Thanks!


NIFI-3955: Filtering out the EventTime searchable field for all implementation of ProvenanceRepository

NIFI-3955:

Filtering out the EventTime searchable field for all implementation of ProvenanceRepository. Prefer user query using the startDate/endDate fields.

+1, merged to master, thanks @mcgilman!


Objects with max null value

Hi,
I have received the following messages when running the adapter
oekofen-json.0 | 2022-07-20 07:12:20.125 | warn | State value to set for "oekofen-json.0.pe1.L_resttimeburner" has value "78" greater than max "null"
oekofen-json.0 | 2022-07-20 07:12:20.125 | warn | State value to set for "oekofen-json.0.pe1.L_runtimeburner" has value "179.4" greater than max "null"
the object is set to max null but receiving higher values
I am using the latest Branch
Looxer

Hi @looxer01 thanks for pointing that out.
It should work now; The conversion to number caused null values instead of undefined.

Thank you. Looks good so far


feat: Change certificate generation based on certificate available date

Pre-Flight checklist

[x] Screenshots and design review for any changes that affect layout or styling

[x] Desktop screenshots

[x] Migrations

[x] Migration is backwards-compatible with current production code

[x] Testing

[x] Code is tested
[x] Changes have been manually tested

What are the relevant tickets?
Part of https://github.com/mitodl/mitxonline/issues/1257
What's this PR do? (As of now, this PR enables us to at least add the certificate end dates manually until we sync them with edX)

Adds a new field certificate_available_date
Bases the certificate generation functionality on certificate_available_date instead of end_date
Moving on we need to sync this date with edX (See https://github.com/mitodl/mitxonline/issues/1257#issuecomment-1346463078)

How should this be manually tested?
You will need a few configurations and setting up things to test this PR end-to-end.

Try generating the certificate with steps mentioned in https://github.com/mitodl/mitxonline/pull/722 with certificate_available_date instead of end_date.
The behavior that was once based on the course run's end_date should now be transferred to certificate_available_date. The existing behavior is mentioned in detail at https://github.com/mitodl/mitxonline/pull/722

Where should the reviewer start?
Configure your edX instance with MITxOnline Locally
Any background context you want to provide?
We added the polling and certificate generation in https://github.com/mitodl/mitxonline/pull/722 where we based the conditions on course end_date
NOTE
Probably, there are going to be some other PRs as well that will enable us to sync the certificate_available_date with edX. But we are not yet ready to start on them, See the comments (https://github.com/mitodl/mitxonline/issues/1257#issuecomment-1354905830, https://github.com/mitodl/mitxonline/issues/1257#issuecomment-1355023307)
Screenshots (if appropriate)

Codecov Report

Merging #1286 (dbb4c36) into main (19b4713) will increase coverage by 3.92%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #1286      +/-   ##
==========================================
+ Coverage   83.07%   87.00%   +3.92%
==========================================
  Files         297      146     -151
  Lines       12075     4162    -7913
  Branches      832        0     -832
==========================================
- Hits        10031     3621    -6410
+ Misses       1813      541    -1272
+ Partials      231        0     -231

Impacted Files
Coverage Δ

courses/api.py

courses/factories.py

courses/models.py

authentication/utils.py

main/apps.py

mail/templatetags/timeago.py

main/serializers.py

cms/wagtail_hooks.py

courses/mail_api.py

...icing/management/commands/update_exchange_rates.py

... and 141 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

How does this work with self-paced courses?

How does this work with self-paced courses?

With self-paced courses it would work the same as before, e.g. generating the certificates whenever a user has completed the course w/ passed grades and the enrollment is paid/verified. certificate_available_date won't matter in this case.

This has been reviewed as part of https://github.com/mitodl/mitxonline/pull/1317.

Closing this PR since https://github.com/mitodl/mitxonline/pull/1317 was merged recently that was rebased on this PR and has now been reviewed and merged so this PR can now be closed.


example: add manual captions test to basic example

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 58.95%. Comparing base (b80904d) to head (89c24a3).
Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #208   +/-   ##
=======================================
  Coverage   58.95%   58.95%
=======================================
  Files          30       30
  Lines        2373     2373
  Branches      132      132
=======================================
  Hits         1399     1399
  Misses        970      970
  Partials        4        4

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


refactor: PRSDM-6980 Upgrade Bootstrap to v5.3.3

Description

Upgraded Bootstrap to v5.3.3.
Removed external resources, see #311
Updated jQuery
Added a required container in the navbar for Bootstrap v5.3.3

Bootstrap Cleanup:

Remove unnecessary Bootstrap components and adjust layout for consistency

Issue

PRSDM-6980

Screenshots
PR Readiness Checks

[x] Your PR title conforms to conventional commits <type>: <jira-ticket-num><title>, for example: fix: PRSDM-123 issue with login with a maximum of 100 characters
[x] You have performed a self-review of your changes via the GitHub UI
[ ] Comments were added to new code that can not explain itself (see reference 1 and reference 2)
[ ] New code adheres to the following quality standards:

Function Length (see reference)
Meaningful Names (see reference)
DRY (see reference)
YAGNI (see reference)

I see you removed the CDN imports for Jquery & bootstrap CSS and moved the jquery and bootstrap js to scripts here. Should the bootstrap CSS not also go to the script file?

@CharlRitterDev I found that the only class we are actually using from Bootstrap CSS is the collapse class. Instead of including the entire Bootstrap CSS file, I opted to keep just the SCSS files responsible for the collapse class. These are:
component_animations.scss
_vendor-prefixes.scss

@julianbyte Oh, interesting. So we can remove the Bootstrap import completely?

@CharlRitterDev I found that the only class we are actually using from Bootstrap CSS is the collapse class. Instead of including the entire Bootstrap CSS file, I opted to keep just the SCSS files responsible for the collapse class. These are:
component_animations.scss _vendor-prefixes.scss


Unexpected BadSqlGrammarException Instead of TimeoutException on Query Timeout

Describe the bug
When using the ClickHouse JDBC driver with JdbcTemplate in a Spring application, a query timeout is set using JdbcTemplate.setQueryTimeout(10). However, when the query exceeds the configured timeout, a BadSqlGrammarException is thrown instead of the expected TimeoutException. This behavior is misleading as it suggests an issue with SQL syntax rather than a timeout.
Steps to reproduce

Configure JdbcTemplate with a query timeout: jdbcTemplate.setQueryTimeout(10) (10 seconds).
Execute a query that takes longer than the configured timeout.
Observe the exception thrown when the query times out.

Expected behaviour
A TimeoutException or another appropriate exception should be thrown to clearly indicate that the query execution exceeded the configured timeout.
Code example
public executeQuery(String query) {
        jdbcTemplate.query(query)
}

Error log
org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [SELECT * FROM large_table];
nested exception is java.sql.SQLException: Code: 159. DB::Exception: Timeout exceeded: elapsed 10.114165251 seconds, maximum: 10: While executing MergeTreeSelect(pool: ReadPoolInOrder, algorithm: InOrder). (TIMEOUT_EXCEEDED) (version <IP_ADDRESS> (official build))

	at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:101)
	at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:70)
	at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:79)
	at org.springframework.jdbc.core.JdbcTemplate.translateException(JdbcTemplate.java:1541)
	at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:393)
	at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:465)

Configuration
Environment

Client version: clickhouse-jdbc:0.4.6
Language version: Java 11
OS: windows 11

ClickHouse server

ClickHouse Server version: <IP_ADDRESS>
ClickHouse Server non-default settings, if any:
CREATE TABLE statements for tables involved:
Sample data for all these tables, use clickhouse-obfuscator if necessary

Good day, @koyr-exem !
Thank you for bringing it up. We will try to look into it soon.


Action fails on ubuntu-24.04 with "This environment is externally managed"

Description
The Ubuntu 24.04 image is now in public beta. Otherwise unmodified workflows that worked on ubuntu-22.04 now fail on ubuntu-24.04 with error: externally-managed-environment.
Minimal workflow to reproduce:
name: Test

on:
  push:

jobs:
  build:
    runs-on: ubuntu-24.04
    permissions:
      contents: read
      id-token: write

    steps:
      - name: Create artefact
        run: |
            mkdir dist
            touch dist/file.txt

      - name: Sign the artifacts with Sigstore
        uses<EMAIL_ADDRESS>        with:
          inputs: dist/*

Here's the full log of the workflow run:
2024-07-03T12:38:43.8015044Z Current runner version: '2.317.0'
2024-07-03T12:38:43.8051724Z ##[group]Operating System
2024-07-03T12:38:43.8052639Z Ubuntu
2024-07-03T12:38:43.8053222Z 24.04
2024-07-03T12:38:43.8053866Z LTS
2024-07-03T12:38:43.8054409Z ##[endgroup]
2024-07-03T12:38:43.8055097Z ##[group]Runner Image
2024-07-03T12:38:43.8056005Z Image: ubuntu-24.04
2024-07-03T12:38:43.8056667Z Version: 20240630.1.0
2024-07-03T12:38:43.8058336Z Included Software: https://github.com/actions/runner-images/blob/ubuntu24/20240630.1/images/ubuntu/Ubuntu2404-Readme.md
2024-07-03T12:38:43.8061120Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu24%2F20240630.1
2024-07-03T12:38:43.8062596Z ##[endgroup]
2024-07-03T12:38:43.8063349Z ##[group]Runner Image Provisioner
2024-07-03T12:38:43.8064352Z 2.0.370.1
2024-07-03T12:38:43.8064942Z ##[endgroup]
2024-07-03T12:38:43.8066420Z ##[group]GITHUB_TOKEN Permissions
2024-07-03T12:38:43.8068658Z Contents: read
2024-07-03T12:38:43.8069368Z Metadata: read
2024-07-03T12:38:43.8070295Z ##[endgroup]
2024-07-03T12:38:43.8074915Z Secret source: Actions
2024-07-03T12:38:43.8076022Z Prepare workflow directory
2024-07-03T12:38:43.8744996Z Prepare all required actions
2024-07-03T12:38:43.8904119Z Getting action download info
2024-07-03T12:38:44.0087330Z Download action repository<EMAIL_ADDRESS>(SHA:61f6a500bbfdd9a2a339cf033e5421951fbc1cd2)
2024-07-03T12:38:44.4755944Z Getting action download info
2024-07-03T12:38:44.5526751Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32)
2024-07-03T12:38:44.6780861Z Download action repository 'softprops/action-gh-release@v1' (SHA:de2c0eb89ae2a093876385947365aca7b0e5f844)
2024-07-03T12:38:44.9151804Z Complete job name: build
2024-07-03T12:38:44.9952532Z ##[group]Run mkdir dist
2024-07-03T12:38:44.9952911Z mkdir dist
2024-07-03T12:38:44.9953197Z touch dist/file.txt
2024-07-03T12:38:45.0008817Z shell: /usr/bin/bash -e {0}
2024-07-03T12:38:45.0009152Z ##[endgroup]
2024-07-03T12:38:45.0486401Z ##[group]Run<EMAIL_ADDRESS>2024-07-03T12:38:45.0487052Z with:
2024-07-03T12:38:45.0487361Z   inputs: dist/*
2024-07-03T12:38:45.0487701Z   staging: false
2024-07-03T12:38:45.0488077Z   verify: false
2024-07-03T12:38:45.0488432Z   upload-signing-artifacts: false
2024-07-03T12:38:45.0488852Z   release-signing-artifacts: false
2024-07-03T12:38:45.0489320Z   internal-be-careful-debug: false
2024-07-03T12:38:45.0489704Z ##[endgroup]
2024-07-03T12:38:45.0705474Z ##[group]Run # NOTE: Sourced, not executed as a script.
2024-07-03T12:38:45.0706080Z # NOTE: Sourced, not executed as a script.
2024-07-03T12:38:45.0706657Z source "${GITHUB_ACTION_PATH}/setup/setup.bash"
2024-07-03T12:38:45.0738703Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-07-03T12:38:45.0739206Z env:
2024-07-03T12:38:45.0739695Z   GHA_SIGSTORE_PYTHON_INTERNAL_BE_CAREFUL_DEBUG: false
2024-07-03T12:38:45.0740175Z ##[endgroup]
2024-07-03T12:38:47.5962677Z error: externally-managed-environment
2024-07-03T12:38:47.5963266Z
2024-07-03T12:38:47.5963782Z × This environment is externally managed
2024-07-03T12:38:47.5964696Z ╰─> To install Python packages system-wide, try apt install
2024-07-03T12:38:47.5965591Z     python3-xyz, where xyz is the package you are trying to
2024-07-03T12:38:47.5966202Z     install.
2024-07-03T12:38:47.5966542Z
2024-07-03T12:38:47.5967116Z     If you wish to install a non-Debian-packaged Python package,
2024-07-03T12:38:47.5968085Z     create a virtual environment using python3 -m venv path/to/venv.
2024-07-03T12:38:47.5969058Z     Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
2024-07-03T12:38:47.5969908Z     sure you have python3-full installed.
2024-07-03T12:38:47.5970397Z
2024-07-03T12:38:47.5971245Z     If you wish to install a non-Debian packaged Python application,
2024-07-03T12:38:47.5972109Z     it may be easiest to use pipx install xyz, which will manage a
2024-07-03T12:38:47.5972954Z     virtual environment for you. Make sure you have pipx installed.
2024-07-03T12:38:47.5974016Z
2024-07-03T12:38:47.5974506Z     See /usr/share/doc/python3.12/README.venv for more information.
2024-07-03T12:38:47.5975046Z
2024-07-03T12:38:47.5976672Z note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
2024-07-03T12:38:47.5978420Z hint: See PEP 668 for the detailed specification.
2024-07-03T12:38:47.6322452Z ##[error]Process completed with exit code 1.
2024-07-03T12:38:47.6511283Z Cleaning up orphan processes

Using actions/setup-python@v5 before<EMAIL_ADDRESS>makes the problem go away and the workflow completes successfully.
Version
<EMAIL_ADDRESS>

Thanks for the report. This is almost certainly PEP 668 rearing its head on Ubuntu 24.04, which I haven't added to the CI yet. Looking into it now.

#145 will fix this. Thanks again for the report!

Thanks a lot for the fast turnaround. Works great.


[Event Request] Codeunit 5981 Service-Post (Yes/No) - PreviewDocument - OnBeforePreviewDocument

Hi,
We would like to extend Codeunint Service-Post (Yes/No) Preview Document procedure like this:

[IntegrationEvent(false, false)]
    local procedure OnBeforePreviewDocument(var ServHeader: Record "Service Header"; IsHandled: Boolean)
    begin
    end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.


Promisify mongoose calls with bluebird

Work in progress here https://github.com/Ks89/My-MEAN-website-server/tree/experimental-promises


BigQuery query method should default to standard SQL syntax

I believe this was a GA requirement from the BigQuery engineering team, that the default SQL syntax be changed at the client-library layer for GA. The backend API still defaults to legacy SQL, but they would like to discourage the use of legacy SQL in most applications.

Hi @tswast! This was added in #640. It's described in our documentation here. (pardon the broken markdown, i just opened a PR to fix that).

Okay, thanks. I guess my problem was that https://github.com/GoogleCloudPlatform/php-docs-samples/tree/master/bigquery/shakespeare is using a very old version. Somehow I didn't end up with the latest version when I deleted the composer lock file and tried to reinstall.
Sorry for the noise!

0.46 is the version of the "umbrella" package which installs every component. When installing just BQ itself you will want to relax the version constraint down to 0.3.1

Ah, gotcha. Might be worth putting a badge on https://github.com/GoogleCloudPlatform/google-cloud-php-bigquery with the latest version?

@tswast that's a great idea!

https://github.com/GoogleCloudPlatform/google-cloud-php-bigquery/pull/1

@tswast thanks for the PRs. However, those repos are Read-Only splits from this one! The file you're looking for is here.


Screenshot in the article displaying the new number code matching function is incorrect

[Enter feedback here]
The screenshot displaying the example of the multifactor authentication number matching is misleading to end users and incorrect in the sense that the example screenshot approve sign in screen says "TAP" the number you see below in your MS Authenticator App. when in fact one is entering the number.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 5d46abb2-6ad9-e86e-8808-b69ca11922d8
Version Independent ID: c4ccd9d6-13c7-6150-6a44-241bdf9b0982
Content: Use number matching in multifactor authentication (MFA) notifications - Azure Active Directory - Microsoft Entra
Content Source: articles/active-directory/authentication/how-to-mfa-number-match.md
Service: active-directory
Sub-service: authentication
GitHub Login: @Justinha
Microsoft Alias: justinha

@adinchadjacobs
Thanks for your feedback! We will investigate and update as appropriate.

To add some context, number matching looks like this on Microsoft accounts:

...although, see https://github.com/MicrosoftDocs/azure-docs/issues/87904 - so not entirely sure if the AAD method uses a 'type in the number' vs 'tap the number' method. This probably explains the discrepancy.

Thanks for your feedback. We need to close all open issues at this time so we can migrate to a new Entra repository. I created a work item to have an expert review this feedback and update docs accordingly. #please-close


Not able to "Fetch Fleet Certificate"

What version of fleet are you using (fleet version --full)?
fleet - version 2.3.0
branch: 	master
revision: 	bc9782e9b0c431ce8c24552861f5527fb0f4c390
build date: 	2019-08-14T18:55:06Z
build user: 	zwass
go version: 	go1.12.8
What operating system are you using?
Ubuntu 18.04.3 LTS
What did you do?
Clicked on Fetch Fleet Certificate.
What did you expect to see?
The certificate to be downloaded from the server.
What did you see instead?
Nothing happened.
I am trying to set up launcher and need to get the certificate from Fleet. For context I've set up Fleet successfully using certbot. Everything is working as intended, I just can't download the certificate.
If there's a workaround where I could copy one of the pems from certbot, that would suffice. Just following the instructions on your help page and other pages I've come across the web that say to do the same thing.

Hello @dmarcsift
I have the exact same issue with the same configuration. Do you know what to do to fix please ?


"value," "crit" and "warn"

The current Grapher.php appends '.value' to $target if legacyMode == false. The graphite_large_args_template then appends '_crit' and '_warn', so we end up with $target$.value_crit. This doesn't work because icinga 2.4 uses $target$.crit.
Maybe Grapher.php should never append '.value'. This can be done by the templates.

Good call @hk298, It would indeed make sense to move value out to the template, also Grapher ->metricPrefix  has become somewhat redundant too.
Do you fancy sending a pull request for that?

@philiphoy I'd rather leave that to someone who knows the code and github.

@hk298 Fair enough i will take it on as as it needs sorting. That said os is fun and i would be happy to help you out if you want. I am sure you would find it easier than you think.

Done as of 515fa97a9e10860fc1d8bfd0ea9e747855fccd9f


Misleading comment in ServerBootstrap

The mechanism of registering handlers seems to have been changed since #5566 has been fixed, making this comment misleading.
I did not have time to inspect the whole history, but the user's initChannel is now not called after the current invocation returns. As an evidence, the code patched with the following change on tag 4.1.9.Final also passes this test case, which was introduced to ensure that #5566 was fixed.
$ git diff
diff --git a/transport/src/main/java/io/netty/bootstrap/ServerBootstrap.java b/transport/src/main/java/io/netty/bootstrap/ServerBootstrap.java
index f9042af46..b73239ffa 100644
--- a/transport/src/main/java/io/netty/bootstrap/ServerBootstrap.java
+++ b/transport/src/main/java/io/netty/bootstrap/ServerBootstrap.java
@@ -174,7 +174,9 @@ public class ServerBootstrap extends AbstractBootstrap<ServerBootstrap, ServerCh
                 if (handler != null) {
                     pipeline.addLast(handler);
                 }
-
+                pipeline.addLast(new ServerBootstrapAcceptor(
+                        ch, currentChildGroup, currentChildHandler, currentChildOptions, currentChildAttrs));
+                /*
                 // We add this handler via the EventLoop as the user may have used a ChannelInitializer as handler.
                 // In this case the initChannel(...) method will only be called after this method returns. Because
                 // of this we need to ensure we add our handler in a delayed fashion so all the users handler are
@@ -185,7 +187,7 @@ public class ServerBootstrap extends AbstractBootstrap<ServerBootstrap, ServerCh
                         pipeline.addLast(new ServerBootstrapAcceptor(
                                 ch, currentChildGroup, currentChildHandler, currentChildOptions, currentChildAttrs));
                     }
-                });
+                });*/
             }
         });
     }

Please consider changing the comment, it could be very confusing for newcomers.
Expected behavior
N/A
Actual behavior
N/A
Steps to reproduce
N/A
Minimal yet complete reproducer code (or URL to code)
N/A
Netty version
4.1.9.Final
JVM version (e.g. java -version)
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
OS version (e.g. uname -a)
Linux herberteuler 4.9.22 #1-NixOS SMP Wed Apr 12 10:41:42 UTC 2017 x86_64 GNU/Linux

@herberteuler Would you be able to make the changes and put in a PR if what you say is accurate? Contributions are welcome!

Alright, so I have investigated further and found that it was ChannelInitializer that was changed to make this comment inaccurate. The change was introduced by 26aa348, in which initChannel was also invoked from handlerAdded to ensure the correct order. I will create a pull request soon.

Fixed by https://github.com/netty/netty/pull/6656#pullrequestreview-35908160


Reapplying a patch on android(32?) causes the game to crash

Only Tested on android32, Not sure if this is a problem with geode or tuliphook.
If you create a patch, disown it and then make a new patch at the address. gd will crash

transferred the issue from tuliphook because most of the patch logic is inside geode. the issue seems to be that calling disownPatch causes Patch::Impl::disable to be called twice, which later invokes ub


Hindustan Petroleum Clone

I am a GSSOC'22 contributor and I want to contribute to this repository.

Assigned u @dsasaank-369


Use simulation mode on MacOS

Specs: MacBook Pro (16-inch, 2019), 2.3 GHz 8-Core Intel Core i9
When following the instructions outlined here.
I get the following error:
./lib/libenclave.a(sgx_trts-e1de7fe3db2688ea.sgx_trts.c1x8g4sc-cgu.2.rcgu.o): In function `sgx_trts::enclave::SgxGlobalData::new::hd171d7330c2c7ee0':
sgx_trts.c1x8g4sc-cgu.2:(.text._ZN8sgx_trts7enclave13SgxGlobalData3new17hd171d7330c2c7ee0E+0xa5): undefined reference to `get_rsrv_base'
sgx_trts.c1x8g4sc-cgu.2:(.text._ZN8sgx_trts7enclave13SgxGlobalData3new17hd171d7330c2c7ee0E+0xb2): undefined reference to `get_rsrv_size'
collect2: error: ld returned 1 exit status
Makefile:155: recipe for target 'enclave/enclave.so' failed
make: *** [enclave/enclave.so] Error 1

Steps to reproduce:
1. docker pull baiduxlab/sgx-rust-stable
2. docker run -v /Users/<user.name>/code/incubator-teaclave-sgx-sdk:/root/sgx -ti baiduxlab/sgx-rust-stable
3. cd sgx/samplecode/helloworld/
4. Edited the Makefile to run in SW mode
5. make && cd bin
6. ./app

I've seen some closed issues in this repo that report similar errors, but none fix this exact issue.
Side Notes:

Do I need to have the linux-sgx sdk installed on my Macbook? (The docker container includes all necessary libraries right?)
I have tried the latest build too and that's gave the same error, baiduxlab/sgx-rust.

Just asking this again - anyone managed to run the intel sgx rust sdk sample code on M1 chip even in simulation mode using docker?

I am using sgx sdk on my MacBook Pro M1, and it failed with simulation mode with the following error:

Unknown host CPU architecture arm64

From here it can be easily found that only x86 arch is supported.
https://github.com/apache/incubator-teaclave-sgx-sdk/blob/master/buildenv.mk#L104
Are there any plans for arm64 support?

I am using sgx sdk on my MacBook Pro M1, and it failed with simulation mode with the following error:

Unknown host CPU architecture arm64

From here it can be easily found that only x86 arch is supported. https://github.com/apache/incubator-teaclave-sgx-sdk/blob/master/buildenv.mk#L104
Are there any plans for arm64 support?

This would be much needed …. Or some kind of workaround at least

hi @Astewart1510 this is almost impossible. teaclave sgx sdk is built on top of intel sgx sdk which is x86 only. we can make rust part compile but i'm afraid we can't make intel sgx sdk compile on arm64.

hi @dingelish Thanks for the clarity. I suppose I would have to start using the TrustZone SDK? Would that be compatible with a Macbook Pro M1 chip?

@Astewart1510 here you go https://github.com/apache/incubator-teaclave-trustzone-sdk

@dingelish , sorry wanted to ask if it's possible to change the docker image in such a way that the architecture of the container is x86 ? Therefore the Intel SGX would be able to run on simulation mode on a M1 Mac using a x86 docker image.

Is it possible to use SGX simulation mode on a 2015 mbp with a 2.2 i7-4770HQ? The main purpose would be for playing/ripping UHD4k Blu-ray Discs as this requires SGX.
If this could work, any advice on installation etc would be very welcome. Thanks Neale


First insatllation failed with a lot of waring

Describe the Bug
Today I got there is a great self-hold analytics tool called umami. So I must have a try.
My os is Ubuntu 22.04.3 LTS. I have installed nvm before.
I create my database using:
CREATE DATABASE umamidb WITH OWNER gitea TEMPLATE template0 ENCODING UTF8 LC_COLLATE 'en_US.UTF-8' LC_CTYPE 'en_US.UTF-8';

I installed this followed by official tutorial
When run yarn install, many warnings pop up.
I did not care about these at first, but when I finally initiate it, I found a big issue: there is no button for me to get my track code.

Database
PostgreSQL
Relevant log output
git@mail:~$ npm install -g yarn

changed 1 package in 2s
git@mail:~$ git clone https://github.com/umami-software/umami.git
cd umami
yarn install
fatal: destination path 'umami' already exists and is not an empty directory.
yarn install v1.22.19
[1/4] Resolving packages...
[2/4] Fetching packages...
warning<EMAIL_ADDRESS>The engine "pnpm" appears to be invalid.
[3/4] Linking dependencies...
warning " ><EMAIL_ADDRESS>has incorrect peer dependency "<EMAIL_ADDRESS>warning "react-spring ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>warning "react-spring ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>|| ^16.8.7-0 || ^16.9.0-0 || ^16.10.1-0 || ^16.12.0-0 || ^16.13.0-0 || ^17.0.0-0 || ^17.0.1-0 || ^17.0.2-0 || ^18.0.0-0".
warning "react-spring ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>warning "react-spring ><EMAIL_ADDRESS>has unmet peer dependency "<EMAIL_ADDRESS>warning "react-spring ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>warning "react-spring ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>warning "react-spring ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>warning " ><EMAIL_ADDRESS>has incorrect peer dependency<EMAIL_ADDRESS>|| 17.x".
warning " ><EMAIL_ADDRESS>has incorrect peer dependency<EMAIL_ADDRESS>|| 17.x".
warning " ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>warning " ><EMAIL_ADDRESS>has incorrect peer dependency<EMAIL_ADDRESS>warning " ><EMAIL_ADDRESS>has incorrect peer dependency "stylelint@>= 11.x < 15".
warning " ><EMAIL_ADDRESS>has incorrect peer dependency<EMAIL_ADDRESS>warning Workspaces can only be enabled in private projects.
[4/4] Building fresh packages...
$ node -e "if (process.env.NODE_ENV !== 'production'){process.exit(1)} " || husky install
husky - Git hooks installed
Done in 18.95s.

### Which browser are you using? (if relevant)

_No response_

### How are you deploying your application? (if relevant)

_No response_

git@mail:~/umami$ node -v
v20.7.0
git@mail:~/umami$ npm -v
10.1.0
git@mail:~/umami$ yarn -v
1.22.19
git@mail:~/umami$

When I tried again using docker, I also can not found the track button just the same.

Click on "Edit > Tracking Code" next to your questionmark to obtain the tracking code

Your app seems to be running fine and those install warnings shouldn't cause any issues. If you still have problems retrieving the tracking code please let us know.

@bjornpijnacker @franciscao633
Thank you for your kind answer. But I remember it clearly that if I click edit or view button, there will be a empty page. I have delete the project so I am sorry I can not put photo here. If you need more details, I can reproduce it.

@franciscao633 For me, it's failed with an unexpected error:
❯ yarn install                                                                                ─╯
yarn install v1.22.19
[1/4] Resolving packages...
[2/4] Fetching packages...
warning<EMAIL_ADDRESS>The engine "pnpm" appears to be invalid.
[3/4] Linking dependencies...
warning " ><EMAIL_ADDRESS>has incorrect peer dependency "<EMAIL_ADDRESS>warning "react-spring ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>warning "react-spring ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>|| ^16.8.7-0 || ^16.9.0-0 || ^16.10.1-0 || ^16.12.0-0 || ^16.13.0-0 || ^17.0.0-0 || ^17.0.1-0 || ^17.0.2-0 || ^18.0.0-0".
warning "react-spring ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>warning "react-spring ><EMAIL_ADDRESS>has unmet peer dependency "<EMAIL_ADDRESS>warning "react-spring ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>warning "react-spring ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>warning "react-spring ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>warning " ><EMAIL_ADDRESS>has incorrect peer dependency<EMAIL_ADDRESS>|| 17.x".
warning " ><EMAIL_ADDRESS>has incorrect peer dependency<EMAIL_ADDRESS>|| 17.x".
warning " ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>warning " ><EMAIL_ADDRESS>has incorrect peer dependency<EMAIL_ADDRESS>warning " ><EMAIL_ADDRESS>has incorrect peer dependency "stylelint@>= 11.x < 15".
warning " ><EMAIL_ADDRESS>has incorrect peer dependency<EMAIL_ADDRESS>warning Workspaces can only be enabled in private projects.
[4/4] Building fresh packages...
[1/7] ⢀ @prisma/engines
[-/7] ⢀ waiting...
[3/7] ⢀ sharp
[-/7] ⢀ waiting...
error /home/limmy/umami/node_modules/sharp: Command failed.
Exit code: 127
Command: (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
Arguments:

ubuntu 22.04
node 18.18.0
yarn 1.22.19
npm 9.8.1

@ChanryLimmy This issue may be related. I would run yarn install --verbose, then troubleshoot from there. It may be the same exact issue.

@ChanryLimmy This issue may be related. I would run yarn install --verbose, then troubleshoot from there. It may be the same exact issue.

Thanks a lot.  after yarn install --verbose, it worked, even though nothing i did but add verbose.
[4/4] Building fresh packages...
[1/7] ⠂ @prisma/engines
[1/7] ⠄ @prisma/engines
[2/7] ⠄ @netlify/esbuild
[1/7] ⠈ @prisma/engines
[6/7] ⠁ prisma
[7/7] ⠁ @prisma/client
[-/7] ⠁ waiting...
[3/7] ⠁ sharp
[-/7] ⠁ waiting...
[-/7] ⠐ waiting...
[-/7] ⠐ waiting...
$ node -e "if (process.env.NODE_ENV !== 'production'){process.exit(1)} " || husky install
husky - Git hooks installed
Done in 15.03s.

fine, it's time to enjoy umami 👍

@ChanryLimmy This issue may be related. I would run yarn install --verbose, then troubleshoot from there. It may be the same exact issue.

Same problem, it seems to occur after July 2023

@ChanryLimmy This issue may be related. I would run yarn install --verbose, then troubleshoot from there. It may be the same exact issue.

I try again, umami is not compatible with node 20 LTS, after change node version to 18, it work. But I have another server with node 20... Amazing


Requires Screen Recording Permission for macOS Catalina (10.15) and later

If you want to apply the color filtering to more than just the desktop background in macOS Catalina (10.15) or later, you need to enable the “Screen Recording Permission”. This wasn’t obvious when I first ran it.
For a detailed description, see:
How do I configure Screen Recording permission on macOS Catalina?

So this would be a documentation fix?

On Jan 5, 2021, at 05:09, Jan<EMAIL_ADDRESS>wrote:

If you want to apply the color filtering to more than just the desktop background in macOS Catalina (10.15) or later, you need to enable the “Screen Recording Permission”. This wasn’t obvious when I first ran it.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

So this would be a documentation fix?

On Jan 5, 2021, at 05:09, Jan<EMAIL_ADDRESS>wrote:

If you want to apply the color filtering to more than just the desktop background in macOS Catalina (10.15) or later, you need to enable the “Screen Recording Permission”. This wasn’t obvious when I first ran it.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

In the meantime. I’m evaluating an actual fix. It involves enabling all the security stuff Apple has added in recent years, detecting the permission and either telling the user it’s not set or behaving normally with it set.

In the meantime. I’m evaluating an actual fix. It involves enabling all the security stuff Apple has added in recent years, detecting the permission and either telling the user it’s not set or behaving normally with it set.

#16 should fix this issue.

#16 should fix this issue.

Upgrading to Sonoma (14.0) seem to require re-enabling Screen Recording.
Before upgrading ColorApp was working (I assume Screen Recording was enabled, but can't check now).
After upgrading, ColorApp ran but did not change the screen. Screen Recoding for ColorApp was enabled. (!)
I disabled Screen Recording and then re-enabled it.  I restarted ColorApp. Now it works fine.


Forward changes from v0.5

Rename last commit Merge branch 'release-0.5' into fwd-changes-from-v0.5 to Forward port changes from v0.5


Update plugin to use elastic-enterprise-search instead of elastic-app-search package

The elastic-app-search package is deprecated and won't be updated for 7.11 and onwards.
Instead this project should use elastic-enterprise-search.
cc @picandocodigo

cc @andsel

This request has been moved to logstash-integration-elastic_enterprise_search an addressed in PR https://github.com/logstash-plugins/logstash-integration-elastic_enterprise_search/pull/3


Trouble with --iterative

I can do a clone but can't seem to do a checkout as advised. I see there were some recent edits, there, so I'm hoping there's some other updates that maybe got another wire crossed.

Debian:~$ git clone https://github.com/zmap/zdns.git
Cloning into 'zdns'...
remote: Enumerating objects: 78, done.
remote: Counting objects: 100% (78/78), done.
remote: Compressing objects: 100% (51/51), done.
remote: Total 1684 (delta 33), reused 42 (delta 15), pack-reused 1606
Receiving objects: 100% (1684/1684), 382.38 KiB | 4.66 MiB/s, done.
Resolving deltas: 100% (930/930), done.
Debian:~$ cd zdns
Debian:~/zdns$ go build
go: finding github.com/hashicorp/go-version v1.2.0
go: finding github.com/liip/sheriff v0.0.0-20190308094614-91aa83a45a3d
go: finding github.com/zmap/dns v1.1.28-zmap-3
go: finding github.com/kavu/go_reuseport v1.4.0
go: finding github.com/asergeyev/nradix v0.0.0-20170505151046-3872ab85bb56
go: finding github.com/sirupsen/logrus v1.4.2
go: finding github.com/zmap/go-iptree v0.0.0-20170831022036-1948b1097e25
go: finding golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa
go: finding golang.org/x/crypto v0.0.0-20200117160349-530e935923ad
go: finding golang.org/x/net v0.0.0-20190923162816-aa69164e4478
go: finding golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550
go: finding github.com/stretchr/testify v1.2.2
go: finding github.com/pmezard/go-difflib v1.0.0
go: finding golang.org/x/sync v0.0.0-20190423024810-112230192c58
go: finding golang.org/x/sys v0.0.0-20190924154521-2837fb4f24fe
go: finding golang.org/x/sys v0.0.0-20190422165155-953cdadca894
go: finding golang.org/x/tools v0.0.0-20191216052735-49a3e744a425
go: finding github.com/stretchr/objx v0.1.1
go: finding github.com/davecgh/go-spew v1.1.1
go: finding golang.org/x/sys v0.0.0-20190412213103-97732733099d
go: finding github.com/konsorten/go-windows-terminal-sequences v1.0.1
go: finding golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3
go: finding golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a
go: finding golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2
go: finding golang.org/x/text v0.3.0
go: finding golang.org/x/net v0.0.0-20190620200207-3b0461eec859
go: finding golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee
go: finding golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898
go: downloading github.com/zmap/dns v1.1.28-zmap-3
go: downloading github.com/sirupsen/logrus v1.4.2
go: downloading github.com/liip/sheriff v0.0.0-20190308094614-91aa83a45a3d
go: downloading github.com/hashicorp/go-version v1.2.0
go: downloading golang.org/x/sys v0.0.0-20190924154521-2837fb4f24fe
go: downloading golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa
go: downloading golang.org/x/crypto v0.0.0-20200117160349-530e935923ad
go: finding github.com/miekg/dns v1.1.27

_Then a make install:_

Debian:~/zdns$ make install
cd zdns && go build
go: downloading github.com/zmap/go-iptree v0.0.0-20170831022036-1948b1097e25
go: downloading github.com/asergeyev/nradix v0.0.0-20170505151046-3872ab85bb56
cd zdns && go install

_I get something to work:_

echo "censys.io" | zdns A
{"data":{"answers":[{"answer":"<IP_ADDRESS>","class":"IN","name":"censys.io","ttl":300,"type":"A"},{"answer":"<IP_ADDRESS>","class":"IN","name":"censys.io","ttl":300,"type":"A"}],"protocol":"udp","resolver":"<IP_ADDRESS>:53"},"name":"censys.io","status":"NOERROR","timestamp":"2020-05-23T16:59:51-04:00"}

#########################################################
_But the iterative mode seems to crash, even on a simple example_

Debian:~/zdns$ echo "censys.io" | zdns A --iterative
panic: runtime error: index out of range

goroutine 21 [running]:
github.com/zmap/zdns/cachehash.(*ShardedCacheHash).getShard(0x9c90394, 0x8280920, 0x9d80090, 0x9d80002)
	/home/user/zdns/cachehash/shardedcachehash.go:42 +0x54
github.com/zmap/zdns/cachehash.(*ShardedCacheHash).Lock(0x9c90394, 0x8280920, 0x9d80090)
	/home/user/zdns/cachehash/shardedcachehash.go:72 +0x31
github.com/zmap/zdns/modules/miekg.(*GlobalLookupFactory).GetCachedResult(0x9c90390, 0x9d5a007, 0x2, 0x10002, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/home/user/zdns/modules/miekg/miekg.go:194 +0x1c8
github.com/zmap/zdns/modules/miekg.(*Lookup).cachedRetryingLookup(0x9d6e030, 0x10001, 0x9d5a000, 0x9, 0x82b9bf4, 0xe, 0x82b739e, 0x1, 0x1, 0x0, ...)
	/home/user/zdns/modules/miekg/miekg.go:545 +0x9a3
github.com/zmap/zdns/modules/miekg.(*Lookup).iterativeLookup(0x9d6e030, 0x10001, 0x9d5a000, 0x9, 0x82b9bf4, 0xe, 0x1, 0x82b739e, 0x1, 0x84aeae4, ...)
	/home/user/zdns/modules/miekg/miekg.go:770 +0x205
github.com/zmap/zdns/modules/miekg.(*Lookup).DoMiekgLookup(0x9d6e030, 0x9d5a000, 0x9, 0x82b9bf4, 0xe, 0x81ba81f, 0x8278280, 0x9c7c6e0, 0x82b742d, 0x2, ...)
	/home/user/zdns/modules/miekg/miekg.go:826 +0x2db
github.com/zmap/zdns/modules/miekg.(*Lookup).DoLookup(0x9d6e030, 0x9d5a000, 0x9, 0x0, 0x0, 0x0, 0x9d5a000, 0x9, 0x0, 0x0, ...)
	/home/user/zdns/modules/miekg/miekg.go:916 +0x45
github.com/zmap/zdns.doLookup(0x83125c0, 0x9c90390, 0x9cb20d0, 0x9c6e5c0, 0x9c6e600, 0x9ce9c80, 0x9c82920, 0x0, 0xa6a52060, 0x5)
	/home/user/zdns/lookup.go:132 +0x7d8
created by github.com/zmap/zdns.DoLookups
	/home/user/zdns/lookup.go:205 +0x287

Thanks!

@theloglizard  What's the output of go version?

go version
go version go1.11.13 linux/386

@theloglizard Try building again using a more recent version of Golang (at least 1.13). It doesn't look like your Go version is respecting the replace directive in the go.mod file.

OK, I upgraded to go 1.14.4, deleted the previous builds (I think), and the issue remains.
The make clean was pretty minimal, so I deleted my gocode/bin and gocode/pkg which seemed to force the refetch of some dependencies, which seemed a generally good thing. The integration tests were a considerable disaster (see attached for stderr), and much like what's  below.
.
_
sudo snap refresh  --classic --channel=1.14/stable go
rm -rf ~/gocode/bin
rm -rf ~/gocode/pkg
make install  (from the source directory with everything)
go build (from the zdns sub directory so I could run the  integration tests which expect zdns/zdns)
Checking the one in my path coincides with the date and time I built-- it does.
Debian:~/zdns$ ls -l `which zdns`
-rwxr-xr-x 1 username username 7103083 Jun  6 08:01 /home/username/gocode/bin/zdns
Debian:~/zdns$  date
Sat 06 Jun 2020 08:07:40 AM
Debian:~/zdns$ go version
go version go1.14.4 linux/386
Debian:~/zdns$ echo "censys.io" | zdns A
{"data":{"answers":[{"answer":"<IP_ADDRESS>","class":"IN","name":"censys.io","ttl":300,"type":"A"},{"answer":"<IP_ADDRESS>","class":"IN","name":"censys.io","ttl":300,"type":"A"},{"answer":"<IP_ADDRESS>","class":"IN","name":"censys.io","ttl":300,"type":"A"}],"protocol":"udp","resolver":"<IP_ADDRESS>:53"},"name":"censys.io","status":"NOERROR","timestamp":"2020-06-06T08:07:55-04:00"}
Debian:~/zdns$ echo "censys.io" | zdns A -iterative
panic: runtime error: index out of range [-3624]
goroutine 12 [running]:
github.com/zmap/zdns/cachehash.(*ShardedCacheHash).getShard(...)
/home/username/zdns/cachehash/shardedcachehash.go:42
github.com/zmap/zdns/cachehash.(*ShardedCacheHash).Lock(0xa062394, 0x82bcc80, 0xa09c090)
/home/username/zdns/cachehash/shardedcachehash.go:72 +0x5b
github.com/zmap/zdns/modules/miekg.(*GlobalLookupFactory).GetCachedResult(0xa062390, 0xa090007, 0x2, 0x10002, 0x3, 0x4, 0x0, 0x0, 0x0, 0x0, ...)
/home/username/zdns/modules/miekg/miekg.go:194 +0x1bc
github.com/zmap/zdns/modules/miekg.(*Lookup).cachedRetryingLookup(0xa096030, 0x10001, 0xa090000, 0x9, 0x82f9b0f, 0x10, 0x82f671d, 0x1, 0x1, 0x0, ...)
/home/username/zdns/modules/miekg/miekg.go:545 +0x825
github.com/zmap/zdns/modules/miekg.(*Lookup).iterativeLookup(0xa096030, 0x10001, 0xa090000, 0x9, 0x82f9b0f, 0x10, 0x1, 0x82f671d, 0x1, 0x851b614, ...)
/home/username/zdns/modules/miekg/miekg.go:770 +0xd1
github.com/zmap/zdns/modules/miekg.(*Lookup).DoMiekgLookup(0xa096030, 0xa090000, 0x9, 0x82f9b0f, 0x10, 0x82031ee, 0x82b4800, 0xa0127e0, 0x82f67a8, 0x2, ...)
/home/username/zdns/modules/miekg/miekg.go:826 +0x281
github.com/zmap/zdns/modules/miekg.(*Lookup).DoLookup(0xa096030, 0xa090000, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/home/username/zdns/modules/miekg/miekg.go:916 +0x45
github.com/zmap/zdns.doLookup(0x8361020, 0xa062390, 0xa10a270, 0xa022800, 0xa022840, 0xa13fd80, 0xa016b90, 0x4, 0x200, 0x28)
/home/username/zdns/lookup.go:132 +0x81a
created by github.com/zmap/zdns.DoLookups
/home/username/zdns/lookup.go:205 +0x2a7_
integration.err.txt

I'm not entirely sure what was going on here, but I just used the used the oldest version of the Go toolchain we now support (Go 1.20) and in Debian installed zdns from source. I'm able to run both --iterative and external lookups without getting this issue.
I'm going to close this, if someone else runs into it again we can investigate from there.


Update fs2-core, fs2-io to 3.6.1

Updates

co.fs2:fs2-core
co.fs2:fs2-io

from 3.4.0 to 3.6.1.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Files still referring to the old version number
The following files still refer to the old version number (3.4.0).
You might want to review and update them manually.
website/yarn.lock

Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "co.fs2" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "co.fs2" }
}]

labels: library-update, early-semver-minor, semver-spec-minor, version-scheme:early-semver, old-version-remains, commit-count:1

Superseded by #481.


404 pages returned for non-400 errors

A related issue is that /50x.html pages (esp. as configured via MAINTENANCE_PAGE_URL) appear not to be served correctly on error.
Possibly related: http://stackoverflow.com/a/13624139/1888721
cc: @ianyamey


Grammar Fix of Inline Comment

This could go either way, If there's not a wait function or If there's no wait function. I felt the second read better.

Thanks!


TryWhere()  TrySelectOrFallback()

TryWhere(predicate)
TryWhere should behave as regular Where, but also filter out the items where the predicate throws exception.
Many times a Where clause may be accessing a dictionary, or it may have have a long invocation chain within the item where null checks are impractical and only the "happy case" should have to be considered.
Of course it is undesirable to create a lot of exception, but the benefit in code clarity can often be worth it. This may also be the difference from using a lambda expression to using a named predicates declared elsewhere. Inlined predicates become less readable with a sprinkled if null , if .ContainsKey() and if .Any().

Originally reported on Google Code with ID 74
Reported by tormod.steinsholt on 2012-06-16 12:23:07

I think TryWhere should receive 2 predicates of T: one to filter method and the other to decide if the element should be included or not in the returned sequence.
public static IEnumerable<TResult> TryWhere<TSource, TResult>(
    this IEnumerable<TSource> source,
    Func<TSource, bool> predicate,
    Func<TSource, bool> include)
{
    foreach(var e in source)
    {
        try
        {
            if (predicate(e))
                yield return e;
        }
        catch(Exception e)
        {
            if (include(e))
                yield return e;
        }
    }
}

Catching exceptions in just about every operation that takes a lambda will violate the Single Responsibility Principle and give way to complexity. It will also bloat the library because you could add a TryXXX version for just about every operator.

Many times a Where clause may be accessing a dictionary

It's better to have an extension method for dictionaries that doesn't throw and use that in Where and elsewhere (pun, pun).

or it may have have a long invocation chain within the item where null checks are impractical and only the "happy case" should have to be considered

C# has null-conditional operators since version 6 and those should be used to make null checks practical. One can also resort to using combinators with optional and trial types if C# 6 isn't an option (pun, pun, pun).
I'm closing this as wont-fix.


rename project slug in config file

When i forked legend to try and fix an issue over on ODP https://github.com/finos/open-developer-platform/issues/181
I found the Github Action was not working, i think it was because of the project slug(name) in the config file.
Changing this on my fork allows the action to pass.
You can see the error here: https://github.com/adamj-codethink/legend/pull/1

Well spotted @adamj-codethink , thank you!
Is this the only issue you found? If you fork the legend repo and enable the GitHub Actions, does your preview website - https://adamj-codethink.github.io/legend - work? Right away, or you had to make other changes?
TY

Hi @maoo thanks for the quick review!
I had some trouble, i firstly did not read the docs (my fault), so wasn't aware you needed a gh-pages branch to exist on the forked repo.
However the fix stated in the docs https://odp.finos.org/docs/development-infrastructure/project-documentation/#known-issue is missing a tip that after selecting "gh-pages" as the source you then to select "docs/" as the relevant folder to serve, otherwise you do not see the correct theme.
Does that sounds correct?

Hi @maoo thanks for the quick review!
I had some trouble, i firstly did not read the docs (my fault), so wasn't aware you needed a gh-pages branch to exist on the forked repo.

But the gh-pages branch always existed in your fork (of legend repo) right? That note is for those who fork a repo that have never deployed website contents.

However the fix stated in the docs https://odp.finos.org/docs/development-infrastructure/project-documentation/#known-issue is missing a tip that after selecting "gh-pages" as the source you then to select "docs/" as the relevant folder to serve, otherwise you do not see the correct theme.

Actually, the folder to use is root, but thanks for the note, I've updated https://odp.finos.org/docs/development-infrastructure/project-documentation/#known-issue , wdyt?

Ah yes of course! That would make more sense, i instinctively just create a new branch like adamjones/foo-bar whenever working haha.
It seemed i only saw the legends theme when selecting the "docs/" folder on my branch, but that updated page looks a lot better! Thanks for doing that :+1:


Page setup orientation error

Preflight Checklist

[X] I agree to follow the Code of Conduct that this project adheres to.
[X] I have searched the issue tracker for a feature request that matches the one I want to file, without success.

Describe the bug
In page setup option when I select 16:9 or 16:10 or 4:3 in portrait mode the page orientation is not portraited instead it uses landscape mode and when I use Landscape mode page becomes in portrait mode.
To Reproduce

File > page setup
Select anyone from 16:9, 16:10, 4:3
Select mode as either portrait or landscape.
Click apply

Expected behavior
In portrait mode, the page should be portrait and same for the landscape.
draw.io version (In the Help->About menu of the draw.io editor):

draw.io version [e.g. 13.5.8]

Desktop (please complete the following information):

OS: Linux
Browser: chrome
Version: 83.0.4103.116 (Official Build) (64-bit)

Reproduced in Chrome Version 84.0.4147.105 (Official Build) (64-bit), win 10 Pro and diagrams.net online 13.6.0


Edit settings descriptions

Any chance you can style in the inputs to more closely match the default?

Any chance you can style in the inputs to more closely match the default?

I can't seem to figure out how to change the styles. I've tried code that should change it but everything just looks the same. I'd rather just get it pushed right now and figure that out later

Yeah, that's fine. I can also take a look at it later.


feat: Adds payment method in dashboard screen

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#52  👈
#48
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @jhechtf and the rest of your teammates on  Graphite

Merge activity

May 26, 4:09 PM EDT: @jhechtf started a stack merge that includes this pull request via Graphite.


.gitignore file it's created without .

When creating an app with create-mf-app the .gitignore file it's created without a .

This causes that my project now has 10K in file changes.

I have no problem changing file extension name by myself but it's like a little bug

Which configuration was this? And which operating system? Were there any error messages during the run?

Windows 10.
This is the configuration

Tried again and happened the same

Can you try again with version 1.0.14?

Same

Figured it out, turns out it wasn't Windows specific at all. Should be good now with 1.0.15.

Very much appreciated jack wonderful tool, will keep using it for further projects


htop (new formula)

htop will support OS X in the next release.

We're no longer accepting new HEAD-only formulae (see https://github.com/Homebrew/homebrew-head-only#homebrew-head-only) but we'll be happy to see it in core when there's a release?

htop2 was added to core recently:
https://github.com/Homebrew/homebrew/pull/49057


Cosmos: Errors should be a concrete custom type

Right now all of the cosmos client methods return a Box<dyn Error>. This leads to less than ideal error handling. Ideally it should be a custom type that at least allows you to easily access the HTTP status code.
This may overlap with some of the discussions that have been had around error handling

I really like your code. I think you have written the code this  library should have 👍.
We should return meaningful, in context error responses (like you do returning AlreadyExists, ItemNotFound etc...). We should preserve the HTTP codes for the user to inspect should they like to but they should not be forced to map an HTTP code to the actual explanation.
We could start with an "inner" error like this (thinking out loud):
struct AzureError {
   HTTPRequest request,
   HTTPResponse response,
   Context context, // this should allow the user to know what he was doing, like "Inserting a document on XXX collection"
   string Description
}

And then having the various crates to build on in:
enum InsertEntityFatalError {
   Duplicate(AzureError),
   PrerequisiteFailed(AzureError), // for etag and such
   // and so on...
}

// this is needed to express transient errors
enum InsertEntityTransientError {
   TransientError(AzureError),
   //   and so on...
}

// and wrap them up
enum InsertEntityError {
   InsertEntityFatalError(InsertEntityFatalError),
   InsertEntityTransientError(InsertEntityTransientError)
}

Ideally we should generalize a way to express a "transient" error in the whole library (so the user know they need to retry). Something like:
trait TransientError {}

impl TransientError for InsertEntityTransientError {}

What do you think?

I really like your code. I think you have written the code this  library should have 👍.
We should return meaningful, in context error responses (like you do returning AlreadyExists, ItemNotFound etc...). We should preserve the HTTP codes for the user to inspect should they like to but they should not be forced to map an HTTP code to the actual explanation.
We could start with an "inner" error like this (thinking out loud):
struct AzureError {
   HTTPRequest request,
   HTTPResponse response,
   Context context, // this should allow the user to know what he was doing, like "Inserting a document on XXX collection"
   string Description
}

And then having the various crates to build on in:
enum InsertEntityFatalError {
   Duplicate(AzureError),
   PrerequisiteFailed(AzureError), // for etag and such
   // and so on...
}

// this is needed to express transient errors
enum InsertEntityTransientError {
   TransientError(AzureError),
   //   and so on...
}

// and wrap them up
enum InsertEntityError {
   InsertEntityFatalError(InsertEntityFatalError),
   InsertEntityTransientError(InsertEntityTransientError)
}

Ideally we should generalize a way to express a "transient" error in the whole library (so the user know they need to retry). Something like:
trait TransientError {}

impl TransientError for InsertEntityTransientError {}

What do you think?

I think it may be simpler to consume if we don't have nested enums as you'd end up writing nested match blocks to introspect the error. I don't know all the different error cases for Cosmos, so maybe that enum would be a bit too big, but I would represent the errors in your example as:
struct CosmosError {
  kind: ErrorKind,
  // Any additional human readable info
  message: Option<String>,
  status_code: http::status::StatusCode,
}

enum ErrorKind {
  EntityExists,
  // Contains the prereq that failed
  PrerequisiteFailed(String),
  // When the error was caused by something not related to Cosmos
  AzureError(AzureError),
  Throttled(ThrottleDetails),
  // and so on...
}

impl CosmosError {
  fn new(kind: ErrorKind, message: Option<String>) -> Self {}
  fn is_transient(&self) -> bool {
    match self.kind {
      ErrorKind::AzureError(_) | ErrorKind::Throttled(_) => true,
      _ => false,
    }
}

As you can see in the example, I don't think every variant needs an internal AzureError. The reason being is that some types of errors (like Duplicate) don't really need the extra embedded error. I am not entirely sure we need something to indicate if it is transient or not as each user could handle each type of error differently, but I included it anyway

I think it may be simpler to consume if we don't have nested enums as you'd end up writing nested match blocks to introspect the error. I don't know all the different error cases for Cosmos, so maybe that enum would be a bit too big, but I would represent the errors in your example as:
struct CosmosError {
  kind: ErrorKind,
  // Any additional human readable info
  message: Option<String>,
  status_code: http::status::StatusCode,
}

enum ErrorKind {
  EntityExists,
  // Contains the prereq that failed
  PrerequisiteFailed(String),
  // When the error was caused by something not related to Cosmos
  AzureError(AzureError),
  Throttled(ThrottleDetails),
  // and so on...
}

impl CosmosError {
  fn new(kind: ErrorKind, message: Option<String>) -> Self {}
  fn is_transient(&self) -> bool {
    match self.kind {
      ErrorKind::AzureError(_) | ErrorKind::Throttled(_) => true,
      _ => false,
    }
}

As you can see in the example, I don't think every variant needs an internal AzureError. The reason being is that some types of errors (like Duplicate) don't really need the extra embedded error. I am not entirely sure we need something to indicate if it is transient or not as each user could handle each type of error differently, but I included it anyway

Not all errors are going to be HTTP related. The user might supply an invalid parameter (which we detect before sending to the API), or perhaps the JSON they receive back is not what they're expecting and we cannot parse it for them (i.e., the request itself was correct but the response was simply different than what the user thought it would be)..
I worry we'll end up tying our error too tightly to HTTP.

Not all errors are going to be HTTP related. The user might supply an invalid parameter (which we detect before sending to the API), or perhaps the JSON they receive back is not what they're expecting and we cannot parse it for them (i.e., the request itself was correct but the response was simply different than what the user thought it would be)..
I worry we'll end up tying our error too tightly to HTTP.

That's fine too! It seemed to me that currently the main errors come from http status codes. But if #129 gets fixed, then that will change. If that is the case, I'd embed the underlying http error in the enum variants where it is needed

That's fine too! It seemed to me that currently the main errors come from http status codes. But if #129 gets fixed, then that will change. If that is the case, I'd embed the underlying http error in the enum variants where it is needed

Fixed by #625


YAML configuration files support

Add support for YAML configuration files
Thanks

That's the job of PreviewYaml.

@smittytone I don't see the point of installing a separate app just for YAML files when PreviewCode supports other types of configuration files like: .conf, .cf, .cfg, .ini, .rc
I guess it's because you need to monetize your work. It is not a criticism.
Still, thanks for making this App open source


Is the windows compiled version supposed to be "2019.09.01" from the releases page instead of "2019.11.22" ?

Checklist

[x ] I'm asking a question
[ ] I've looked through the README and FAQ for similar questions
[ ] I've searched the bugtracker for similar questions including closed ones

Question

Is the windows compiled version supposed to be "2019.09.01" from the releases page instead of "2019.11.22" ?
https://github.com/ytdl-org/youtube-dl/releases/download/2019.11.22/youtube-dl.exe
youtube-dl --version
2019.09.01
Algorithm       Hash                                                                   Path

SHA256          580F74BC64F2D73FB53BE2A1A1185D9426631479DCC4941B144DCA0F351FB920

It's supposed to be 2019.11.22 and it is.


Remove the tooltip style brightness filter

This style plugins/plugin-client-common/web/scss/components/Tooltip/PatternFly.scss (line: 68) contains brightness filter
In dark themes, this style makes the text unreadable.
Thanks.

Howdy, thanks for the bug report! To help with tracking this, could you attach a screenshot at your convenience? Thanks!

Hello. On my win10 kui looks like this:

Thanks! 👍

Thank you. Looks good, but there is still a small "problem" in displaying the tooltip on a button with a question mark


sellatresistance selling at little to no profit.

Hi,
I have recently turned on sellatresistance.
Since then my ETH-GBP bot has made a single buy and sell but for whatever reason it only achieved a 0.13% profit.

I though it was meant to make atleast 1% before it sold?
Config is as follows:
"config": {
"base_currency": "ETH",
"quote_currency": "GBP",
"live": 1,
"sellatloss": 0,
"disablebullonly": 1,
"disablebuyobv": 1,
"disablebuyelderray": 1,
"disablebuynearhigh": 1,
"sellatresistance": 1,
"disablelog": 1,
"disabletracker": 1
}

I had a this too. Some did well.  Seems that the timeframe should shorten when that triggers if they start tanking quickly. @whittlem is that possible?  To change granularity to 5 or even 1 min when sellatresitance triggers?

Hello,
I also noticed that with sellatresistance=1, it sells very close to the buying price and sometimes it sells lower than the buying price even with sellatloss=0 when it goes with anegative margin and come back to positive that's what's happened to me today.
Config as below:
"config" : {
            "base_currency" : "ETH",
            "quote_currency" : "EUR",
            "live" : 1,
	    "sellatloss": 0,
	    "trailingstoploss": -2,
	    "disablebullonly" : 1,
            "disablebuyobv" : 1,
            "disablebuyelderray" : 1,
	    "sellatresistance": 1,
	    "disablebuynearhigh": 1,
	    "disablelog": 1,
	    "disabletracker": 1
 }

Here is my logs:
2021-05-05 18:28:09 | ETH-EUR | 900 | Current Price: 2825.47
! Profit Bank Triggered (Selling At Resistance)

2021-05-05 16:30:00 | ETH-EUR | 900 | Close: 2843.51 | *^ EMA12/26: 2796.52 > 2792.29 ^* | *^ MACD: 4.23 > 0.91 ^* | SELL | Last Action: BUY | 0.59% (delta: 26.72)
support: 2820.91, resistance: 2906.44, fibonacci (l): 2785.03, fibonacci (u): 2929.7

 2021-05-05 16:30:00 | ETH-EUR 900 | Close: 2843.51 | SELL
 Fibonacci Retracement Levels: {'ratio0_286': 2785.03, 'ratio0': 2929.7}

ETH balance before order: 0.5305
EUR balance before order: 0.0
{'product_id': 'ETH-EUR', 'type': 'market', 'side': 'sell', 'size': 0.5305}
ETH balance after order: 0.0
EUR balance after order: 1494.23
2021-05-05 18:38:13 | ETH-EUR | 900 | Current Price: 2835.27
2021-05-05 18:43:14 | ETH-EUR | 900 | Current Price: 2836.1
2021-05-05 16:45:00 | ETH-EUR | 900 | Close: 2860.0 | ^ EMA12/26: 2804.75 > 2796.57 ^ | ^ MACD: 8.18 > 2.38 ^ | WAIT | Last Action: SELL
2021-05-05 18:53:16 | ETH-EUR | 900 | Current Price: 2860.3

Here we see that the bot want to sell at €2843.51 but when I see my order history on Coinbase Pro it sold at €2,826.54 so this make a 0.59% difference.
I think maybe it's needed to make limit orders or to reduce timeframe.

@JShezz, the margin calculation in the bot seems under calculated. I plan to look at it this evening. It should not sell under 1% but as it's miss calculating the margin too low it's selling too low.
@djhemp, can look at reducing the polling. The reason I set it to 5 is a lot of API calls happen each time it polls. I didn't want the exchange to start dropping requests if multiple bots are running. I will reduce it to 3 minutes and see how that goes.
@iThom, I know the resistance level or Fibonacci band can be very close to the buy price sometimes. That is why I said it should only sell if the margin is over 1%. The problem really is the margin is being calculated too low. I will look at this tonight.
I had a couple of buys and sell today which sold at resistance for profit but small amounts sub-1%. I haven't had any losses but it's still not ideal. I'll see what I can do.

Added --lastaction BUY or --lastaction SELL
This will allow you to manually set the starting state, otherwise it will figure it out like it is doing now.

@whittlem Did you close the wrong Issue?

It seems so :)

The way the bot is calculating the margin now is wrong. I've been updating it this evening and created a new branch as I want to test it properly myself live before releasing it to all of you. I also noticed that the sell at resistance option wasn't actually checking for a minimum margin. I've added that now so I suggest you pull the update down. This should resolve this issue but I'll leave it open until I've fixed the margin as well.

I've fixed this and pushed and update. Can you confirm it has resolved your issue?

@whittlem It is still possible to get a -1% profit as it doesn't appear to take into account the fee when selling.
E.g. I just had a trade sell at 1.08% meaning margin was actually only 0.85% due to the 0.35% fee

Hello,
I confirm problem with fees calculation with last version of the bot.
I recently had a trade with sellatresistance on (with bug not selling at once in this case it was good for me as the price was even bigger when selling).
Margin calculated by the bot (should be removing fees): 4.51%
Real margin fees removed: 4.35%
Diff: 0.16% (my Coinbase Pro fees are currently 0.35%)

I increased the minimum margin from 1% to 2%. Did this help?

No response in 25 days. Closing this.


ddev delete needs to clean up ddev-global-cache sections

ddev delete doesn't currently clean up things in ddev-global-cache volume that pertain to individual projects. For example, subdirectories of nvm_dir, bash_history, mysqlhistory.

removing the volume with docker volume rm ddev-global-cache wipes the global cache volume. You will lose composer cache and other global ones.

Is ddev delete the same as ddev stop --remove-data and if not, shouldn't ddev stop --remove-data remove this project specific global cache?

Yes, ddev delete is the same as ddev stop --remove-data and the code that should be updated here is the same code used by both. ddev stop --remove-data was an older way to do this.


Demo

Thanks for sharing your work. can you provide the correct commend to test the model on image and video please. I test with:
python demo_vita/demo.py  --video-input clips/0d40b015f4.mp4  --output out/
but I just have a video without segmentation and this line in the terminal:
<EMAIL_ADDRESS>global obsensor_uvc_stream_channel.cpp:156 getStreamChannelGroup Camera index out of range
Thanks in advance.

Hey @assia855 Have you figured out the problem of running the demo on video?

I suppose that's due to opencv environment conflict. Could you please elaborate?


Add example cloud function + search

@zeroasterisk I think it would make sense to change the folder name to just cloud-function or something like that.
Also, be sure to run the black formatter.
From the repository root on your local machine, run:
nox -s blacken

Closing due to inactivity. Please re-open when the comments are answered.


Added option --objc-base-lib-include-prefix

Extracted from #28 and based on the latest master for easier review

Pushed as 86e6e0d39cf4723df25aff135f205fab87ddbdff


PDFTable section headers and footers

Is it possible to have table section headers and footers? I've tried merging cells for a row, but I keep getting index errors. I'm probably not adding my section heading properly though. I want the section heading to span across all columns for a single row.
Something like this:

sectionsContent.append(section)
let tableRow = table[row: row]
tableRow.merge()
tableContent.append(sectionsContent)

section is just a String. There could be any number of columns, so just getting the row and merging all the columns seemed like the thing to do. Is this the wrong approach for getting section headers and footers into a table? Once I get the section headers part working, then I'll focus on the section footers.

Just to clarify about the problem I'm encountering, the content that I'm adding to my Table looks like this:
▿ 3 elements
  ▿ 0 : 4 elements
    - 0 : "Name"
    - 1 : "Artist"
    - 2 : "Album"
    - 3 : "Genre"
  ▿ 1 : 1 element
    - 0 : "2"
  ▿ 2 : 4 elements
    - 0 : "22 Years"
    - 1 : "Styx"
    - 2 : "The Serpent Is Rising!"
    - 3 : "Rock"```

The first section are the field names. The second section (index 1) is the header. Yes, just the number 2 at the moment. The third section is the data record. So the issue seems to be with the second section (index 1) because the data only spans 1 column.

Do I need to pad the sections with empty values to make column spanning work?

Ok, I've figured this all out. Yes it seems you do have to pad the row so that every column has a value before you merge the row. So I just set the row to contain the same value as the first column for all the columns. Then I can merge without getting index range errors.
So I'm going to close this now.


typescript definition

is possible to add typescript definition?

http://npmjs.com/package/@types/sdp-transform

I should have checked this before doing https://github.com/clux/sdp-transform/tree/typescript.
BTW who does maintain those sdp-transform typescript definitions?
I think we can close this issue since having to install @types/xxxxx is a common pattern, although I'll modify my PR to just document it.