Deploy "hello world" contract to Evmos

The first goal for Evmos is to be able to accept an extremely simple EVM contract. This will provide some validation that the networking and storage of the various components is working as expected.
A "hello world" style contract that writes and reads from state should be sufficient
contract Storage {

   uint256 number;

   /**
    * @dev Store value in variable
    * @param num value to store
    */
   function store(uint256 num) public {
       number = num;
   }

   /**
    * @dev Return value
    * @return value of 'number'
    */
   function retrieve() public view returns (uint256){
       return number;
   }
}

[ ] https://github.com/celestiaorg/evmos/issues/14
[ ] https://github.com/celestiaorg/optimint/issues/295
[ ] https://github.com/celestiaorg/optimint/issues/294
[ ] https://github.com/celestiaorg/optimint/issues/291

Current state is that simple peer-to-peer transactions work. However, when you attempt to retrieve the transaction with something like foundry's cast cast tx $tx_hash we hit this line in ethermint.

The first issue with retrieving the transaction is that the transaction is submitted with the following tendermint Query.Conditions

conditions: []query.Condition{query.Condition{CompositeKey:"ethereum_tx.ethereumTxHash", Op:0x4, Operand:"0x23375d55471fec45a7dfdf5b2152f083bd665a39cd80ca86c33b083053838251"}}

where the CompositeKey uses Ethermint's evmtypes.TypeMsgEthereumTx and evmtypes.AttributeKeyEthereumTxHash but the Optimint lookForHash function was checking for Tendermint's types.TxHashKey.
I resolved this manually which got the GetTransactionByHash call to reach Optimint's store.BadgetKv.Get call. The internal call to txn.Get(key)  seems to be panicking:

t=2022-01-31T23:19:32+0000 lvl=eror msg="RPC method eth_getTransactionByHash crashed: key not found\ngoroutine 924 [running]:\ngithub.com/ethereum/go-ethereum/rpc.(*callback).call.func1()\n\tgithub.com/ethereum/go-ethereum@v1.10.11/rpc/service.go:200 +0x89\npanic({0x1f17880, 0xc0005931f0})\n\truntime/panic.go:1038 +0x215\ngithub.com/celestiaorg/optimint/state/txindex/kv.(*TxIndex).Get(0xc00013cd30, {0xc008d6cc80, 0x236a191, 0x40})\n\tgithub.com/celestiaorg/optimint@v0.0.0-20220118134333-5bdc6d3b8868/state/txindex/kv/kv.go:52 +0x2a8\ngithub.com/celestiaorg/optimint/state/txindex/kv.(*TxIndex).Search(0x2920ac0, {0x296f568, 0xc000054050}, 0xc0097c2e58)\n\tgithub.com/celestiaorg/optimint@v0.0.0-20220118134333-5bdc6d3b8868/state/txindex/kv/kv.go:205 +0x44c\ngithub.com/celestiaorg/optimint/rpc/client.(*Client).TxSearch(0xc001b0edb0, {0x296f568, 0xc000054050}, {0xc001b59020, 0x5f}, 0xfe, 0xc00937cc80, 0x0, {0x0, 0x0})\n\tgithub.com/celestiaorg/optimint@v0.0.0-20220118134333-5bdc6d3b8868/rpc/client/client.go:422 +0x21d\ngithub.com/tharsis/ethermint/rpc/ethereum/backend.(*EVMBackend).GetTxByEthHash(0xc00880f000, {0x23, 0x37, 0x5d, 0x55, 0x47, 0x1f, 0xec, 0x45, 0xa7, ...})\n\tgithub.com/tharsis/ethermint@v0.8.0/rpc/ethereum/backend/backend.go:740 +0x1e2\ngithub.com/tharsis/ethermint/rpc/ethereum/backend.(*EVMBackend).GetTransactionByHash(0xc00880f000, {0x23, 0x37, 0x5d, 0x55, 0x47, 0x1f, 0xec, 0x45, 0xa7, ...})\n\tgithub.com/tharsis/ethermint@v0.8.0/rpc/ethereum/backend/backend.go:657 +0x11e\ngithub.com/tharsis/ethermint/rpc/ethereum/namespaces/eth.(*PublicAPI).GetTransactionByHash(0xc0087352c0, {0x23, 0x37, 0x5d, 0x55, 0x47, 0x1f, 0xec, 0x45, 0xa7, ...})\n\tgithub.com/tharsis/ethermint@v0.8.0/rpc/ethereum/namespaces/eth/api.go:658 +0x105\nreflect.Value.call({0xc0088b9d40, 0xc000011a28, 0x7f9a190105b8}, {0x2244108, 0x4}, {0xc0086fe0a0, 0x2, 0xc008c9ac80})\n\treflect/value.go:556 +0x845\nreflect.Value.Call({0xc0088b9d40, 0xc000011a28, 0x5429c8}, {0xc0086fe0a0, 0x2, 0x3})\n\treflect/value.go:339 +0xc5\ngithub.com/ethereum/go-ethereum/rpc.(*callback).call(0xc0088b9da0, {0x296f530, 0xc0013c1140}, {0xc000053470, 0x18}, {0xc0097c2e10, 0x1, 0x500917})\n\tgithub.com/ethereum/go-ethereum@v1.10.11/rpc/service.go:206 +0x3f0\ngithub.com/ethereum/go-ethereum/rpc.(*handler).runMethod(0xc00954b360, {0x296f530, 0xc0013c1140}, 0xc008d745b0, 0x23, {0xc0097c2e10, 0x0, 0x203001})\n\tgithub.com/ethereum/go-ethereum@v1.10.11/rpc/handler.go:389 +0x45\ngithub.com/ethereum/go-ethereum/rpc.(*handler).handleCall(0xc008711a70, 0xc007cd5d70, 0xc008d745b0)\n\tgithub.com/ethereum/go-ethereum@v1.10.11/rpc/handler.go:337 +0x239\ngithub.com/ethereum/go-ethereum/rpc.(*handler).handleCallMsg(0xc008711a70, 0xc000600000, 0xc008d745b0)\n\tgithub.com/ethereum/go-ethereum@v1.10.11/rpc/handler.go:298 +0xd1\ngithub.com/ethereum/go-ethereum/rpc.(*handler).handleMsg.func1(0xc007cd5d70)\n\tgithub.com/ethereum/go-ethereum@v1.10.11/rpc/handler.go:139 +0x35\ngithub.com/ethereum/go-ethereum/rpc.(*handler).startCallProc.func1()\n\tgithub.com/ethereum/go-ethereum@v1.10.11/rpc/handler.go:226 +0xc5\ncreated by github.com/ethereum/go-ethereum/rpc.(*handler).startCallProc\n\tgithub.com/ethereum/go-ethereum@v1.10.11/rpc/handler.go:222 +0x92\n"

There is an issue when making a Type Assertion on  the msg in the Block to an Ethermint evmtypes.MsgEthereumTx here. This failing type assertion gives us false for ok and nil for ethMsg.
The type assertion is from sdk.MsgSend to evmtypes.MsgEthereumTx where
sdk => github.com/cosmos/cosmos-sdk/types
evmtypes => github.com/tharsis/ethermint/x/evm/types

https://gist.github.com/jbowen93/6e811954cac6aa634e123dfd12ab6321

To get passed errors returned due to lack of support for Validators I've commented out the relevant section. See https://github.com/jbowen93/ethermint/commit/76a1739de78b94cf832ad6692cf3de716c2e4e62

By manually setting the gas here: https://github.com/jbowen93/ethers-rs/commit/e79f4b6ee9ac5da3a91811e0fc2c6bc19d41e9eb
I'm able to get a successful contract deployment.
method: "eth_getTransactionReceipt"
params: "[\"0xc45f96457d1619effb3217c38586ff14f88f6bc68cc3b8baaffb8813c2ff9ca7\"]"
Deployer: 0x0a62b19c95cc8f6099ecac4bc8009ff8f94f1175
Deployed to: 0x72f92b82575c8032314f96c75698ade48f1f0d85
Transaction hash: 0xc45f96457d1619effb3217c38586ff14f88f6bc68cc3b8baaffb8813c2ff9ca7

The contract (shown above) doesn't seem to be actually storing data on the store call.
Logs from cast call $ADDR "retrieve()"
10:45PM INF ABCIQuery data=0AFD017B2266726F6D223A22307830303030303030303030303030303030303030303030303030303030303030303030303030303030222C22746F223A22307837326639326238323537356338303332333134663936633735363938616465343866316630643835222C22676173223A6E756C6C2C226761735072696365223A6E756C6C2C226D6178466565506572476173223A6E756C6C2C226D61785072696F72697479466565506572476173223A6E756C6C2C2276616C7565223A6E756C6C2C226E6F6E6365223A6E756C6C2C2264617461223A2230783265363463656331222C22696E707574223A6E756C6C2C226163636573734C697374223A5B5D7D10C0F0F50B module=events path=/ethermint.evm.v1.Query/EthCall result={"code":0,"codespace":"","height":"248","index":"0","info":"","key":null,"log":"","proofOps":null,"value":"CkIweDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAaIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAKI62AQ=="}

Logs from cast call $ADDR "store(uint256)" 1
10:46PM INF ABCIQuery data=0ABD027B2266726F6D223A22307830303030303030303030303030303030303030303030303030303030303030303030303030303030222C22746F223A22307837326639326238323537356338303332333134663936633735363938616465343866316630643835222C22676173223A6E756C6C2C226761735072696365223A6E756C6C2C226D6178466565506572476173223A6E756C6C2C226D61785072696F72697479466565506572476173223A6E756C6C2C2276616C7565223A6E756C6C2C226E6F6E6365223A6E756C6C2C2264617461223A223078363035373336316430303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303634222C22696E707574223A6E756C6C2C226163636573734C697374223A5B5D7D10C0F0F50B module=events path=/ethermint.evm.v1.Query/EthCall result={"code":0,"codespace":"","height":"250","index":"0","info":"","key":null,"log":"","proofOps":null,"value":"CkIweDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAo/NMC"}

Closing this because contract deployment works.
Contract calls also work, I just needed to use cast send not cast call. See https://github.com/celestiaorg/evmos/issues/22


Title Updates

Okay so i've added the options hide_title and custom_title to the query string.
I tried to implement hide_title as a boolean (like area) but for some reason that caused the build to fail.. so i've implemented it as a true/false string.
In the utils file i've added the appropriate logic to update the title based on these arguments.
I've also updated the README to include these new features.
Let me know if you have any feedback. If you like what i've done I have a few more ideas and could put in more pull requests.
Thanks,
Jacob

I have a few more ideas and could put in more pull requests.

Sure, just raise an issue first for the requested feature and then go for it 🚀


Support importing/pasting/dropping audio files

We just added support for more file types when you attach/paste/drop them.  We also have support for turning audio into text, see (src/lib/speech-recognition.ts).  Let's add support for importing audio files, which converts the audio to text and includes it in a new message.

@AryanK1511 it's all yours

@humphd, just to confirm my understanding of this issue:
Are we aiming to add the ability to attach, paste, or drop audio files into the chatbot, have their content transcribed into text, and then include that text in a new message? For example, if I attach a pre-recorded audio file saying, "Tell me about the universe," the chatbot would transcribe it and include the text as part of the conversation.
Is that correct?

Correct.  It should work for the audio file types that we can natively send to the LLMs.  Later we could add a step to transcode if necessary, but let's not start with that.
So if I have a podcast MP3, I can attach this file and chat with it (the transcript will be injected into the chat for me).

I see! So for the user, they will attach a file, chat with it and then when they hit send, they will see the transcript and their prompt in the chat right?

Hi @humphd,
After spending a few hours digging into this codebase, I managed to fix the issue. However, the solution I’ve come up with feels quite hacky. I’d like to explain my approach, why I believe the current structure makes a clean solution difficult, and seek your guidance on how to improve it.
Background
The issue involves handling audio files similarly to how PDFs are processed in the src/hooks/use-file-import.tsx file. Here's the current flow for PDFs:

The following snippet converts PDF files to text via an external API and returns the content:
if (file.type === "application/pdf") {
    const contents = await pdfToMarkdown(file);
    assertContents(contents);
    return contents;
}

Later in the same file, the content is added to the chat:
else if (file.type === "application/pdf") {
    const document = (contents as JinaAiReaderResponse).data;
    chat.addMessage(new ChatCraftHumanMessage({ text: `${document.content}\n` }));
}

My Approach for Audio Files
To implement audio file handling, the ideal approach would be to mimic the PDF workflow by creating an audioToText function:
if (file.type.startsWith("audio/")) {
    const contents = await audioToText(file);
    assertContents(contents);
    return contents;
}

However, since the project already includes functionality for transcribing audio in src/lib/speech-recognition.ts, I leveraged the transcribe method:
async transcribe(audio: File): Promise<string> {
    const transcriptions = new OpenAI.Audio.Transcriptions(this._openai);
    const transcription = await transcriptions.create({
        file: audio,
        model: this._sttModel,
    });
    return transcription.text;
}

The Problem
Using transcribe requires creating an instance of the SpeechRecognition class, which depends on a model and client:
constructor(sttModel: string, openai: OpenAI) {
    this._sttModel = sttModel;
    this._openai = openai;
}

This initialization is done in src/components/PromptForm/MicIcon.tsx via the useModels hook:
const { getSpeechToTextClient, isSpeechToTextSupported, allProvidersWithModels } = useModels();
const sttClient = await getSpeechToTextClient();
const sttProvider = allProvidersWithModels.find((p) => p.apiUrl === sttClient.baseURL);
const sttModel = sttProvider?.models.find((model) => isSpeechToTextModel(model.name))?.name;
speechRecognitionRef.current = new SpeechRecognition(sttModel, sttClient);

The issue is that hooks like useModels can only be used in React components, whereas the file-processing logic resides outside of React components.
Current Solution
Currently, I bypass the clean integration by doing the following:

Returning an empty string for audio files in use-file-import.tsx:
if (file.type.startsWith("audio/")) {
    return "";
}

Handling audio files downstream in the React component:
else if (file.type.startsWith("audio/")) {
    const sttClient = await getSpeechToTextClient();
    const sttProvider = allProvidersWithModels.find((p) => p.apiUrl === sttClient.baseURL);
    const sttModel = sttProvider?.models.find((model) => isSpeechToTextModel(model.name))?.name;
    const sr = new SpeechRecognition(sttModel, sttClient);
    const text = await sr.transcribe(file);
    chat.addMessage(new ChatCraftHumanMessage({ text: `${text}\n` }));
}

This approach works, as shown in the attached video.
https://github.com/user-attachments/assets/25886dc4-7d13-402f-8d5e-07ddd8bb8f5e
However, it introduces two significant problems:

The file processing component never gets triggered:const progressId = progress({
    title: `Processing file${files.length > 1 ? "s" : ""}`,
    progressPercentage: 0,
});

The logic for audio file handling is fragmented and not aligned with the existing structure.

Request for Guidance
I believe the current structure of the code makes it difficult to implement audio file handling in a clean way. How would you suggest proceeding?
Would you like me to open a PR with my current changes so we can work collaboratively on refining the solution? Note that I’ve made a few other changes to ensure the input works as expected.
Thank you for your guidance!

Yeah, that's not ideal.  We are going to need to refactor the logic for dealing with AI out of hooks, so background processes like this can do it as well.
For now, let's make this work with OpenAI and whisper, per https://platform.openai.com/docs/guides/speech-to-text.  You can use settings.ts to call getSettings() and see if you have an OpenAI API Key stored.  If you do, you can show the audio files as an option, otherwise, don't bother.  Then pass that key and an OpenAI client instance to SpeechRecognition.
I'd make this work at all, then we can figure out how to make it work in general.  To be honest, the TTS stuff is kind of separate from our other Chat model handling, so we should probably extract it out similar to what we're doing with Jina.ai.
cc'ing @Amnish04, who might have other thoughts.

Sounds good to me 🫡

@humphd Everything works perfectly now and the code is very consistent too. You can have a look at the PR that I just sent in and lemme know if everything looks good


Rank under Messages sent on my User profile not updating (for months)

If you go to my user profile here: https://xqc.pajbot.com/user/aubycek , you can see that I am Rank 473 under Messages sent. "Messages sent" alone works perfectly, but my Rank has been stuck on this particular number (473) for several months now (6+months). Don't know about other people, but I'm pretty sure it's broken at least for me.

Might be related to #970 or #966

Sounds interesting

Entirely intentional.


[WIP] Fix TreeBuilder i18n

Old TreeBuilder conversions used N_ and PostponedTranslation, but since then we've settled on directly doing the translating with _.
Fixing the remaining..
(WIP because this was originally a part of #13272, and depends on it.)
also TODO: app/presenters/tree_builder_alert_profile.rb:31:    text = PostponedTranslation.new(N_("%{model} Alert Profiles")) do
and make the commits nice :)

Moved to https://github.com/ManageIQ/manageiq-ui-classic/pull/17, closing.


Flickering prompt string and window resize in tiling WM

Which operating system does the issue occur on?
Ubuntu 16.04 amd64

If on linux, are you using X11 or Wayland?
X11

If compare with gnome-terminal - cmd promt flickering much more often.
On gif in attach this is badly noticeable, but in fact the gnome terminal blinks once a second if you hold enter.
Also flickering when window resize (user WM bspwm).
It's possible to reduce flickering?
So that would be visually faster:

Launch (the window is drawn for a long time. Just like the gnome-terminal speed)
Less flickering everiwhere. With daily seating in the console (12h work) - it's very annoying
Do not blink while resizing the window

Also filelist flickering on simple ls -lha command:

Hello, it looks like the flickering is a duplicate of #869, this should be fixed with winit 0.15.1, so make sure you're up to date!
The other flickering might be related to the redraw-rate of Alacritty, if I had to take a guess Alacritty is redrawing while the text is being written, which causes a perceived "flickering".


Color class correction

Signed-off-by: randybrunopiverger<EMAIL_ADDRESS>Regarding: https://redhat-internal.slack.com/archives/CU4QXLPQB/p1676060559580869

/cherry-pick release-2.7


Fix incorrect import path that causes failure with metadata

Task linked: CU-861n3mnch metadata generates incorrect manifest

coverage: 91.69% (-0.2%) from 91.91% when pulling 6add6da0eb845ce27f156b25d478f7cb45a6d36b on CU-861n3mnch into 3d4c62fcce59f21bb3754b46c5097d055e172a28 on development.


3.0.7 still verbose for reconnections after Miele-side disconnect

Hello Sander,
I saw this morning you published the 3.0.7 version to reduce the amount of logs due to reconnection.
The previous issue is closed, but I am sorry to report that for some reason, what should be "ServerConnectionLost" as a reason for reconnection is seen as "SelfInitiated" hence the logs continue to flow here :-( (I put it in Debug mode, it shows that the connection is correctly detected lost from Miele side (lines marked with '*') but the reconnection reason is wrong and therefore logged as "info" instead of "debug" (lines marked with '**').
[29/03/2021 11:24:41] [Miele] Discovered devices:  [
  'mac-001D63FFFE0203B2',
  '000127334912',
  'mac-001D63FFFE0203B4',
  '000116035470',
  'mac-00124B0005AB4D8B'
]
[29/03/2021 11:24:41] [Miele] Discovered device: id: mac-001D63FFFE0203B2, name: Four, model: Unknown Miele model
[29/03/2021 11:24:41] [Miele] Ignoring unsupported device. Device: "Four" with raw type value: 12.
[29/03/2021 11:24:41] [Miele] Discovered device: id:<PHONE_NUMBER>12, name: Taque, model: Unknown Miele model
[29/03/2021 11:24:41] [Miele] Ignoring unsupported device. Device: "Taque" with raw type value: 14.
[29/03/2021 11:24:41] [Miele] Discovered device: id: mac-001D63FFFE0203B4, name: Lave-vaisselle, model: Unknown Miele model
[29/03/2021 11:24:41] [Miele] Re-creating existing accessory with data from cache: Lave-vaisselle
[29/03/2021 11:24:41] [Miele] Discovered device: id:<PHONE_NUMBER>70, name: Lave-linge, model: Unknown Miele model
[29/03/2021 11:24:41] [Miele] Re-creating existing accessory with data from cache: Lave-linge
[29/03/2021 11:24:41] [Miele] Discovered device: id: mac-00124B0005AB4D8B, name: Frigo, model: Unknown Miele model
[29/03/2021 11:24:41] [Miele] Re-creating existing accessory with data from cache: Frigo
[29/03/2021 11:24:41] [Miele] Lave-vaisselle: Initial connection with Miele event server successfully established.
[29/03/2021 11:24:41] [Miele] Lave-linge: Initial connection with Miele event server successfully established.
[29/03/2021 11:24:41] [Miele] Frigo: Initial connection with Miele event server successfully established.
[29/03/2021 11:24:41] [Miele] Lave-vaisselle: Event 'device' received.
[29/03/2021 11:24:41] [Miele] Lave-vaisselle: Updating characteristic Active to 1.
[29/03/2021 11:24:41] [Miele] Lave-vaisselle: Remaing Duration update received: 0[s]
[29/03/2021 11:24:41] [Miele] Frigo: Event 'device' received.
[29/03/2021 11:24:41] [Miele] Frigo: Updating characteristic CurrentHeatingCoolingState to 2.
[29/03/2021 11:24:41] [Miele] Frigo: Updating characteristic TargetHeatingCoolingState to 2.
[29/03/2021 11:24:41] [Miele] Frigo: Zone 1 Current Temperature update received: 400[C/100]
[29/03/2021 11:24:41] [Miele] Frigo: Updating characteristic CurrentTemperature to 4.
[29/03/2021 11:24:41] [Miele] Frigo: Zone 1 Target Temperature update received: 400[C/100]
[29/03/2021 11:24:41] [Miele] Frigo: Updating characteristic TargetTemperature to 4.
[29/03/2021 11:24:41] [Miele] Lave-linge: Event 'device' received.
[29/03/2021 11:24:41] [Miele] Lave-linge: Remaing Duration update received: 0[s]
[29/03/2021 11:24:42] [Miele] Lave-vaisselle: Returning 1 for Active.
[29/03/2021 11:24:42] [Miele] Lave-vaisselle: Returning 0 for InUse.
[29/03/2021 11:24:42] [Miele] Lave-vaisselle: Returning 0 for RemainingDuration.
[29/03/2021 11:24:42] [Miele] Frigo: Returning 2 for CurrentHeatingCoolingState.
[29/03/2021 11:24:42] [Miele] Frigo: Returning 2 for TargetHeatingCoolingState.
[29/03/2021 11:24:42] [Miele] Frigo: Returning 4 for CurrentTemperature.
[29/03/2021 11:24:42] [Miele] Frigo: Returning 4 for TargetTemperature.
[29/03/2021 11:24:42] [Miele] Frigo: Returning 0 for TemperatureDisplayUnits.
[29/03/2021 11:24:42] [Miele] Lave-linge: Returning 0 for Active.
[29/03/2021 11:24:42] [Miele] Lave-linge: Returning 0 for InUse.
[29/03/2021 11:24:42] [Miele] Lave-linge: Returning 0 for RemainingDuration.
*[29/03/2021 11:27:10] [Miele] Frigo: Miele event server connection lost. Auto-reconnect after 60[s]
*[29/03/2021 11:27:10] [Miele] Lave-linge: Miele event server connection lost. Auto-reconnect after 60[s]
*[29/03/2021 11:27:10] [Miele] Lave-vaisselle: Miele event server connection lost. Auto-reconnect after 60[s]
[29/03/2021 11:27:28] [Miele] Lave-vaisselle: Returning 1 for Active.
[29/03/2021 11:27:28] [Miele] Lave-vaisselle: Returning 0 for InUse.
[29/03/2021 11:27:28] [Miele] Lave-vaisselle: Returning 0 for RemainingDuration.
[29/03/2021 11:27:28] [Miele] Frigo: Returning 2 for CurrentHeatingCoolingState.
[29/03/2021 11:27:28] [Miele] Frigo: Returning 2 for TargetHeatingCoolingState.
[29/03/2021 11:27:28] [Miele] Frigo: Returning 4 for CurrentTemperature.
[29/03/2021 11:27:28] [Miele] Frigo: Returning 4 for TargetTemperature.
[29/03/2021 11:27:28] [Miele] Frigo: Returning 0 for TemperatureDisplayUnits.
[29/03/2021 11:27:28] [Miele] Lave-linge: Returning 0 for Active.
[29/03/2021 11:27:28] [Miele] Lave-linge: Returning 0 for InUse.
[29/03/2021 11:27:28] [Miele] Lave-linge: Returning 0 for RemainingDuration.
**[29/03/2021 11:28:10] [Miele] Lave-vaisselle: Self initiated reconnect with Miele event server successful.
**[29/03/2021 11:28:10] [Miele] Lave-linge: Self initiated reconnect with Miele event server successful.
**[29/03/2021 11:28:10] [Miele] Frigo: Self initiated reconnect with Miele event server successful.

It is like the line "this.reconnectReason = ReconnectReason.ServerConnectionLost" is overwritten with "this.reconnectReason = ReconnectReason.SelfInitiated;", possibly at the "setTimeout()" definition ?
Maybe one way is to move the "this.reconnectReason = ReconnectReason.SelfInitiated;" line from the setTimeout() definition to higher in the code, as an initialisation ?
I tried like this (modified lines tagged with "// RMI"):
        // Error handling.
        this.eventSource.onerror = (error) => {
            var _a;
            (_a = this.eventSource) === null || _a === void 0 ? void 0 : _a.close();
            const errorObj = error;
            this.reconnectReason = ReconnectReason.SelfInitiated; // RMI
            // If Miele closed the connection on their end, EventSource raises an empty error object.
            if (!errorObj.status) {
                this.platform.log.debug(`${this.accessory.context.device.displayName}: Miele event server ` +
                    `connection lost. Auto-reconnect after ${settings_1.EVENT_SERVER_RECONNECT_DELAY_S}[s]`);
                this.reconnectReason = ReconnectReason.ServerConnectionLost;
            }
            else {
                this.platform.log.error(`${this.accessory.context.device.displayName}: Error received from Miele event server. ` +
                    `Status: ${errorObj.status}. Message: '${errorObj.message}'`);
                this.mainService.setCharacteristic(this.platform.Characteristic.StatusFault, this.platform.Characteristic.StatusFault.GENERAL_FAULT);
                this.platform.log.info(`${this.accessory.context.device.displayName}: Will attempt to reconnect to the Miele event server after` +
                    ` ${settings_1.EVENT_SERVER_RECONNECT_DELAY_S}[s].`);
                this.reconnectReason = ReconnectReason.Error;
            }
            setTimeout(() => {
                                  // RMI
                this.connectToEventServer();
            }, settings_1.EVENT_SERVER_RECONNECT_DELAY_S * 1000);
        };

This modification works (reconnection due to Miele side disconnect is now in debug and correctly labeled as such), but I have no idea if this modification has no side effect as I do not have any other issue happening leading to another type of disconnection.

Thanks for your suggestion, I will need to move it to the constructor in setInterval (where I also intended it but misplaced it in onerror :-( ). To prevent it will otherwise never get the SelfInitiated value due to the else-branch.
I maybe should have just gone for the simple solution, all messages as debug, but that is another discussion ;-)

:tada: This issue has been resolved in version 3.1.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

Thanks Sander, so far so good ;-)


Change checkboxes and radios to validate on click

This library has made the choice to validate inputs on blur; this is a totally reasonable choice for normal text inputs as it stops potential off-putting switching between validity states as the user types. For radios and checkboxes though there’s no typing to be done, yet the library still waits for a blur event (i.e. a click somewhere else or a tab away) before updating the visual validity state.
This commit adds a click event handler for radios and checkboxes that will evaluate the validity immediately after the browser changes the internal state of the control. This provides immediate feedback to the user that they’ve corrected the problem.
Example without this change (this is particularly bad as it’s using a custom form control that is basing its styling on validate’s state):

Example with this change:

Good call. Definitely the way to go. Thanks!


3.1.8 doesn't launch

I double click the exe, get the popup for allowing the exe to run anyway and then... nothing.
Running as admin shows nothing either.

ok launching rebirthitemtracker.exe is not the correct way to launch it. :P


Clarify Authentication documentation

The Authentication doc seems a bit outdated.
I think it would be good to list each method from Spotify Authorization's documentation and explain for each if they are supported (or planned to be).
rspotify does exactly that which I found very helpful.

The Authentication doc seems a bit outdated.

Yes, I am afraid this is true for most of the documentation.

I think it would be good to list each method from Spotify Authorization's documentation and explain for each if they are supported (or planned to be).

Unless someone is working on getting OAuth flows working (I think there was an effort by librespot-java), the current authentication scheme is the only one without any others planned by me I'm afraid.
Feel free to update any docs you find!

So you confirm that none of the 4 officials OAuth flows are currently supported?
I may be interested implementing the Authorization code flow as that will allow me to share the token between librespot and rspotify. Do you foresee any issue? This should work with the API implemented by librespot right?
That or I re-implement the catalog search using librespot as discussed here. I'm not sure yet which way is best.

I cannot say with certainty as I have never tried myself.
But it is what I gather also from the unfinished effort over at https://github.com/librespot-org/librespot-java/pull/446.
I have some doubts whether tokens gotten through OAuth will be valid to authenticate on Mercury. They might be. But they (web vs. Mercury) may also well be different contexts. I remember someone theorising that one certain OAuth scope might also work on Mercury. I can't remember where I read that... probably also at the Java port or maybe on Gitter.

Hi, I've been looking recently into registering the client into my account using an access token, I would like to share what I discovered in case it's interesting to someone, this project has helped me a lot https://github.com/TimotheeGerber/spotify-connect. It seems to be posible to authenticate a speaker using an api token as long as it has the "streaming" scope. I have tried to modify the tool to force it to use the same token I'm using with the web SDK (an access token obtained through oauth) and when using the tool auth method default-token everything seems to work.
# in my test I have modified the change_to_token_credentials method to use my token and confirm that I can register the speaker using an api token.
./spotify-connect -a default-token $DISCOVERY_IP $DISCOVERY_PORT

Auth method access-token exposed by the library doesn't seem to work with librespot.
Tested agains librespot v0.4.2.
Regards!

Maybe I misunderstand what you are saying. Is it that you have one token, which includes the "streaming" scope (and presumably a load others also) and that one token can be used with the Web API and librespot? Assuming that's the right understanding, where did you get that token from?

The same token I use when my web application relays on the Web Playback SDK, I only use librespot for the electron version.
A token obtained using this flow https://developer.spotify.com/documentation/general/guides/authorization/code-flow/ with an application client id liked to my account.
Yes I have another scopes on the token, but I think based on the tool code that only the "streaming" one is required. I think it's enough for Librespot to authenticate to the Spotify account and swap to a reusable credentials.
But I have little knowledge about Librespot and the protocols it uses, just wanted to shared want I found in case someone wants to look more on it.

As a personal opinion, I think that having the https://github.com/TimotheeGerber/spotify-connect functionalities exposed in someway by librespot itself could be awesome, this way you can use same binary to authenticate the speaker through command line, and even, exposing a manual authentication page should be easy to build. I'm telling all this without knowing if authenticating with the access token have some side effect, but for me doesn't seem to be the case.
Regards!

Ahh, excellent! I didn't realise you could legitimately get the streaming scope as I have never used the Web Playback SDK. And your modification was to change the client ID to your own?
I wonder if you do need some extra scopes for some parts of librespot to work.

Ahh, excellent! I didn't realise you could legitimately get the streaming scope as I have never used the Web Playback SDK. And your modification was to change the client ID to your own?

I changed the whole line that call get_token by a literal string containing a token generated by my app backend.
I tested it with and without this modification and both times it worked. And without the modification the token is just generated the token with the streaming scope.

I wonder if you do need some extra scopes for some parts of librespot to work.

I think that not, because I think that the token that you are providing is not used by librespot anymore after the authentication, it generate it's own credentials and from those it is able to generate new tokens (using other client ids, so I think the client id you generated the token with is also not used at all).

OK, with you now. Sorry, it's been a while.
Yes and you're right, librespot does it's own fresh login afrer decrypting the blob sent by that spotify-connect tool. It does seem like there are some useful alt login methods that could be documented. i.e. from username and Web API "streaming" token. That could be a useful method to have if you want to share the same token.
It looks like supporting "accesstoken" login in librespot's might just be a case of fixing (https://github.com/librespot-org/librespot/blob/dev/discovery/src/server.rs#L111)[handle_add_user] to check the "tokenType" response param and skipping all the decryption and just return Credentials::with_access_token(username,blob) in that case.

@GiviMAD : thanks a lot! I manually created a Credentials with AuthenticationType::AUTHENTICATION_SPOTIFY_TOKEN and was able to play medias using my GStreamer element!
I faced an issue when trying to request lyrics from librespost though. librespot::metadata::Lyrics::get get a MercuryResponse with status_code: 403.
Maybe another scope is required but I didn't find any looking at the doc as they all seem related to user specific infos.

There is a request for allowing the Device Authorization Grant flow on the Spotify forum - it is very useful for embedded usecases like a remote control.

Oauth is implemented and documented now.


Admin page keeps loading + error on Query log page

This admin panel got installed with pi-hole. The said i should ask it here, because they didn't make the admin panel.
The admin page keeps loading: https://s12.postimg.org/4p40hworh/screen.png
The error on the Query log page: https://s15.postimg.org/cy56k42vv/screen2.png
Let me know if you need logs

The log was to big.
Running "/usr/local/bin/pihole flush"  did the job (cleans the log file)


Setting CheckBox.IsChecked does not update UI with .NET Framework 4.6.1, working on .NET Core

Repro using Menu page in ControlCatalog:
https://gist.github.com/wieslawsoltes/be3fcf4bccaed1aa21eafd0cb38b21b8
Try toggling items in Debug menu using Desktop build and NetCore build of ControlCatalog.
Version: 0.5.1-build3602-alpha

I have seen something similar with RadioButton but not Framework related, I get the same behaviour on .net 461 and netcore2.0
https://files.gitter.im/AvaloniaUI/Avalonia/R7Wb/scenegraph-bug.gif

and upload a repro soon

@grokys I have seen the issue also on .NET Core but on different machine (with slower CPU, running Win7 instead of Win10) and in Release build (reported issue was in Debug mode).

I can also confirm deferredrender: true / false does not make a difference.

My guess would be that it does the render pass before the state has updated.
If you trigger another full render (i.e. by window resize) it then renders correctly (for me anyway)

@wieslawsoltes' problem seems to be caused by the fact that the CheckBox is in a Popup and it's not getting updated correctly.
@danwalmsley you said you'd upload a repro? I suspect your problem might be different.


Corrupted metadata JSON files caused by bug #297

The bug #297 frequently leads to corrupted JSON files when multiple instances mount a shared NFS directory as storage. We’ve encountered numerous cases in production where this causes certificate failures with the error message decoding certificate metadata: invalid character '}' after top-level value, rendering the affected sites completely unusable.
The number of corrupted files is increasing, and I can’t restart the service.
I see the bug fix is in the master branch. Please release a new version that includes this fix. Additionally, how can I identify and repair the already corrupted files among thousands without deleting all files? Alternatively, is there a way to ignore corrupted files during loading?

If you're in a hurry, you can make a build yourself with this:
xcaddy build --with github.com/caddyserver/certmagic@16e2e0b

Curious what evidence you have that this and #297 are related -- how do you know that superfluous ARI requests is corrupting files?

I’m using certmagic as a library for a web server that serves front-end files and supports custom domains. When the error occurred, I checked the logs and file modification times and found that an extra ‘}’ appeared after multiple ARI updates. After reviewing the source code, I suspect this issue is due to concurrent writes to NFS.
Prior to the error, I was already preparing to migrate from NFS to S3 and implemented a custom storage to access S3. After the error, I expedited the migration process. Since S3 writes are atomic, it prevented the issue, even though redundant ARI requests still remain.

NFS has known bugs related to synchronization, that might be the actual problem.
S3 does not provide atomic operations for us to be able to safely offer synchronization, even if writes are synced. I recommend using a database like MySQL/Postgres/Redis for high concurrency distributed storage.

Moving another discussion with @Zenexer into here:

As far as I can tell, this isn't fixed by 16e2e0b3443037882be32c731d1e85a90cb69014.  I'm able to repro the extra } error reliably--multiple times per hour--regardless of the lengths of the original and new files, so it's not just a simple truncation issue.  It's always one extra }, even though file sizes often differ by more than one character.
Removing the extraneous closing braces restores sanity, but only briefly.  They keep reappearing in new files.

Originally posted by @Zenexer in #297

Anyway, @lqs, from what you're saying:

Using NFS, files have extra }
Using S3, files don't have extra } but still redundant ARI requests still happen

This actually checks out with known issues with both of those storage backends (as noted just above). NFS has sync/flush issues when it comes to concurrent users over a network; and S3 doesn't provide atomic operations, so proper locking/syncing of an operation like an ARI request is impossible.
@Zenexer, are you also using NFS perchance?

I spent about a dozen hours debugging this yesterday, and I believe my initial comment was incorrect: rather than the bug persisting, I believe I just hadn't sufficiently cleaned all of the existing corrupt files. There were situations in which there were two trailing bytes at the end of a file (\n}), and my cleanup script didn't account for that.
I am using NFS, but it does appear to support locking correctly with my current mount options--or, at least, in a way that is compatible with this patch. I'm not a huge fan of NFS and generally don't trust it, but it should work with this lock/write pattern. I doubt it would ever make sense to officially support NFS given how fickle it is, but the locking code in certmagic is straightforward enough that I should be able to debug and patch it if there are further issues.
The one thing that still has me a little worried is a disconcerting number of requests to the on-demand ask endpoint. The docs make it sound as though that's to be expected, but it's accompanied by a large number of log entries related to ARI. I don't think it's a bug--it's probably just a coincidence--but I happened to notice it while troubleshooting.

That's good to hear! Yours was the only feedback so far that it didn't fix the issue, so it's reassuring that it was an oversight.

That's a relief, thanks for the follow-up.

The one thing that still has me a little worried is a disconcerting number of requests to the on-demand ask endpoint. The docs make it sound as though that's to be expected, but it's accompanied by a large number of log entries related to ARI. I don't think it's a bug--it's probably just a coincidence--but I happened to notice it while troubleshooting.

The ask endpoint can be busy... we could potentially ease this with a bloom filter or something, that we just reset every 5 or 10 minutes (or something like that). But ideally I'd rather the ask endpoint itself do the caching since it knows better logic than we can guess.
I'd be curious if the ARI log entries are redundant (same hostname) or not. I really want that to be fixed (AFAIK it should be already).

The ask endpoint can be busy... we could potentially ease this with a bloom filter or something, that we just reset every 5 or 10 minutes (or something like that). But ideally I'd rather the ask endpoint itself do the caching since it knows better logic than we can guess.

I would assume that any Caddy user with that sort of traffic probably has caching on their ask endpoint anyway and can keep it fast.  From my perspective, though, I'd like to be able to log exactly when I've told Caddy it was authorized to go out and request a certificate: having that logging on my application helps me with troubleshooting, since I can use that to determine where in the stack a problem is arising that might be leading to excessive certificate requests.  As it currently stands, I don't know whether an ask is for a certificate request/renewal, an ARI request, or some other maintenance task being performed by Caddy--it's a black box.

I'd be curious if the ARI log entries are redundant (same hostname) or not. I really want that to be fixed (AFAIK it should be already).

I'll try to figure that out, but Caddy is spitting out gigabytes of log data with on-demand TLS enabled, so I'm still trying to sort out what's important and what's not.

I think most of the log entries are the result of various hosting providers and registrars trying to request or renew certificates for domains that no longer point to them, with no checking on their end prior to starting the ACME challenge process.  That makes it really difficult to tell the difference between legitimate ACME-related error messages and errors that I can safely ignore.  (Ugh, I really wish CAs wouldn't waive rate limiting on challenge failures for large integrators--it hurts everyone.)  I don't think that should affect ARI log messages, so I'll let docker compose logs -fn0 | grep -F ARI and see what turns up.

I'm not seeing any overlap between ARI requests so far.  Each one is unique.

I'd like to be able to log exactly when I've told Caddy it was authorized to go out and request a certificate; having that logging on my application helps me with troubleshooting, since I can use that to determine where in the stack a problem is arising that might be leading to excessive certificate requests. As it currently stands, I don't know whether an ask is for a certificate request/renewal, an ARI request, or some other maintenance task being performed by Caddy--it's a black box.

To make sure I understand, you want a way for the 'ask' request to distinguish whether a certificate is being obtained or something else?
The only times the 'ask' endpoint is invoked are currently when a certificate needs to be obtained or renewed. It does not guard ARI requests or other maintenance, per-se, though in theory it should be guarding them implicitly, because if you cannot obtain or renew a cert, you cannot maintain it either.
Technically, 'ask' is invoked before even trying to load a certificate from storage (as that can be expensive depending on the storage backend).
So I guess, to your request, I would say that it shouldn't matter, but I'm open to discussing this further if desired.

I think most of the log entries are the result of various hosting providers and registrars trying to request or renew certificates for domains that no longer point to them, with no checking on their end prior to starting the ACME challenge process. That makes it really difficult to tell the difference between legitimate ACME-related error messages and errors that I can safely ignore. (Ugh, I really wish CAs wouldn't waive rate limiting on challenge failures for large integrators--it hurts everyone.) I don't think that should affect ARI log messages, so I'll let docker compose logs -fn0 | grep -F '"updated ACME renewal information"' and see what turns up.

I was one who advocated for exemptions to the rate limits when conforming to ARI, out of concerns that certificate renewals would be rejected -- sometimes past their expiration -- on account of rate limits, even though it was the CA who specified the renewal window. So to ensure certificates can be renewed even if they have to be squished into a narrow window, Let's Encrypt (rightly) exempts clients from rate limits in that situation. Why do you think it hurts everyone?
Is Caddy attempting to renew lots of certificates for you and failing?

I'm not seeing any overlap between ARI requests so far. Each one is unique.

That's good, so it sounds like the synchronization is working. :+1:

To make sure I understand, you want a way for the 'ask' request to distinguish whether a certificate is being obtained or something else?

Yes, mostly for debugging purposes. If I see that multiple Caddy instances are all trying to get certs at the same time, that's a sign something is amiss. It would likely help when troubleshooting future concurrency issues, but a lightweight plugin could probably serve the same purpose.
I'm still not 100% confident the remaining errors I'm seeing are benign, but I'll have more data over the next few days.

I was one who advocated for exemptions to the rate limits when conforming to ARI, out of concerns that certificate renewals would be rejected -- sometimes past their expiration -- on account of rate limits, even though it was the CA who specified the renewal window. So to ensure certificates can be renewed even if they have to be squished into a narrow window, Let's Encrypt (rightly) exempts clients from rate limits in that situation. Why do you think it hurts everyone?

Sorry, I realized I forgot to answer this question. I don't have an opinion on the scenario you mentioned. What appears to be happening is twofold:

foo.example used to point to , but now points to me. Said hosting provider doesn't even bother to check whether the domain points to them before trying to renew their certificates. It might not have pointed to them for months, and they just don't have any reason to bother cleaning up.
Vulnerability scanners are hitting /.well-known/acme-challenge/*

Access logs have since shown that the second issue accounts for the majority of the "no challenge data found" warnings I was seeing. Disabling HTTP-01 mostly resolved that.
The first issue is far more problematic: I can't stop other hosting providers from requesting certs, and they're wasting PKI resources. It also wastes my time because they cause concerning log entries. These large hosting providers don't really have any incentive to check whether hosts point to them before requesting a cert; they just offload that to the CA.
Meanwhile, I have to check that a domain points to me--and only to me--when Caddy hits the ask endpoint. Caddy's docs explicitly say I shouldn't do this, but I can't go around requesting certs for domains just because I think they might point to me. I have to actually verify that, then cache the results for a while. If I don't do that, I'll get rate limited pretty quickly.

@Zenexer It sounds like your ask endpoint needs to check to make sure you are expecting to be getting a certificate for those domain names, that should solve both 1 and 2.

I have to check that a domain points to me--and only to me--when Caddy hits the ask endpoint.

Not exactly; why not check your database (or whatever is relevant to your application/service) to see if you should be expecting to maintain a certificate for a hostname? That's the purpose of the ask endpoint and it should solve both problems, yeah?

It sounds like your ask endpoint needs to check to make sure you are expecting to be getting a certificate for those domain names.

It already does that.  I am expecting to get certificates for those domains.
Scenario 1: I control example.com.  Large hosting provider used to control example.com, and they used to have certificates for it.  They keep trying to renew those certificates.  My ask endpoint needs to return 200, because I need certificates.
Scenario 2: I control example.com.  Someone runs Acunetix against example.com.  For whatever reason, it starts brute forcing random paths with a prefix of /.well-known/acme-challenge/--for example, /.well-known/acme-challenge/xmlrpc.php.  I want certificates for example.com, so the ask endpoint returns 200.

Not exactly; why not check your database (or whatever is relevant to your application/service) to see if you should be expecting to maintain a certificate for a hostname? That's the purpose of the ask endpoint and it should resolve the rate limit problems, yeah?

I do.  However, I'm running a free service to which non-technical users can point their domains.  They might use my service for a while, decide they don't like it, and point their domain elsewhere.  I can't trust that the users are going to maintain a list of domains that point to me, so I do have to validate the A/AAAA records before requesting a certificate.  The result gets cached in Redis for a while, so subsequent calls to the ask endpoint are very fast.

I think to simplify this, what @Zenexer is stating is that a third party actor can initiate a request to /.well-known/acme-challenge/$token - and caddy will hit the ask endpoint even if caddy knows that the challenge is bogus, or the challenge is real but caddy can't solve it (e.g, it was a token that caddy does not know about).
I believe for security reasons caddy already needs to keep a track of the list of tokens its generated through applying for ACME challenges. Checking for a string in a list of strings is probably significantly faster to do first, before sending a request to the ask endpoint (even if its not recommended for the ask endpoint to take long, its still a request to a third party thing).
Would it make sense to flip the order of operations here? Caddy does the first initial sanity check (e.g. "I don't even know what this token is, trash it?)
I guess if you have a very slow storage driver, then maybe that operation will be slower... but maybe my naive view is that the storage driver is likely to be faster than the ask endpoint the vast majority of the time, and if the ask request returns a 200 then you're still going to need to hit the storage driver anyway.

I believe for security reasons caddy already needs to keep a track of the list of tokens its generated through applying for ACME challenges.

It does, but that's kept in the storage. What Matt is saying is that in some setups, the storage lookup is more expensive than the ask lookup.
Caddy can be run in a cluster, so it must use the storage to see whether another Caddy instance initiated issuance. It can't rely on an in-memory cache.
The ask endpoint should only do a DB lookup, it should not have side effects. If it has side effects, it's incorrectly implemented.

The ask endpoint should only do a DB lookup, it should not have side effects. If it has side effects, it's incorrectly implemented.

That's impossible. It has to do DNS lookups at any sort of scale, despite the documentation. There's just no use case for on-demand TLS that doesn't necessitate frequently double-checking DNS. Calling it incorrect isn't helpful.

You should not be doing DNS checks. That doesn't make sense. It's not your ask endpoint's responsibility to do that, it's Caddy's. You should have an allow list of domains in your database that you compare against.

That's impossible. It has to do DNS lookups at any sort of scale

Not really, this fully depends on the use case defined here imo. In your use case, you're going to need to do a DNS lookup, but that's not necessarily universally applicable.

That's what it was originally until about a year ago, storage was checked first, but that was bad for some users.

Yeah that makes sense tbh.
Maybe the ability to choose at which stage ask is performed can be useful?

Not really, this fully depends on the use case defined here imo.

I'm having a hard time envisioning common use cases for on-demand TLS in which the operator of Caddy has exclusive ownership and control over all of the domain names pointed to it.
The obvious use case seems to be a hosting provider or integrator. They need to regularly verify that any domains provided by end users actually, truly point to them before requesting certificates. Caching that information for too long is risky, especially since DNS is prone to misconfiguration.

I'm having a hard time envisioning common use cases for on-demand TLS in which the operator of Caddy has exclusive ownership and control over all of the domain names pointed to it.

Company setting where they have one global load balancer, but a secondary system checks that a domain is owned by the company before being put into a given database.
This is different from your use case where you let arbitrary users point their domains to your infrastructure with no registration/signup process (e.g. you don't know whats even linked to you until you get a request!)
Both of these use cases should work IMO

@Zenexer What you should do is have your customers register their domain via your app's settings, and you add it to your DB allow list. Then all the ask endpoint does is compare against that list. That's how it's meant to work.

@Zenexer’s use case allows people to point their domains to their infrastructure without any registration, etc.
From what I’m understanding, there is no registration or direct user involvement. The only involvement is going to DNS and changing name servers or A records.

@Zenexer What you should do is have your customers register their domain via your app's settings, and you add it to your DB allow list. Then all the ask endpoint does is compare against that list. That's how it's meant to work.

That's the erroneous assumption that leads to issue 1. Hosting providers assume that just because a domain is registered with them, they will pass DV.
I don't make that assumption. Hosting providers often do, which is why we're here. Caddy sees ACME challenges that were started by other hosting providers who don't bother to check DNS before requesting a cert.

I don't understand. Your ask endpoint would reject it so Caddy wouldn't issue a cert.

I don't understand. Your ask endpoint would reject it so Caddy wouldn't issue a cert. I don't see how what "hosting providers" do matters here.

Hosting Provider A runs Caddy with on-demand TLS.
Hosting Provider B also runs Caddy with on-demand TLS.

Alice owns example.com.
Alice points example.com to Hosting Provider A.
Hosting Provider A tries to get a cert for example.com.
Hosting Provider A's ask endpoint returns 200.
Let's Encrypt issues a cert to Hosting Provider A.
Alice changes example.com's DNS to point to Hosting Provider B.
Alice doesn't tell Hosting Provider A that example.com has been moved.
Caddy at Hosting Provider A tries to renew its cert.
Hosting Provider A's ask endpoint sees example.com in its database and returns 200.
Caddy at Hosting Provider A goes to Let's Encrypt and starts the challenge process.
Let's Encrypt attempts to complete the HTTP-01 challenge by making a request to http://example.com/.well-known/acme-challenge/something.
Caddy at Hosting Provider B receives this challenge request.
Caddy at Hosting Provider B receives a 200 response from its ask endpoint.
Caddy at Hosting Provider B checks for challenge data in its storage, but finds nothing. It logs a warning.

This happens dozens of times per second. I'm Hosting Provider B in this scenario.

I don't understand. Your ask endpoint would reject it so Caddy wouldn't issue a cert. I don't see how what "hosting providers" do matters here.

There is no external database to check in this circumstance. There is no UI. There is no app.
There is a page with instructions: if you want to park your domain with ${whatever}, please point your A and AAAA records to ${whatever}.
The database is DNS.
The answer here may be that this is not a supported use case of caddy, but imo this can very easily be a supported use case with slight modifications.

Just catching up after feeding the baby and running some kids around... sorry!
I've been drafting this reply while several new replies have come in -- I wish GitHub would show that someone was replying or at least show the new replies. I feel like the conversation went off-track and got confused by some things, but maybe I did instead. In any case, here's my attempt to bring it back:
@Zenexer
Thanks for clarifying above. It seems to me you have an extraordinary situation that is not common from what I know of existing large-scale CertMagic deployments. That's not bad, just something that is worthy of discussion/understanding.

Scenario 1: I control example.com. Large hosting provider used to control example.com, and they used to have certificates for it. They keep trying to renew those certificates.

So, in this case, the hosting provider would fail their own ACME challenge, but your server would likely get pinged with a TLS handshake or HTTP request in an attempt to solve the challenge.
The HTTP-01 challenge does not use TLS, so those would not issue a certificate. You'd see junk in your logs, but what's new (it's the Internet).
The TLS-ALPN-01 challenge does use TLS, but with a special ALPN value. When it sees a handshake of this sort, it only follows a special code path that serves the challenge solution certificate (if it doesn't find one it just returns an error and aborts the handshake).
Neither case will initiate an ACME challenge that you end up failing and getting rate limited for. (If they are, that's a bug I'd like more details, likely in a separate issue.)

Scenario 2: I control example.com. Someone runs Acunetix against example.com. For whatever reason, it starts brute forcing random paths with a prefix of /.well-known/acme-challenge/--for example, /.well-known/acme-challenge/xmlrpc.php. I want certificates for example.com, so the ask endpoint returns 200.

@aaomidi

I think to simplify this, what @Zenexer is stating is that a third party actor can initiate a request to /.well-known/acme-challenge/$token - and caddy will hit the ask endpoint even if caddy knows that the challenge is bogus, or the challenge is real but caddy can't solve it (e.g, it was a token that caddy does not know about).

The ask endpoint is only invoked if the client tries to establish a TLS handshake using a domain name it does not have a certificate for (and isn't itself a challenge handshake). Are the servers accessing this plaintext endpoint over HTTPS? That would be the only way this is possible, but is extremely broken / in violation of spec.

The ask endpoint is only invoked if the client tries to establish a TLS handshake using a domain name it does not have a certificate for (and isn't itself a challenge handshake).

That seems to run contrary to what was said elsewhere in this thread: it seems that the ask endpoint is being hit prior to checking storage for the existence of a certificate. If that's not the case, there might still be a concurrency issue somewhere that needs debugging.

Catching up on the flood of new replies while I was typing... and also revisiting some of the earlier replies...
@Zenexer

The first issue is far more problematic: I can't stop other hosting providers from requesting certs, and they're wasting PKI resources.

This should only be negatively affecting them (and I guess the CAs, which is why they rate limit them). If they initiate an ACME challenge for a domain pointed to you, you will see junk log entries at most. Or what wasted PKI resources are you referring to?

I do. However, I'm running a free service to which non-technical users can point their domains. They might use my service for a while, decide they don't like it, and point their domain elsewhere. I can't trust that the users are going to maintain a list of domains that point to me, so I do have to validate the A/AAAA records before requesting a certificate.

I should have mentioned that this is what I was referring to that is a bit unconventional.  If setting DNS records is itself the act of "signing up" with your service, then it makes sense that you have to check DNS records. This is the only time I have heard of this being the case... and for a few reasons (complexity, reliability, etc) I don't recommend this... unless, perhaps, you do like what our Caddy homepage demo does, where a user can point a specific subdomain ("caddydemo" in our case) to your IP, and then no signup is required. This still prevents abuse because it only works for one specific identifier per registered domain, hence there's a relatively significant cost barrier to abuse it.

I don't make that assumption. Hosting providers often do, which is why we're here. Caddy sees ACME challenges that were started by other hosting providers who don't bother to check DNS before requesting a cert.

To clarify once more, if someone else initiates an ACME challenge for a hostname that fails repeatedly, that doesn't rate limit you. It only rate limits their ACME account, not yours.
@aaomidi (and of course @francislavoie) Thanks for chiming into this discussion. Your feedback and perspectives are much appreciated!!
@aaomidi

I believe for security reasons caddy already needs to keep a track of the list of tokens its generated through applying for ACME challenges. Checking for a string in a list of strings is probably significantly faster to do first, before sending a request to the ask endpoint (even if its not recommended for the ask endpoint to take long, its still a request to a third party thing).
Would it make sense to flip the order of operations here? Caddy does the first initial sanity check (e.g. "I don't even know what this token is, trash it?)

When CertMagic initiates an ACME challenge, it puts the challenge info in storage so that any other instances in the cluster can solve the challenge (as opposed to just in process memory). So when a challenge request comes in, we don't know whether it's junk until we access storage.

Beyond that there's also the problem that ask is operating with far, far less information than caddy is. I suppose the ask request can hook into the same storage and check for the token being there or not. But for that to work, the token would also need to be part of the ask payload.

This is true, and we could add more info here; however, I've tried to keep the abstraction as pure as possible with the understanding that it shouldn't matter: is this identifier known to / allowed by the server or not? In theory, I don't think other information should matter. But I'm open to exploring this more.
@Zenexer

The ask endpoint is only invoked if the client tries to establish a TLS handshake using a domain name it does not have a certificate for (and isn't itself a challenge handshake).

That seems to run contrary to what was said elsewhere in this thread: it seems that the ask endpoint is being hit prior to checking storage for the existence of a certificate. If that's not the case, there might still be a concurrency issue somewhere that needs debugging.

Ah, I think I see where the confusion is. What I said here is true, and doesn't preclude what I said earlier. 'Ask' is consulted before checking to see if storage has a certificate to satisfy the TLS handshake (I'm talking about normal handshakes here, not TLS-ALPN-01 challenge handshakes which use only a very specific, minimal code path). If 'ask' returns 200, storage is checked. If storage returns no cert, then an ACME challenge is initiated to obtain one. Does that make sense?

@Zenexer  Sorry, one more thing:

Caddy at Hosting Provider A goes to Let's Encrypt and starts the challenge process.
Let's Encrypt attempts to complete the HTTP-01 challenge by making a request to http://example.com/.well-known/acme-challenge/something.
Caddy at Hosting Provider B receives this challenge request.
Caddy at Hosting Provider B receives a 200 response from its ask endpoint.

Step 13 is not (or should not be) the case. A request over HTTP does not invoke the ask endpoint because no certificate is required because there is no TLS handshake to complete. In other words, this part does not invoke or utilize on-demand TLS at all.
So, if my understanding is correct, your main concern is spammy log entries?

This should only be negatively affecting them (and I guess the CAs, which is why they rate limit them). If they initiate an ACME challenge for a domain pointed to you, you will see junk log entries at most. Or what wasted PKI resources are you referring to?

To be clear, the only reason I'm bringing up any of this is because it's made debugging this concurrency issue difficult. If everything worked perfectly as described all the time, it would just be a matter of ignoring the logs--the same as I was already doing.
It's only a problem because I can't tell which log entries and ask requests are a result of my Caddy instances trying to get certs (or encountering concurrency issues), versus which log entries are from someone else's lazy misuse of ACME.
By "wasting PKI resources," I'm referring to the fact that DV isn't free, especially at scale. There's an expectation that integrators should make a reasonable effort to ensure a domain actually points to them (and that an ACME challenge will succeed) prior to trying to get a cert.

I should have mentioned that this is what I was referring to that is a bit unconventional.  If setting DNS records is itself the act of "signing up" with your service, then it makes sense that you have to check DNS records. This is the only time I have heard of this being the case... and for a few reasons (complexity, reliability, etc) I don't recommend this... unless, perhaps, you do like what our Caddy homepage demo does, where a user can point a specific subdomain ("caddydemo" in our case) to your IP, and then no signup is required. This still prevents abuse because it only works for one specific identifier per registered domain, hence there's a relatively significant cost barrier to abuse it.

I think we're getting a bit caught up with my specific use case, which isn't too relevant here. Let's use the example I provided in a recent comment with two hosting providers.

To clarify once more, if someone else initiates an ACME challenge for a hostname that fails repeatedly, that doesn't rate limit you. It only rate limits their ACME account, not yours.

Correct. I'm not being rate limited and am not proposing that as a concern.
If I follow the instructions in the docs and stop performing DNS checks, then I become Hosting Provider A in my example, and I will get rate limited.

Ah, I think I see where the confusion is. What I said here is true, and doesn't preclude what I said earlier. 'Ask' is consulted before checking to see if storage has a certificate to satisfy the TLS handshake (I'm talking about normal handshakes here, not TLS-ALPN-01 challenge handshakes which use only a very specific, minimal code path). If 'ask' returns 200, storage is checked. If storage returns no cert, then an ACME challenge is initiated to obtain one. Does that make sense?

Yes, that was my initial impression. That design strongly disincentivizes ask endpoints from performing sufficient checks. That leads to an ecosystem problem as Caddy becomes more popular.
It invites Caddy users to put themselves in the position of Hosting Provider A above.

@aaomidi

There is no external database to check in this circumstance. There is no UI. There is no app. The database is DNS.

I just want to reiterate. This is not okay to do. The problem is that any bad actor can point a wildcard domain to your server and then infinitely make requests like a.badguy.com then b.badguy.com to infinity, making your server issue a cert per domain, resulting in a DDOS (rate limits getting hit, or storage exhaustion). That why you must have an allow-list, you cannot trust DNS because it's not under your control.

By "wasting PKI resources," I'm referring to the fact that DV isn't free, especially at scale. There's an expectation that integrators should make a reasonable effort to ensure a domain actually points to them (and that an ACME challenge will succeed) prior to trying to get a cert.

Caddy's internal rate limiting should prevent that from being a concern. It slows itself down if too many attempts are happening.

If I follow the instructions in the docs and stop performing DNS checks, then I become Hosting Provider A in my example, and I will get rate limited.

No, because only a TLS handshake that actually reaches your server (meaning the DNS is already configured correctly) can trigger On-Demand TLS. Any domain which doesn't point to your server will not renew when the time comes because only a TLS handshake would trigger a renewal attempt.
I've never seen any evidence of the usecase you're describing being a real issue.

Step 13 is not (or should not be) the case. A request over HTTP does not invoke the ask endpoint because no certificate is required because there is no TLS handshake to complete. In other words, this part does not invoke or utilize on-demand TLS at all.

I don't understand. Caddy will check storage in this situation without querying the ask endpoint?
There's a misunderstanding or bug here somewhere, because if I go to http://example.com/.well-known/acme-challenge/foobar in this scenario, the error logs indicate that it's doing something.

No, because only a TLS handshake that actually reaches your server (meaning the DNS is already configured correctly) can trigger On-Demand TLS. Any domain which doesn't point to your server will not renew when the time comes because only a TLS handshake would trigger a renewal attempt.

Customers misconfigure DNS in such a way that it will hit both hosting providers all the time (e.g., adding Hosting Provider B's nameservers without removing A's). There are also a plethora of automated services out there that will continue making requests to the old IP address, because that's just what they do. Most are security and OSINT companies. Some are malicious actors.
In any case, Hosting Provider A is still going to see TLS handshakes  for a long time to come.

@Zenexer

To be clear, the only reason I'm bringing up any of this is because it's made debugging this concurrency issue difficult. If everything worked perfectly as described all the time, it would just be a matter of ignoring the logs--the same as I was already doing.

That's understandable. There's a lot of moving parts (under the hood, at least) that we have to wade through.

It's only a problem because I can't tell which log entries and ask requests are a result of my Caddy instances trying to get certs (or encountering concurrency issues), versus which log entries are from someone else's lazy misuse of ACME.

CertMagic emits logs when it -- and not a third party -- initiates an ACME challenge. Are you referring to log entries for HTTP challenge resources specifically? How are the logs ambiguous? If CertMagic didn't initiate the ACME challenge you won't see log entries that indicate trying to solve challenge. So if you see other logs for ACME challenges but no "trying to solve challenge" logs, then it's a third-party.

By "wasting PKI resources," I'm referring to the fact that DV isn't free, especially at scale. There's an expectation that integrators should make a reasonable effort to ensure a domain actually points to them (and that an ACME challenge will succeed) prior to trying to get a cert.

I see... I suppose you might be using a loose definition of the term "PKI" in that you're talking about CA resources in general, because a failed ACME challenge doesn't allocate any PKI resources (at least, not publicly -- the website still generates a CSR I guess): no certificates, no precertificates, no CT logs, no revocations/CRLs, no OCSP staples, etc.
We've discussed this a lot in the past, including with Let's Encrypt themselves, and with their community, and the overall consensus seems to be that because DNS is a matter of perspective, doing the lookup yourself is seldom helpful. And we've seen from experience (with the lego project, before we diverted from it) that doing DNS lookups oneself got in the way more often than it helped.

I think we're getting a bit caught up with my specific use case, which isn't too relevant here. Let's use the example I provided in a recent comment with two hosting providers.

Fair; but I still care about helping you make your service functional. (We may need to discuss a sponsorship to really dig in and figure out how we can improve the situation for your infrastructure, as it's not a common case at all.) That said, the example with steps 1-14 doesn't seem quite right to me (see last comment).

Correct. I'm not being rate limited and am not proposing that as a concern.
If I follow the instructions in the docs and stop performing DNS checks, then I become Hosting Provider A in my example, and I will get rate limited.

Ah, I see. In that ca,se when a domain is pointed away from a server but the host still thinks it serves that domain, TLS handshakes naturally stop coming in because DNS lookups resolve somewhere else; and CertMagic's On-Demand TLS lets the certificate expire and then it gets deleted. It does not keep trying to renew the cert -- that only happens if TLS handshakes keep coming in (and ask returns 200).

Yes, that was my initial impression. That design strongly disincentivizes ask endpoints from performing sufficient checks. That leads to an ecosystem problem as Caddy becomes more popular.
It invites Caddy users to put themselves in the position of Hosting Provider A above.

Okay, I think I see what you mean with this. But the implementation of On-Demand TLS should account for this as it naturally lets certs expire that are no longer pointing clients at it.

Step 13 is not (or should not be) the case. A request over HTTP does not invoke the ask endpoint because no certificate is required because there is no TLS handshake to complete. In other words, this part does not invoke or utilize on-demand TLS at all.

I don't understand. Caddy will check storage in this situation without querying the ask endpoint?

It checks storage for a challenge token, not for a certificate. This flow has nothing to do with On-Demand TLS.

Customers misconfigure DNS in such a way that it will hit both hosting providers all the time (e.g., adding Hosting Provider B's nameservers without removing A's). There are also a plethora of automated services out there that will continue making requests to the old IP address, because that's just what they do. Most are security and OSINT companies. Some are malicious actors.
In any case, Hosting Provider A is still going to see TLS handshakes for a long time to come.

And this problem is orthogonal to Caddy/CertMagic, and it exists across the Web no matter what server or cert automation you're using. Ultimately the CA has to decide whether the DNS configuration authorizes a certificate for the server, mess or no mess.

We've discussed this a lot in the past, including with Let's Encrypt themselves, and with their community, and the overall consensus seems to be that because DNS is a matter of perspective, doing the lookup yourself is seldom helpful. And we've seen from experience (with the lego project, before we diverted from it) that doing DNS lookups oneself got in the way more often than it helped.

If I'm not mistaken, Let's Encrypt's rate limiting is much stricter for requests that fail DV--that is, you're likely to run into rate limiting if you're requesting a lot of certs and not doing DNS checks on your own.

It checks storage for a challenge token, not for a certificate. This flow has nothing to do with On-Demand TLS.

I've since confirmed this: the ask endpoint won't receive a request.
I think a self-contained example would be helpful here.  It should go without saying, but to anyone coming across this discussion: this example does not use best practices; it's intended to demonstrate a particular issue with minimal code.  Do not use this in production.

Caddyfile
{
	email<EMAIL_ADDRESS>	acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
	default_sni invalid
	on_demand_tls {
		ask http://<IP_ADDRESS>:8002/ask
	}
}

http://<IP_ADDRESS>:8002 {
	log
	route /ask {
		@known query domain=known.example
		respond @known 200
		respond 400
	}
}

http:// {
	log
}

https:// {
	log
	tls {
		on_demand
	}
}

docker-compose.yml
services:
  caddy:
    image: "library/caddy:latest"
    ports:
      - "<IP_ADDRESS>:8080:80"
      - "<IP_ADDRESS>:8443:443"
      - "<IP_ADDRESS>:8443:443/udp"
    volumes:
      - "./Caddyfile:/etc/caddy/Caddyfile"

In my use case, I'm getting a lot of requests to /.well-known/acme-challenge/* from various CAs and vulnerability scanners.  Neither Caddy nor anything else in my infrastructure was trying to get a certificate, so Caddy doesn't have any challenge data.
Command:
% curl -sv --connect-to unknown.example:80:<IP_ADDRESS>:8080 http://unknown.example/.well-known/acme-challenge/foobar
* Connecting to hostname: <IP_ADDRESS>
* Connecting to port: 8080
*   Trying <IP_ADDRESS>:8080...
* Connected to <IP_ADDRESS> (<IP_ADDRESS>) port 8080
> GET /.well-known/acme-challenge/foobar HTTP/1.1
> Host: unknown.example
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 308 Permanent Redirect
< Connection: close
< Location: https://unknown.example/.well-known/acme-challenge/foobar
< Server: Caddy
< Date: Sat, 28 Sep 2024 16:15:44 GMT
< Content-Length: 0
<
* Closing connection

Caddy logs:
% docker compose logs -fn0 caddy
caddy-1  | {"level":"warn","ts":1727540193.833114,"logger":"http","msg":"looking up info for HTTP challenge","host":"unknown.example","remote_addr":"<IP_ADDRESS>:56906","user_agent":"curl/8.7.1","error":"no information found to solve challenge for identifier: unknown.example"}
caddy-1  | {"level":"info","ts":1727540193.8339462,"logger":"http.log.access","msg":"handled request","request":{"remote_ip":"<IP_ADDRESS>","remote_port":"56906","client_ip":"<IP_ADDRESS>","proto":"HTTP/1.1","method":"GET","host":"unknown.example","uri":"/.well-known/acme-challenge/foobar","headers":{"User-Agent":["curl/8.7.1"],"Accept":["*/*"]}},"bytes_read":0,"user_id":"","duration":0.002686333,"size":0,"status":308,"resp_headers":{"Server":["Caddy"],"Connection":["close"],"Location":["https://unknown.example/.well-known/acme-challenge/foobar"],"Content-Type":[]}}

For comparison, curl -sv --connect-to unknown.example:443:<IP_ADDRESS>:8443 https://unknown.example/ will show a query to the ask endpoint, as will curl -sv --connect-to known.example:443:<IP_ADDRESS>:8443 https://known.example/.  The former will end with a TLS alert, while the latter will try to get a cert from LE's staging endpoint.
The most notable log message here is "error":"no information found to solve challenge for identifier: unknown.example".  That's emitted even without the log directive.  While trying to debug #303, I didn't really have any way of determining whether these were the result of concurrency issues, storage issues, or third-parties making bogus requests to /.well-known/acme-challenge/.  It turned out to be mostly the latter.
As you pointed out, ask isn't consulted in this scenario--on-demand TLS doesn't come into play.  I wasn't certain of that at the time.
In my case, there were a combination of issues:

My underlying storage was flaky.  I was using NFS, which should be fine here, but I had incorrect mount options that were preventing locking from working correctly.
Because storage was flaky, a small percentage of these error messages were the result of legitimate ACME attempts, but Caddy just couldn't see the challenge token in storage yet.
Even with that resolved, #303 meant that locking wasn't working correctly.
Even with the fix for #303 applied, there was a lot of corrupt metadata JSON still sitting around.
Caddy was sending a lot of requests to the ask endpoint because certificates weren't being synchronized.  There seemed to be a correlation between the no information found to solve challenge error messages and the ask endpoint requests.  This was probably a coincidence.

I knew there were likely multiple issues, which is why debugging was so complicated.
I do think the PR here resolves #303, although it doesn't fix the already-broken metadata.  My only complaint was that the design decisions surrounding ask made it difficult to gain introspection into Caddy's logic and determine what was actually happening.  In an ideal world without any bugs, the behavior wouldn't really matter.

If I'm not mistaken, Let's Encrypt's rate limiting is much stricter for requests that fail DV--that is, you're likely to run into rate limiting if you're requesting a lot of certs and not doing DNS checks on your own.

Caddy was designed such that it shouldn't be an issue in practice, both because of the internal rate limiting, and the fallback to staging on first failure attempt. See https://caddyserver.com/docs/automatic-https#errors

I've since confirmed this: the ask endpoint won't receive a request. [...]

Thanks for the detailed writeup! I think that aligns with our view of it as well. I agree changes could probably be made for better introspection. What would you think would help in terms of logging etc?

Interesting discussion. I'd like to better understand the consequences of a world full of providers like Hosting Provider A, whereby they only check that their user added the domain and disregard misconfigured DNS, which it sounds like is the vast majority of Caddy users?
Mixing name servers of various providers ("multi-provider DNS"), both accidentally and intentionally, is common enough to warrant a solution for it; the data on its prevalence can be observed across registry zone files, or aggregation tools like DomainTools.com, DNS.Coffee, etc.
Let's assume most of those multi-provider occurrences are by users who have added the domains to each hosting provider that they use (e.g., Wix and Shopify). Perhaps they want to A/B test the two providers without realizing this is the wrong way to do it.
In this example, both Wix and Shopify should fail DV, but they won't realize that if all they're checking is whether a user added the domain to an account with them.
Not a big deal if it's rare, but it's not that rare.

Ultimately the CA has to decide whether the DNS configuration authorizes a certificate for the server, mess or no mess.

@mholt, at what point does the CA decide that too many are failing and that the provider should be taking proactive steps to reduce those failures? What are the consequences of that CA's decision on hosting providers?

Caddy was designed such that it shouldn't be an issue in practice, both because of the internal rate limiting, and the fallback to staging on first failure attempt. See https://caddyserver.com/docs/automatic-https#errors

@francislavoie, what if there are thousands of domains with this issue being visited daily? That's entirely plausible for a medium-sized provider.

What would you think would help in terms of logging etc?

Changing the log level for no information found to solve challenge for identifier to warn or debug would be my best recommendation.  Adding it to the docs would also help, since it's probably not clear what it means to anyone who isn't familiar with ACME.
I don't think it makes sense to query the ask endpoint prior to checking storage.  Requests to /.well-known/acme-challenge/* go directly to storage anyway, so it has to be fast.

Catching up after the weekend...

If I'm not mistaken, Let's Encrypt's rate limiting is much stricter for requests that fail DV--that is, you're likely to run into rate limiting if you're requesting a lot of certs and not doing DNS checks on your own.

Somewhat. From Let's Encrypt Rate Limits:

There is a Failed Validation limit of 5 failures per account, per hostname, per hour.

Note that it's per account, per hostname, (and per hour). So, kind of, but if one person's domain fails to verify, that won't block others.

I've since confirmed this: the ask endpoint won't receive a request.
...
I knew there were likely multiple issues, which is why debugging was so complicated.
I do think the PR here resolves https://github.com/caddyserver/certmagic/issues/303, although it doesn't fix the already-broken metadata. My only complaint was that the design decisions surrounding ask made it difficult to gain introspection into Caddy's logic and determine what was actually happening. In an ideal world without any bugs, the behavior wouldn't really matter.

Thanks so much for investigating this! And thanks for the follow-up. I'll see if we can make things clearer.
@bracketforward  Good questions.

at what point does the CA decide that too many are failing and that the provider should be taking proactive steps to reduce those failures? What are the consequences of that CA's decision on hosting providers?

I think Let's Encrypt's rate limits are a good example of this. They have many rate limits, and if you reach them, you could take steps to reduce those failures. On the other hand, they exist to prevent excessive use of resources, so hitting them isn't "bad" per-se, especially when the domain is truly misconfigured (i.e. trying again won't help). The only time it's not good is when it's a "false positive" like if DNS records have been set but haven't propagated yet (it's not really a "false positive" either, I just don't know a better term).
I've seen LE staff reach out to integrators that are having notable difficulty. But it's always been reasonable IMO.
My browser is crashing so I'll brb to finish this

The internal rate limits are pretty generous now, I think it's 1 per second.

To be exact, 10 per 10 seconds, but effectively the same.

I am having the same issue.
xcaddy build --with github.com/caddyserver/certmagic@16e2e0b

is not working.
FROM caddy:2.8.4-builder AS builder

RUN xcaddy build --with github.com/caddyserver/certmagic=github.com/caddyserver/certmagic@16e2e0b
RUN xcaddy build --with github.com/caddyserver/transform-encoder

FROM caddy:2.8.4

RUN apk update && apk add nss-tools
COPY --from=builder /usr/bin/caddy /usr/bin/caddy
COPY Caddyfile /etc/caddy/Caddyfile

HEALTHCHECK --start-period=5s --timeout=5s CMD wget -q --spider http://localhost/up || exit 1

On curl https://<domain> I get:
<IP_ADDRESS> - - [26/Nov/2024:17:31:41 +0000] "GET <domain>/.well-known/pki-validation/5A556C0F984DDA70D70AD194C4192B47.txt HTTP/1.1" 200 93 0.000229289
{"level":"info","ts":1732642301.309626,"logger":"tls.issuance.zerossl","msg":"served HTTP validation credential","validation_path":"http:/<domain>/.well-known/pki-validation/5A556C0F984DDA70D70AD194C4192B47.txt","challenge":"http-01","remote":"<IP_ADDRESS>:46661","distributed":true}
{"level":"info","ts":1732642304.292847,"logger":"tls.issuance.zerossl","msg":"served HTTP validation credential","validation_path":"http://<domain>/.well-known/pki-validation/5A556C0F984DDA70D70AD194C4192B47.txt","challenge":"http-01","remote":"<IP_ADDRESS>:58892","distributed":true}
<IP_ADDRESS> - - [26/Nov/2024:17:31:44 +0000] "GET <domain>/.well-known/pki-validation/5A556C0F984DDA70D70AD194C4192B47.txt HTTP/1.1" 200 93 0.000245963
{"level":"info","ts":1732642305.123074,"logger":"tls.issuance.zerossl","msg":"validations succeeded; waiting for certificate to be issued","identifiers":["<domain>"],"cert_id":"bee1d239e3dc03215de7ec7f787abb2f","verification_method":"HTTP_CSR_HASH"}

But the certificate is not written to disk.

"validations succeeded; waiting for certificate to be issued"

The CA has not issued the certificate yet; CM is still waiting for it.


⚠ Bofrost.es has degraded performance

In 4e365ca, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1525 ms

Resolved: Bofrost.es performance has improved in d990c1f after 16 minutes.


bijiben: remove webkitgtk override

https://bugzilla.gnome.org/show_bug.cgi?id=728293 is marked fixed in the
bug tracker and a release was made after the fix was committed.

Thanks!


Edf2Mat on Linux

First of all, thanks for a great toolbox!
I just have a few comments if anyone else wants to use it on Linux.

Correct me if I am wrong but the toolbox seems to rely on edf2asc.exe which is a Windows executable. For that to run on linux, one needs wine installed.
Matlab needs to be run without admin privileges (no sudo). This is a safety feature of wine.
The user running the matlab instance needs to have permissions to read and write the edf files.

Then all works fine :) I just thought that it might be useful to have here.

I added the info to the README (#9). Thanks for sharing it!


iOS: White screen after a large API request load

Bug description:

React Native WebView would render a white screen during a large API request.
This behavior is really depending on how much data you are requesting, smaller dataset would be working just fine, only when the data gets larger the problem would appear. Now I know there's two sides of this issue, just wondering if there're some setup that I'm missing
To Reproduce:

Dispatch a request within the website (roughly 7000kb of data), request won't finish and the WebView is dead, rendering a white screen with.
Expected behavior:

Should render as normal
Screenshots/Videos:

2020-06-18 08:39:15.996582-0400 myApp[2928:529484] [shim] BKSProcessAssertion "ConnectionTerminationWatchdog" initialized with invalid pid
2020-06-18 08:39:16.157852-0400 myApp[2928:529484] [ProcessSuspension] 0x10436f450 - ProcessAssertion::processAssertionWasInvalidated()

Environment:

OS: iOS
OS version: 13.5.1
react-native version: 0.61.5
react-native-webview version: 10.3.1

Hi, i got the same problem. Did you find a solution?

Hi, i got the same problem. Did you find a solution?

No unfortunately, had to do some stuff with the api to reduce the load.

Hi, i got the same problem. Did you find a solution?

No unfortunately, had to do some stuff with the api to reduce the load.

@DevArenaCN I'm also having the same issue, in my case I'm loading a url with body (around 15+ params) and with method: "POST", I'm getting white screen, can you tell me how to reduce the load?

Hi @DevArenaCN , I'm able to solve this issue, Here is a workaround that worked for me :
Add below in your source props:
headers: { 'Content-Type': 'application/x-www-form-urlencoded', }
For example:
<WebView
style={styles.WebViewStyle}
source={{
uri: web_url,
method: "POST",
headers: { 'Content-Type': 'application/x-www-form-urlencoded', },
body: "name=test"
}}
/>
Try it and let me know.

@mayursarode4 I am running into the same issue and this solution did not help me. What is the theory behind it being a solution?

I am also interested in a full solution.

I tried the solution indicated by @mayursarode4 but it doesn't work.

I have a similar issue here:
https://stackoverflow.com/questions/67571345/ios-iphone-11-flashing-black-and-white-while-browsler-is-constantly-reloading-r
Did you find any solution to this?

still searching for solutions


Add automatic mode for DAILY_BUILD_ID pass 2

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

[ ] The toolchain has been rebuilt successfully (or no changes were made to it)
[ ] The toolchain/worker package manifests are up-to-date
[ ] Any updated packages successfully build (or no packages were changed)
[ ] Packages depending on static components modified in this PR (Golang, *-static subpackages, etc.) have had their Release tag incremented.
[ ] Package tests (%check section) have been verified with RUN_CHECK=y for existing SPEC files, or added to new SPEC files
[ ] All package sources are available
[ ] cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json, .github/workflows/cgmanifest.json)
[ ] LICENSE-MAP files are up-to-date (./SPECS/LICENSES-AND-NOTICES/data/licenses.json, ./SPECS/LICENSES-AND-NOTICES/LICENSES-MAP.md, ./SPECS/LICENSES-AND-NOTICES/LICENSE-EXCEPTIONS.PHOTON)
[ ] All source files have up-to-date hashes in the *.signatures.json files
[ ] sudo make go-tidy-all and sudo make go-test-coverage pass
[ ] Documentation has been updated to match any changes to the build system
[ ] If you are adding/removing a .spec file that has multiple-versions supported, please add @microsoft/cbl-mariner-multi-package-reviewers team as reviewer (Eg. golang has 2 versions 1.18, 1.21+)
[ ] Ready to merge

Summary

Refresh/Reimplementation of https://github.com/microsoft/azurelinux/pull/7873.
Based on top of https://github.com/microsoft/azurelinux/pull/9856.
Add a new mechanism to support DAILY_BUILD_ID=lkg which automatically updates the toolchain/chroot manifests to match the most recent successful daily build.
The new daily builds will no longer offer the modified azurelinux-repos package with modified repos, so we need to use REPO_LIST=... to pass the new repo file.
From #7873 I've dropped DAILY_BUILD_ID=lkg-force, it just prints an error and tells the user to revert the modified manifests.
Change Log

Support DAILY_BUILD_ID=lkg
Support using REPO_LIST for DAILY_BUILD_ID repo

Does this affect the toolchain?

NO
Test Methodology

Local testing
Testing still in progress

NOTE: Need to take https://github.com/microsoft/azurelinux/pull/9856 first, and re-target this to 3.0-dev.


Publish the windows container image for windows server 2022 (ltsc2022)

The windows container image mcr.microsoft.com/windows:ltsc2022 for Windows Server 2022 (aka ltsc2022 aka 21H1 aka 10.0.20348.169) is missing from https://mcr.microsoft.com/v2/windows/tags/list.
Why wasn't it published like mcr.microsoft.com/windows/nanoserver:ltsc2022 and mcr.microsoft.com/windows/servercore:ltsc2022?
Can it be published?

We announced back in April that moving forward the new Server image: https://techcommunity.microsoft.com/t5/containers/announcing-a-new-windows-server-container-image-preview/ba-p/2304897

Oh I did not notice that. Thank you for pointing it out!


Refactor to Python3 style

Refactor to Python3 style

Thanks so much!  The code looks better now :)
@Gwill


mruby: Make Ruby detectable

Have Ruby default to enabled, only if it's available.
@remi6397 Something like this perhaps? Adding Ruby as a dependency to build TIC-80 is a big ask. While Ruby could be available on many systems, it unfortunately won't be available on ALL systems. Having it fallback could help.
RE: https://github.com/nesbox/TIC-80/pull/1006

While I agree that we should search both for rake and Ruby to increase code readability, I would advise you to consider marking Ruby as required on all compatible platforms, because making it optional will probably result in incompatibility between custom and official builds and discourage potential contributors from improving our MRuby API.
If you still insist on making Ruby a second-class citizen, you should also move associated fallback clauses to the bottom. 😞

I do not mean to make Ruby is not a second class citizen... Ruby is just not available everywhere where TIC-80 is built. If we add Ruby as a dependency to build TIC-80, it will mean breaking compilation for older platforms. I would absolutely love for it to be available everywhere.
I see an old PR to add CMake support, and can understand why they chose to roll with Rake instead of CMake. Wonder if there could be a way to get that up and running again.

If we want to support ancient systems that Ruby isn't available on, everything seems reasonable to me then. 😄
But there are still those fallback clauses in e.g. console.c. Shall we leave them in current order, or rather move them to the bottom-most position?

No objections then? In that case, I'm merging this very patch.

Thanks for the merge, there's certainly a few changes and clean up we could do upon this too, so thanks for adapting.

Shall we leave them in current order, or rather move them to the bottom-most position?

Either order is good :wink:


Remove PKCS12 support

PKCS#12 is awful. It's a very complicated spec that's not inline with modern cryptographic best practices. It's common for implementations to have unsafe cryptographic defaults, even to this day. RC2, DES, MD5, etc. remain in common use! In addition, common implementations don't follow the password string encoding method from the specification.
There's no reason for us to support this.
This requires a major version bump of the mbedtls crate.

Done by https://github.com/fortanix/rust-mbedtls/pull/213


 For instance the .bib entry:
@article{isles_what_1992,
 title = {What evidence is there that 2{\textasciicircum}65536 is a natural number?},
 journal = {Notre Dame Journal of Formal Logic},
 author = {Isles, David}
}

leads to a .bbl entry containing:
  \field{labeltitlesource}{title}
  \field{journaltitle}{Notre Dame Journal of Formal Logic}
  \field{title}{What evidence is there that 2{^}65536 is a natural number?}

which leads to BibLaTeX throwing a series of "Missing $ inserted", "Missing { inserted", "Missing $ inserted" etc errors. This is described in a tex.stackexchange question here, where commentors advised posting this as an issue here (I don't know biber well enough to know whether this is unintended).
The behaviour only occurs when "\textasciicircum" is placed within braces, as "{\textasciicircum}", which is how Zotero outputs it.

Fixed in 2.12 DEV.


escaping single and double quotes in grid

Hello all,
I have successfully followed the serenity examples on how to add a stringformatter on a certain column.
However, the data for this column contains both single and double quotes.
This incorrectly interrupts the string and only shows half of the data.
At which point on the data retrieval do i have to make a certain replace ?
Thanks in advance,
Joris

Q.htmlEncode


Possible security issue: hard-coded password

Greetings,
We are security researchers and we are looking for insecure coding patterns and configurations in the microservice architecture repositories. In your repository, we have found instances of hard-coded passwords. According to CWE, "A hard-coded password typically leads to a significant authentication failure that can be difficult for the system administrator to detect."
Hopefully, you agree and will fix it.
Source: https://github.com/openlab-red/quarkus-mtls-quickstart/blob/master/quarkus-server-mtls/src/main/resources/application.properties and https://github.com/openlab-red/quarkus-mtls-quickstart/blob/master/quarkus-client-mtls/src/main/resources/application.properties

Thanks @akondasif but it's a showcase application not a production grade application. I will make a note on the properties file.
Just for your information the "hard coded" value can be used for local development, and all those properties can be overridden at runtime with an external configuration file.
Where can I open an issue against your bunch of scripts to make it smarter?

@akondasif
Thanks for the feedback.

Where can I open an issue against your bunch of scripts to make them smarter?

As soon as our paper is peer-reviewed we will make our tool open source and let the community to make the tool better :)

Awesome! you should consider to publish under OpenAccess.