<html>
<head>
<style>
.sw_message P{margin:0px;padding:0px;}
.sw_message {FONT-SIZE: 12pt;FONT-FAMILY:Tahoma,Arial,Helvetica,sans-serif;background:white;}
.sw_message blockquote{margin-left:5px;padding-left:5px;border-left:2px solid #144fae;color: #144fae;}
.sw_message blockquote blockquote{border-left:2px solid #006312;color: #006312;}
.sw_message blockquote blockquote blockquote{border-left:2px solid #8e5656;color: #8e5656;}
.sw_message blockquote blockquote blockquote blockquote{border-left:2px solid #888;color: #888;}
</style>
</head>
<body class="sw_message">
<div> Before prompt engineering, first you have to understand how a generative AI LLM works.</div><div>It was trained once, at a fixed point in history, on a specific data set.</div><div>Don't try to ask time based questions, newer than your training date.</div><div><br></div><div>Commercial LLMs offer a hybrid model.</div><div>e.g. Windows copilot gives you Bing, which uses ChatGPT (3.5 or possibly newer now)</div><div>which also falls back to Bing.</div><div><br></div><div>But do, try to carry on a conversation, and argue with it.</div><div>Just remember to reset the conversation when you want to start fresh.</div><div> </div><div id="editor_signature"></div><blockquote type="cite">--- Original message --- <br><b>Subject:</b> Re: [u-u] AI - Llama 2 an open source AI that can run on aRaspberry PI <br><b>From:</b> Giles Orr <gilesorr@gmail.com> <br><b>To:</b> Colin McGregor <colin.mc151@gmail.com> <br><b>Cc:</b> UU <u-u@unixunanimous.org> <br><b>Date:</b> Sunday, 14/01/2024 1:25 PM<br><br>On Wed, 10 Jan 2024 at 21:21, Colin McGregor <colin.mc151@gmail.com> wrote:<br><blockquote type="cite"><br> This is a lightly edited version of a post that I made to another<br> mailing list recently (sorry, not sorry).<br><br> As noted before this evening's meeting I am running the open source<br> (but NOT GPL) Llama 2 AI on a Raspberry PI 5. Llama 2 is a product of<br> Meta (parent company of Facebook), and as you will see in clause 2 of<br> their licence agreement, there is a poison pill clause aimed at the<br> likes of Google (for a hobbyist interested in seeing / playing with a<br> local AI there are no issues) : <a target="_blank" href="https://ai.meta.com/llama/license/">https://ai.meta.com/llama/license/</a> .<br><br> Here is the guide I used for getting Llama 2 up and running on a<br> Raspberry PI (I am using the PI 5, but as the video notes (subject to<br> conditions) it can run on a PI 4) :<br> <a target="_blank" href="https://youtu.be/idZctq7WIq4?si=kekFWQHty60dp2ce">https://youtu.be/idZctq7WIq4?si=kekFWQHty60dp2ce</a><br><br> There is an old saying "The amazing thing about a dancing bear is not<br> how well the bear dances, but that the bear can dance at all.". So,<br> yes, I will to some degree mock Llama 2 on a Raspberry PI in the<br> following, but... It is kind of amazing that you can get a functional,<br> local AI running on a machine as modest as Raspberry Pi, so the flaws<br> are very forgivable.<br><br> I have run Llama 2 on a Raspberry Pi 5 with 4GB of RAM and one with<br> 8GB of RAM, with no obvious difference in performance. At present the<br> key issue appears to be storage I/O performance, to try to attack that<br> I currently have a PCIe to NVMe adapter on order from Pimoroni that<br> will let me use NVMe SSD storage with the PI (I want to see if/how<br> much that helps). I have also tried the 7b and 13b language models on<br> the Pi, the 7b runs (crawls?) faster than the 13b model, but the 13b<br> model gives better (but far from perfect) answers. I am currently<br> normally using the 13b model and accept the slowness. As it is,<br> obviously different questions take different amounts of time, but as<br> things stand, I will ask a question in the evening and see how things<br> are coming the next morning after a good night's sleep.<br><br> I have been keeping a log of the questions / answers I have had with<br> Llama 2 and I will start with a question I asked both the 7b and 13b<br> models. For those who didn't take (or failed) Canadian history in<br> school, let me point out that Lester B. Pearson didn't have any<br> significant role in the Cuban Missile Crisis of the 1960s (but his<br> role in the 1950s Suez Crisis got him a Nobel Peace Prize). Also, Joe<br> Clark was Prime Minister of Canada from June 4, 1979 to March 3, 1980<br> so Pierre Trudeau both preceded and succeeded Joe Clark. These are two<br> of the issues (there are others) in the 7b answer. There is some<br> legitimate room for debate on which Prime Ministers should be on lists<br> like these, but I don't consider any of the names presented as clearly<br> wrong.<br><br> 7b :<br> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-<br><blockquote type="cite"> Who was Canada's best Prime Minister?<br></blockquote> Canada has had several effective and well-respected Prime Ministers<br> throughout its history, making it difficult to pinpoint a single<br> individual as the "best." However, some notable Prime Ministers who<br> have made significant contributions to the country include:<br><br> 1. Sir John A. Macdonald (1867-1873, 1878-1880): Often considered one<br> of Canada's most important Prime Ministers, Macdonald played a crucial<br> role in shaping the country's political and economic systems. He was<br> instrumental in creating the National Policy, which helped establish<br> Canada as a major industrial power, and he also negotiated the<br> Northwest Passage Agreement with the United States, securing Canada's<br> western border.<br> 2. William Lyon Mackenzie King (1921-1930, 1935-1948): King was a<br> skilled diplomat and leader who guided Canada through two major<br> crises: World War I and the Great Depression. He implemented policies<br> that helped stabilize the economy and provided support to those<br> affected by the economic downturn. Additionally, he played a key role<br> in establishing the Canadian Broadcasting Corporation (CBC), which has<br> become an important institution for Canadian culture and identity.<br> 3. Lester B. Pearson (1963-1967): Pearson is best known for his role<br> in resolving the Cuban Missile Crisis, which brought the world closer<br> to nuclear war. He also played a crucial part in establishing the<br> United Nations, for which he was awarded the Nobel Peace Prize in<br> 1957. As Prime Minister, Pearson implemented policies that promoted<br> peace and disarmament, and he worked tirelessly to improve Canada's<br> international reputation.<br> 4. Pierre Trudeau (1968-1984): Trudeau is remembered for his<br> charismatic leadership and his efforts to promote Canadian identity.<br> He implemented a number of progressive policies, including the<br> patriation of the Canadian Constitution, which gave the province of<br> Quebec greater autonomy, and he also legalized same-sex marriage in<br> 1979. Additionally, Trudeau was instrumental in establishing the<br> Canadian Human Rights Commission, which has helped protect the rights<br> of marginalized communities.<br> 5. Jean Chrétien (1993-2003): Chrétien is credited with helping to<br> stabilize Canada's economy and promote international trade. He also<br> played a key role in the Belfast Agreement, which brought an end to<br> the Troubles in Northern Ireland. Additionally, he implemented<br> policies that helped to promote diversity and inclusion within<br> Canadian society.<br> Of course, there are many other notable Prime Ministers in Canadian<br> history, but these five individuals have had particularly significant<br> impacts on the country's political landscape and international<br> reputation.<br> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-<br> 13b<br> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-<br><blockquote type="cite"> Who was Canada's best Prime Minister?<br></blockquote> Determining the "best" Prime Minister of any country is subjective<br> and can depend on various factors such as political affiliation,<br> policy preferences, and the current events that occurred during their<br> tenure. However, here are some of the most highly-regarded Canadian<br> Prime Ministers based on historical assessments and public opinion:<br><br> 1. Pierre Trudeau (1968-1979, 1980-1984): Known for his charisma,<br> progressive policies, and efforts to promote Canadian unity, Trudeau<br> is often considered one of Canada's greatest Prime Ministers. He<br> implemented policies such as official bilingualism, the Canadian<br> Charter of Rights and Freedoms, and the creation of the Canadian Human<br> Rights Commission.<br> 2. John A. Macdonald (1867-1873, 1878-1891): As Canada's first Prime<br> Minister, Macdonald played a crucial role in shaping the country's<br> Confederation and establishing many of its foundational institutions.<br> He is credited with developing the National Policy, which aimed to<br> promote economic growth and unity through tariffs and transportation<br> infrastructure.<br> 3. Lester B. Pearson (1963-1968): Pearson's tenure as Prime Minister<br> saw significant social and political changes in Canada, including the<br> introduction of universal healthcare, the establishment of a new flag,<br> and the devolution of power to Quebec. He was also instrumental in<br> promoting Canadian diplomacy and peacekeeping efforts globally.<br> 4. Wilfrid Laurier (1896-1911): Laurier is often praised for his<br> leadership during Canada's early years, when the country faced<br> challenges such as population growth, economic development, and the<br> need for greater autonomy from Britain. He is credited with promoting<br> French-English relations and expanding the scope of federal powers.<br> 5. Jean Chrétien (1993-2003): Chrétien's tenure saw significant<br> economic growth, the repayment of Canada's national debt, and the<br> negotiation of several major international agreements, including the<br> Free Trade Agreement with the United States. He also played a key role<br> in promoting Canadian unity during the Quebec sovereignty referendum<br> in 1995.<br><br> Of course, these are just a few examples of highly-regarded Prime<br> Ministers in Canadian history. Other notable leaders include William<br> Lyon Mackenzie King, Joe Clark, and Brian Mulroney, among others.<br> Ultimately, the "best" Prime Minister is a matter of personal opinion<br> and depends on one's values, priorities, and perspectives on Canadian<br> history.<br> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-<br> As someone who lives near Yonge & Eglinton, I found the following<br> answer out of the 13b model ALMOST amusing:<br> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-<br><blockquote type="cite"> When will Toronto's Eglinton Crosstown LRT line open?<br></blockquote> The Eglinton Crosstown Light Rail Transit (LRT) line is a major<br> transit project under construction in Toronto, Canada. The project has<br> been underway since 2013 and is expected to be completed in two<br> phases. Here's the current status of the project and when you can<br> expect it to open:<br><br> Phase 1:<br><br> * The first phase of the Eglinton Crosstown LRT line will run from<br> Kennedy Station in the east to Brentcliffe Road in the west, with 25<br> stops along the route.<br> * This phase is expected to be completed and open to the public in spring 2023.<br><br> Phase 2:<br><br> * The second phase of the Eglinton Crosstown LRT line will extend the<br> line from Brentcliffe Road to the Toronto Pearson International<br> Airport, with 15 additional stops.<br> * This phase is expected to be completed and open to the public in the<br> fall of 2024.<br><br> Please note that these dates are subject to change based on various<br> factors, including construction progress and any unforeseen delays.<br> The Toronto Transit Commission (TTC) will provide updates on the<br> project's status and opening dates as more information becomes<br> available.<br> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-<br> Two questions I asked Llama 2 13b related to itself, which I assume to<br> be totally correct:<br> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-<br><blockquote type="cite"> Does Llama 2 learn from past questions?<br></blockquote> Yes, I can learn from past questions! As a large language model, I<br> have the ability to retain information and use it to inform my<br> responses to future questions. This allows me to improve my<br> performance over time and provide more accurate and helpful answers.<br><br> For example, if you ask me a question that I have seen before, I can<br> draw on my past knowledge to provide a more informed response.<br> Additionally, I can use my understanding of language and context to<br> recognize when a question is similar to one I have seen before, and<br> adapt my response accordingly.<br><br> However, it's important to note that my ability to learn from past<br> questions is limited by the data that I have been trained on. While I<br> have been trained on a vast amount of text data, there may be certain<br> topics or questions that I am not familiar with or cannot answer<br> accurately. In these cases, I will do my best to provide a helpful<br> response based on the information that I do have.<br><br><blockquote type="cite"> Do you access the internet when answering questions?<br></blockquote> Yes, I do access the internet when answering questions! As a large<br> language model, my primary function is to assist with tasks and answer<br> questions to the best of my ability. To do this, I rely on a vast<br> amount of text data that I have been trained on, as well as the<br> ability to access the internet for more up-to-date information.<br><br> When you ask me a question, I use a combination of natural language<br> processing (NLP) and machine learning algorithms to understand the<br> context and intent behind your question. I then draw on my training<br> data and any relevant information available online to provide an<br> accurate and helpful response.<br><br> However, it's important to note that while I can access the internet<br> for information, I do not have the ability to browse or search the web<br> in the same way that a human would. My primary focus is on providing<br> answers to specific questions based on the information that I have<br> been trained on and what is available online.<br> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-<br> I find the Llama 2 answers to be interesting enough to keep the<br> program around, but I wouldn't use the answers it provides without<br> some extra checking / verification.<br><br> Any event, I could go on, but... Any questions for me or Llama 2?<br></blockquote><br><br>According to 7b, Pierre Trudeau "... also legalized same-sex marriage<br>in 1979." I lived through that, and I definitely don't remember it.<br>Mostly because it didn't happen. If you're curious, see<br><a target="_blank" href="https://en.wikipedia.org/wiki/Same-sex_marriage_in_Canada">https://en.wikipedia.org/wiki/Same-sex_marriage_in_Canada</a> - Wikipedia<br>is far from perfect, but I think it's more accurate on that subject.<br>I notice that the 13b version fixed that particular problem, but I'm<br>pretty sure it only switched it out for more subtly incorrect answers.<br>This is something that really alarms me about AI right now: you have<br>to fact-check it like crazy to get good answers, and 99% of the users<br>will instead take the answers as gospel.<br><br>My apologies: I'm stepping over from the computing aspects of this<br>into the philosophical ... but I do find it quite alarming. And this<br>is before we get into the wonderful potential for _deliberate_ bias,<br>which will undoubtedly be exploited as we progress with AI.<br><br>-- <br>Giles<br><a target="_blank" href="https://www.gilesorr.com/">https://www.gilesorr.com/</a><br>gilesorr@gmail.com<br>_______________________________________________<br>u-u mailing list<br>u-u@mail-ml.infra-service.ca<br><a target="_blank" href="http://www2.infra-service.ca/mailman/listinfo/u-u">http://www2.infra-service.ca/mailman/listinfo/u-u</a><br></blockquote><br>
</body></html>